Export SourceFileObject and friends

[kwasimensah]

🔍 Search Terms

AST parse SourceFileObject

✅ Viability Checklist

• This wouldn't be a breaking change in existing TypeScript/JavaScript code
• This wouldn't change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
• This isn't a request to add a new utility type: https://github.com/microsoft/TypeScript/wiki/No-New-Utility-Types
• This feature would agree with the rest of our Design Goals: https://github.com/Microsoft/TypeScript/wiki/TypeScript-Design-Goals

⭐ Suggestion

I'm following the suggestion at #3662 (comment) but the prototypes to patch aren't exposed (i.e. https://github.com/microsoft/TypeScript/blob/main/src/services/services.ts#L1003).

I'm happy to hack around this for my testing but wanted to see if there's a path to exposing this publicly.

Related Issues
#26871
#30235
#54256

📃 Motivating Example

Allowing external build tools that run tsc invocations in parallel to be faster by not repeating the source file parse.

💻 Use Cases

1. What do you want to use this for?
   I'm working on a build system and would like to make parallel invocations of tsc faster by not having to reparse declarations coming from shared dependencies.
2. What shortcomings exist with current approaches?
   Commonly used declaration files like @types/node get reparsed multiple time when splitting tsc compilation across multiple invocations.
   Ideally we'd allow each of the invocations to live in their own sandbox. So we wouldn't rely on a shared language service to keep the parsed files in memory.
3. What workarounds are you using in the meantime?
   Living with multiple calls to tsc repeating work (especially for the expensive node types)/relying on managing typescript services

[jakebailey]

I doubt that we'd want to export these internal objects just so someone can patch the prototype. They are an implementation detail which may change; exporting them closes the door to improvements behind the scene (and we have at least two different performance related changes in the works that would modify them or remove them entirely).

Can you be more specific as to what sort of "patching" you are trying to do? How does being able to touch these objects relate to your use cases?

[kwasimensah]

Debugging it more, it was more an issue with SourceFile.pragmas being a Map and getting errors about functions like get not being defined properties and getting TypeErrors. I special case that in what I'm testing. There's also these functions that are also part of the public interface https://github.com/microsoft/TypeScript/blob/main/src/services/types.ts#L159 but they haven't been an issue so far since I'm not using services.

I guess a more generic ask is making sure that if we reserialize the JSON-able parts of SourceFile, that there aren't any properties or methods that would be called on it (and objects it contains) that are dependent on it having a non-Object prototype

[kwasimensah]

Note, this does already seem to be the case (outside of the Services and Map use case I pointed out above) but I wanted to make sure this is a constraint the Typescript team is interested in maintaining especially give the compiler API is still marked as not stable and if it made sense for the Typescript team to own serialize/deserialize (#26871) if this is not the case

[jakebailey]

The SourceFiles are not serializable, correct; it's not totally clear that we'd add an API to do that without also making use of that API internally (e.g. the parallelism change you linked, except that the prototype for that does not serialize, it uses shared structs).

Is doing what you need impossible externally? There's a lot of internal stuff on loads of objects. I would almost imagine it'd be easier to fix those up externally (and use ts-expose-internals) or something, but I can understand not wanting to do that.

Right now, we really only ensure diagnostics are serializable, as that is needed for monaco and so on.

[kwasimensah]

I'm still working on this. Got sidetracked with work from my day job and I'll get back to you on if I can use ts-expose-internals

[jakebailey]

#58045 and other PRs are messing with these classes, and we are also wanting to do away with them entirely by moving them into the compiler. I'm not super inclined to export these, at least not until we do away with the "allocator" construct entirely.