Why is typedoc so slow on Windows?

[fr0]

• I have checked issues with bug label and found no duplicates

Expected Behavior

I'd like typedoc to finish in less than 30 seconds.

Actual Behavior

I have a repo with about 180 TypeScript files, and typedoc takes about 7 seconds on my iMac Pro, and... more than 6 minutes on my WIndows 10 machine.

Both are 64-bit machines, and both have way more RAM than typedoc is using. Also both have SSD hard drives and multiple cores.

Steps to reproduce the bug

I wish I could share my repo here, but unfortunately it is private. Maybe try any repo with more than 100 .ts files, and compare on Windows vs. other operating systems.

Environment

• Typedoc version: 0.14.2
• Node.js version: 10.15.0
• OS: Windows 10 Enterprise 64-bit

[Gerrit0]

Could you provide your TypeDoc configuration? I suspect something odd is going on with the exclude option, but includeDeclarations can also cause a performance hit.

[fr0]

Thanks for the quick response. Here's my entire tsconfig.json:

{
  "compileOnSave": false,
  "compilerOptions": {
    "outDir": "./dist/out-tsc",
    "baseUrl": "src",
    "sourceMap": true,
    "declaration": true,
    "module": "es2015",
    "moduleResolution": "node",
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "alwaysStrict": true,
    "downlevelIteration": true,
    "skipLibCheck": true,
    "target": "es6",
    "typeRoots": [
      "node_modules/@types"
    ],
    "lib": [
      "es2017",
      "esnext.array",
      "dom",
      "dom.iterable"
    ]
  },
  "include": [
    "src/**/*.ts",
    "node_modules/cypress",
    "cypress/**/*.ts"
  ],
  "typedocOptions": {
    "out": "./docs/",
    "name": "atlas-dev-tools",
    "excludePrivate": true,
    "exclude": "**/*.*spec.ts",
    "excludeNotExported": true,
    "target": "ES6",
    "mode": "file"
  }
}

I invoke it via yarn run typedoc src/app/.

[aciccarello]

I'm curious what the difference in time is for an initial typescript compile between windows/mac. Is it comparable?

[resynth1943]

Yeah mine takes about a whole minute generating:

typedoc --out ./docs/ ./index.d.ts --includeDeclarations --theme minimal --ignoreCompilerErrors --mode modules --exclude ./node_modules/**/*

Is this a feature or a bug? 😛

[Gerrit0]

Definitely a bug, just one that is unfortunately incredibly difficult to track down. I'm still not sure why it got so slow when just upgrading the TS version. The investigation is being tracked in #975

[Gerrit0]

If anyone is having issues with typedoc taking significantly longer to run than typescript takes to compile, please open a new issue with a reproduction. TypeDoc works pretty well, with the vast majority of the run time spent in the compiler in my experiments recently.

[xmo-odoo]

@Gerrit0 I've a situation where typedoc takes a very long time, but that's trying to build documentation from a .d.ts so typescript compilation is not really a thing I think? Is there a way to get the runtime information easily?

Trying to --logLevel Verbose I get the following:

Debug: Using TypeScript 4.4.4 from ./node_modules/typescript/lib
Debug: Converting with 1 programs 1 entry points
Debug: Finished getting entry points in 2656ms
Debug: Begin readme search at ./packages/openapi-types
Debug: Finished conversion in 11168ms
Debug: Renderer: Loading highlighter took 752ms

and then it just pegs a core and logs nothing more (for at least 10 minutes).

This is trying to build the documentation for https://github.com/octokit/openapi-types.ts which is a bit of a monster (the entry point is 1.5MB), it doesn't use typedoc by default but I added the following tsconfig.json to make typedoc do something:

{
    "compilerOptions": {
        "strict": true
    },
    "typedocOptions": {
        "entryPoints": ["packages/openapi-types/types.d.ts"],
        "out": "docs",
        "readme": "none",
    }
}

this may not be correct at all though.

edit: ended up killing the process after

4957.74s user 3.66s system 99% cpu 1:22:42.55 total

it never logged anything after what's earlier.

[Gerrit0]

Please open a new issue rather than reviving one that's been closed for ages

[xmo-odoo]

Sure.

[pjeby]

This is mainly a note for future searchers trying to find why their Windows doc build is slow...

If your process takes minutes to run on Windows, and you're using Cygwin to run it (and/or a VSCode cygwin shell), but there's very little CPU or I/O showing for node in resource monitor, try adding disableGit: true to your configuration, or adjusting your PATH to point to a Windows or MSYS git.

The issue is that typedoc runs git over and over and over again with spawnSync, which on other operating systems is fast, but on Wiindows is slow. It's not helped by the fact that it appeared to have no caching or batching whatsoever -- it was rerunning the same 4 calls to git over and over and over.

The root issue was that typedoc created a "repository" for every source reference, instead of caching one per file path. On further investigation, I found that this was because the paths typedoc was sending to git were of the form c:/cygwin/home/blah... and the paths cygwin git was returning were of the form /home/blah.... So in typedoc's SourcePlugin, it was cache-missing every time for the repo path and creating a new one, causing git to be called over and over.

So the workaround is to use --disableGit when running from Cygwin, or to tweak PATH while running typedoc such that it sees Windows' git rather than Cygwin. (You can do this from typedoc.config.cjs in a pinch.)

It would be nice if tryCreateRepository were to detect that topLevel.output isn't a prefix of the input path and if so, prefix it accordingly before creating the repository. This would make the caching compatible with Cygwin git.

(Alternately, having a way to specify a path to the correct Git instance to use would be helpful, as I could make my config.cjs file just set that based on the environment rather than hacking PATH.)

But since my workaround is just two lines in .config.cjs and another in my .envrc, it's not a huge deal if neither of these get changed, especially since Cygwin users are probably a bit rare these days.