ERR_UNKNOWN_FILE_EXTENSION when using ts-node-esm

[kylerush]

First off just wanted to say that I love this package. It's so simple and straightforward. Thank you to everyone who has worked on it 🙏

I am attempting to migrate my current TypeScript/CJS project to TypeScript/ESM and I'm having some issues with this package. I am getting this error:

ts-node-esm ./node_modules/.bin/node-pg-migrate -j ts up

/Users/kylerush/projects/backend/node_modules/ts-node/dist-raw/node-internal-modules-esm-get_format.js:93
        throw new ERR_UNKNOWN_FILE_EXTENSION(ext, fileURLToPath(url));
              ^
CustomError: ERR_UNKNOWN_FILE_EXTENSION  /Users/kylerush/projects/backend/node_modules/node-pg-migrate/bin/node-pg-migrate
    at defaultGetFormat (/Users/kylerush/projects/backend/node_modules/ts-node/dist-raw/node-internal-modules-esm-get_format.js:93:15)
    at defer (/Users/kylerush/projects/backend/node_modules/ts-node/src/esm.ts:296:7)
    at entrypointFallback (/Users/kylerush/projects/backend/node_modules/ts-node/src/esm.ts:304:22)
    at getFormat (/Users/kylerush/projects/backend/node_modules/ts-node/src/esm.ts:338:26)
    at /Users/kylerush/projects/backend/node_modules/ts-node/src/esm.ts:245:17
    at addShortCircuitFlag (/Users/kylerush/projects/backend/node_modules/ts-node/src/esm.ts:409:21)
    at load (/Users/kylerush/projects/backend/node_modules/ts-node/src/esm.ts:239:12)
    at load (/Users/kylerush/projects/backend/node_modules/ts-node/src/child/child-loader.ts:18:36)
    at nextLoad (node:internal/modules/esm/loader:163:28)
    at ESMLoader.load (node:internal/modules/esm/loader:602:26)

I have this in my package.json:

"migrate": "ts-node-esm ./node_modules/.bin/node-pg-migrate -j ts --ignore-pattern *.md"

Any idea on something I can do on my end to get the migration to run?

[Shinigami92]

Please try out https://github.com/salsita/node-pg-migrate/releases/tag/v8.0.0-rc.0
Maybe you need to use node --experimental-specifier-resolution=node --loader ts-node/esm

node-pg-migrate/.github/workflows/postgres-test.yml

Line 725 in 0d510a2

run: node --experimental-specifier-resolution=node --loader ts-node/esm test/ts/customRunnerDBUrl.ts

[nealoke]

I too love this package a lot have a similar issue when trying to get up and running with ts files for migrations. Specifying my tsconfig doesn't work either, nor does calling it like ts-node node_modules/.bin/node-pg-migrate -j ts

TS (not working)

> dotenv -e ./../../.env node-pg-migrate -- -t db_migration -j ts up

Error: Can't get migration files: TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /usr/src/app/packages/api/migrations/1658847592963_0.ts

TS specifying tsconfig (not working)

> dotenv -e ./../../.env node-pg-migrate -- -t db_migration -j ts -tsconfig ./tsconfig.json up
0.0.1 <-- I'm not sure where this comes from

JS (working)
> dotenv -e ./../../.env node-pg-migrate -- -t db_migration up

FYI I'm on "node-pg-migrate": "8.0.1"

[brenoepics]

I noticed this yesterday while working on #1425, I'll take a look to fix it in a few hours.

[Shinigami92]

I noticed this yesterday while working on #1425, I'll take a look to fix it in a few hours.

Just an idea without checking what is required for that, but we could have a look into jiti 👀
eslint and tools like that uses jiti

[nealoke]

@brenoepics I take it you weren't able to look at this yet, I tried looking for an issue in this repo but wasn't able to find it (I'm not really familiar with it). Would be great if you could have a look 🙏

[brenoepics]

I've been experimenting with Jiti, it's really interesting. It allows you to run TypeScript files without needing ts-node or a bunch of other tooling. (I managed to solve this issue with jiti and I don't even need ts-node etc.)

That said, I think a lot of the issues people encounter when using ts-node are probably due to misconfigurations on the user's side.

I've noticed some quirks that depends on a lot of things:

• With pnpm, things behave one way; with npm, it's different.
• Sometimes, it's just that the user forgot to pass --tsconfig tsconfig.json.
• Other times, it is related to esm resolution or configs like needing to run something like:
  ts-node --esm node_modules/.bin/node-pg-migrate

There's also that strange behavior I reported in #1425 with pnpm.

So here's what I'm wondering:

@Shinigami92 Do you think we should always use Jiti, or maybe make it optional through a CLI flag, similar to what we already do with config, dotenv, and ts-node?

And if we do go with Jiti (either always or optionally), what should we do with ts-node? Should we deprecate it? Keep it as a fallback? Maybe we just improve the docs to help users avoid these pitfalls?

Also, Node.js will soon support type stripping natively.

node-pg-migrate/src/runner.ts

Lines 185 to 187 in 68069b0

	extname(filePath) === '.sql'
	? await migrateSqlFile(filePath)
	: await import(`file://${filePath}`);

[brenoepics]

Also, thinking about #1390 and #1424, we should add integration tests for ts-node and related logic. I've been considering cleaning up the open TypeScript-related issues as well.

[brenoepics]

> dotenv -e ./../../.env node-pg-migrate -- -t db_migration -j ts -tsconfig ./tsconfig.json up
0.0.1 <-- I'm not sure where this comes from

JS (working) > dotenv -e ./../../.env node-pg-migrate -- -t db_migration up

FYI I'm on "node-pg-migrate": "8.0.1"

This 0.0.1 comes from your own package version.

Note that -tsconfig is not valid, use --tsconfig instead
I could make it work using this command:

But why this happens? actually you found something funny.

1. When you use -tsconfi, yargs is likely parsing this incorrectly
2. In yargs, multiple single-dash options can be combined (like -abc for -a -b -c)
3. So -tsconfig could be interpreted as -t -s -c -o -n -f -i -g

Therefore, when you use any parameter that contains or starts with -tsconf**i**, yargs might be interpreting it as including the version flag and outputting the package version. -i

When I tried -tsconfj ./tsconfig.json I've got

Invalid values:
  Argument: migration-file-language, Given: "./tsconfig.json", Choices: "js", "ts", "sql"

If you fix it, it will probably work if you don't have any of the issues I mentioned in the first comment.

[Shinigami92]

@brenoepics 🚀

IMO we can go full jiti (not just deprecate ts-node and tsx support, but fully remove it and release a new major)
jiti is in use with tools like eslint nowadays anyways and so potentially the by far better option

node's stripping mode wont help in this case, as we need to import dynamically at runtime .ts files

[brenoepics]

Jiti also supports transform for babel and tryNative is also great because it allows compatibility with the Bun runtime without requiring any changes.

Would it make sense to create a new branch from main for the upcoming major release? I can open separate PRs for each change, such as adding Jiti, integrating Commander, and applying other fixes, to avoid disrupting the current major version.

[Shinigami92]

Would it make sense to create a new branch from main for the upcoming major release? I can open separate PRs for each change, such as adding Jiti, integrating Commander, and applying other fixes, to avoid disrupting the current major version.

IMO start with higher priorities first and lets not do to much in one major version

Start with jiti, commander is nothing important right now, where as jiti and removing ts-node/tsx support is helping the community directly by a improved DX

I have no problems with doing a v10, v11, v12 and so on