Error including `bytemd` dependency

[benmccann]

Describe the bug

Someone on Discord reported that they can't get bytemd to work. It looks to me like vite-plugin-svelte is not detecting that bytemd's lodash.debounce dependency needs to be prebundled. There are also issues reported in the bytemd repo and on Discord where the error occurs on other dependencies

A couple things I noticed that are off about the packaging:

• bytemd uses .esm.js and .cjs.js extensions instead of .mjs and .cjs
• lodash.debounce has no main or any other entry point specified in the package.json

Even when I manually edit those I still get the same error.

Reproduction

https://github.com/benmccann/sveltekit-bytemd-bug-reproduction

Logs

8:07:24 PM [vite] Error when evaluating SSR module /node_modules/lodash.debounce/index.js?v=dc88d06f:
ReferenceError: module is not defined
    at /node_modules/lodash.debounce/index.js?v=dc88d06f:377:1

System Info

reproduction provided

Severity

The top two bugs in the bytemd repo are about how the project doesn't work with SvelteKit: https://github.com/bytedance/bytemd/issues/96 and https://github.com/bytedance/bytemd/issues/139

[benmccann]

I don't think this is actually fixed unfortunately. I just upgraded to the latest and am still seeing the same error

[bluwy]

My bad. I assumed this was a pre-bundling bug and only fixed that, looks like the error is coming from loading the module in SSR 🤔

[bluwy]

TL;DR: ssr.noExternal on a dep considers the dep to be source code (Everything is ESM including its transitive deps, like pre-bundling)

---

Wow this seems like a huge deal than I expected. Through some testing, I realized the issue is that ssrLoadModule is called on lodash.debounce, but that doesn't work because ssrLoadModule works for ESM code only. Ideally Vite expects this to never happen in the first place.

For our case, it happens because ssrLoadModule is called by ssrImport, which happens when you do import debounce from "lodash.debounce" in your code. And in this case, bytemd's is doing so.

However, this isn't an issue in most cases, as long as lodash.debounce is externalized, which Vite does so by default. When it is externalized, Vite's import analysis will keep the import as is, so you get this in Vite's complex SSR transformation:

const debounce = __vite_ssr_import__("lodash.debounce")

If not externalized, you get:

const debounce = __vite_ssr_import__("/node_modules/lodash.debounce/index.js")
// notice this is the path of the error message of the issue

The different path is important as it gets resolved differently by ssrImport.

In Svelte's case, this is not externalized because bytemd is Svelte library, and it's noExternal by default, which means any deps below it is not externalized, including lodash.debounce.

A solution is to externalize lodash.debounce explicitly, but bytemd has a lot more CJS-only deps that needs to be externalized. vite-plugin-svelte could use the same deps reinclusion logic from prebundling for here as well to automate this, but I think for dev only. And I'm not sure how safe it'll work in pnpm.

Or maybe adjustments can be made in Vite to externalized transitive deps by default.

[dominikg]

replacing the unmaintained lodash.debounce with https://github.com/angus-c/just#just-debounce-it might be a quickfix for bytemd

[bluwy]

There are a lot more deps there that has issues, like lodash.throttle, unified, and the remark plugins IIRC.

[dominikg]

For lodash.throttle there is just-throttle too. but remark plugins don't always have an alternative that works with esm too i guess.

Looks like bytemd is actively working on their package structure, adding an exports field (see history from today) https://github.com/bytedance/bytemd/commits/main/packages/bytemd/package.json

[benmccann]

I don't understand why we're only hitting this with this library. I thought there are lots of Svelte libraries with CJS deps that work.

[benmccann]

I put a console.log statement at the end of resolveSSRExternal and found:

externals: ["@sveltejs/adapter-auto","@sveltejs/kit","svelte","@bytemd/plugin-gfm","bytemd > codemirror-ssr","bytemd > hast-util-sanitize","bytemd > lodash.debounce","bytemd > lodash.throttle","bytemd > rehype-raw","bytemd > rehype-sanitize","bytemd > rehype-stringify","bytemd > remark-parse","bytemd > remark-rehype","bytemd > unified","bytemd > unist-util-visit","bytemd > vfile","bytemd > word-count"]

But @bluwy said it wasn't being externalized, so I think perhaps something is going wrong in the Vite code in between where I was looking and where he was looking since I do see it being externalized

[bluwy]

I don't understand why we're only hitting this with this library. I thought there are lots of Svelte libraries with CJS deps that work.

Me too. This looks like something we would have spotted long ago, maybe I'm missing something that's only an issue with bytemd specifically, but it looks to me that it's a general issue 🤔

I put a console.log statement at the end of resolveSSRExternal and found:

I tried the same and got [ '@sveltejs/adapter-auto', '@sveltejs/kit', 'svelte', '@bytemd/plugin-gfm' ] though. And I don't think the > syntax works with SSR externals. It would be a bug if there's a way to reproduce that.

I'll try to reproduce this in vite-plugin-svelte again tonight and see if it's a general issue not specific to bytemd

[benmccann]

I tried the same and got [ '@sveltejs/adapter-auto', '@sveltejs/kit', 'svelte', '@bytemd/plugin-gfm' ] though. And I don't think the > syntax works with SSR externals. It would be a bug if there's a way to reproduce that.

Oh, I think that's because I accidentally had checked in experimental.prebundleSvelteLibraries: true. I just pushed a change to remove that (benmccann/sveltekit-bytemd-bug-reproduction@9d661d7). Hopefully you'll see the same as I do now