Should dev-server also serve node_modules packages?

[yohcop]

I don't have a simple example on hand, but I can make one if needed.
It seems that the dev_server packages requirejs and then all of my modules fine.

But then when I do require(["__main__/my/index"]), this tries to pull a ton of other dependencies, that are normally under node_modules, by making many requests to the dev_server. It wouldn't be much of an issue if my project used one or two packages, I could include them manually in my index.html file, but with tens of dependencies, it is a challenge.

What is the recommended way of doing this? I can try to maintain a genrule that concat together the files my project needs, output a node_deps.js file and have a <scritp src="node_deps.js">. I would also need to wrap things correctly for require js actually.

Is it in the plans that the dev server would also pack and serve files from node_modules?

[alexeagle]

Such dependencies can't be bundled into the app, because our bundle format is AMD. They should be served as a separate vendor script (or scripts)

@thelhevold has worked on this and documented one example in http://www.syntaxsuccess.com/viewarticle/angular-bazel-build-with-dev-server - using SystemJS as the runtime loader to fetch these third party libraries.

Indeed, we need to document how this is done.

[appsforartists]

From what I've gathered skimming through angular-bazel-example and @thelgevold's example, bundling/serving external dependencies with the devserver is currently left as an exercise for the developer, especially if the node_module being depended on doesn't publish an AMD version.

This is unfortunate. I expect a build tool to identify my dependencies, optimize them, and bundle them together. I expect a devserver to interoperate with the same flow as a build tool (bundling them and serving them to the same URL, though probably without optimizations). I know the open source TypeScript rules are relatively new and a work-in-progress - I'm hoping these devserver limitations are merely a reflection of that, not an intentional design decision.

Since TypeScript is a superset of JavaScript, I suspect we can pass ES2015 modules through the current toolchain. Imagine something like this:

es2015_module(
  name = "external-lib",
  node_modules = "//:node_modules",
)

# which could generate something like 

ts_library(
  module_name = "external-lib",
  module_root = "//node_modules/external-lib/",
  srcs = ["//node_modules/external-lib/**.js", "//node_modules/external-lib/**.ts"],
  tsconfig = "really-loose-tsconfig-that-allows-implicit-any"
)

We already have a toolchain that consumes ts_librarys, combines them, and serves them as AMD for ts_devserver. This would extend that toolchain to also work with external libraries. I suspect you could even write another layer on top of this one that allows you to use package.json as the source of truth for JS dependencies and generates the es2015_module rules for BUILD.

It would be fantastic to have a tool that allows a developer to write TypeScript, import external modules from NPM, and have it all bundled for development & unit testing, + tree-shaken/minified for production. I'm excited for Bazel because configuring all the tools in the JS ecosystem to work together is a painful distraction, and I'd rather spend that time writing product code.

I have no idea if there are already better plans for node_modules support than what I've outlined above. In any case, I hope this feedback is helpful - thanks for all the effort you've been investing into this.

[alexeagle]

Short answer: yes, we need to support third-party libraries.

Longer answer: working with third-party libraries is much harder than sources we control. It will require more user configuration, just because there is so much diversity - for example, do you need to tell the toolchain where the typings are? How to translate import statements to the global symbol at runtime? Whether the library needs some loader shim and module transpilation at runtime? I think we won't be able to make a non-leaky abstraction where you just depend on a third-party library and everything just works - there will probably be several ways and some ways work with some libraries. It will be a challenge to document. And it needs design thought.

[alexeagle]

I think we need:

• example/e2e test for a couple representative/popular third-party libs
• should be served by devserver (probably need a systemjs setup since they can't be concat'ed into the bundle)
• should ensure this doesn't degrade the dev RTT
• should be packaged into the rollup_bundle for production
• Update markdown doc
• add them to alexeagle/angular-bazel-example as well

[solomatov]

Are there any updates on this? I want to use libraries, and it's hard to find how to do it correctly.

[alexeagle]

This has gotten a lot better - @gregmagolan has improved the automatic translation we do on node_modules so it's more similar to how other bundlers work. It still requires manually adding @npm//foo:foo__umd target to your deps, but in theory this can be automated too.

Keeping this open for the missing documentation.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity for 90 days. It will be closed if no further activity occurs in two weeks. Collaborators can add a "cleanup" or "need: discussion" label to keep it open indefinitely. Thanks for your contributions to rules_nodejs!

[github-actions]

This issue was automatically closed because it went two weeks without a reply since it was labeled "Can Close?"