chore: reorganize prod dependencies

[regisb]

This is to allow npm install --omit=dev across all MFE apps.

[openedx-webhooks]

Thanks for the pull request, @regisb!

This repository is currently maintained by @openedx/committers-frontend.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

• If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
  • This process (including the steps you'll need to take) is documented here.
• If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

• Dependencies

  This PR must be merged before / after / at the same time as ...

• Blockers

  This PR is waiting for OEP-1234 to be accepted.

• Timeline information

  This PR must be merged by XX date because ...

• Partner information

  This is for a course on edx.org.

• Supporting documentation
• Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

---

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

• Overview of Review Process for Community Contributions
• Pull Request Status Guide
• Making changes to your pull request

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

• The size and impact of the changes that it introduces
• The need for product review
• Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.71%. Comparing base (e0656d1) to head (0a466be).

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #787   +/-   ##
=======================================
  Coverage   86.71%   86.71%           
=======================================
  Files          48       48           
  Lines        1393     1393           
  Branches      293      293           
=======================================
  Hits         1208     1208           
  Misses        172      172           
  Partials       13       13           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[brian-smith-tcril]

I think I need more context to understand why npm install --omit=dev isn't currently working for you.

If this PR landed as it stands now it would definitely break things. I'd like to understand the intent before trying to figure out a path forward for this that won't break things.

[bradenmacdonald]

@brian-smith-tcril What will this break? If it were going the other way I can see it breaking stuff, but how would moving devDeps into deps break anything?

npm install --omit=dev should install everything required to build the MFE, so the status quo is clearly wrong. We need react and paragon to build the MFE. devDependencies should be test libraries, linters, code formatters, etc.

[brian-smith-tcril]

how would moving devDeps into deps break anything?

Because we currently support MFEs using multiple versions of some of the dependencies being moved via peer deps. Specifically Paragon 22 and 23, and React 17 and 18.

[bradenmacdonald]

@brian-smith-tcril Then shouldn't we be specifying the dependency here as e.g. Paragon ^22 || ^23 ? Or if this repo really only works with Paragon 23 and not 22, then I guess I don't understand something about how that peerDeps approach is working.

[brian-smith-tcril]

@bradenmacdonald by specifying the version ranges in peerDependencies we're saying we expect the consumers of frontend-platform (the MFEs themselves) to have one of those versions included as a direct dependency. We then have a single version of each added in devDependencies so the tests have a version installed when running.

I know it is possible to set version ranges that span multiple major versions in direct dependencies for libraries, but it is my understanding that the peerDependency strategy is much more common for situations like ours.

This brings me back to my original question. Why/where is npm install --omit=dev not working?

Of the packages this PR is moving from devDependencies to dependencies

   "@openedx/frontend-build": "^14.3.0",
   "@openedx/paragon": "^23.3.0",
   "react": "18.3.1",
   "react-dom": "18.3.1",
   "react-redux": "^8.1.1",
   "react-router-dom": "^6.6.1",
   "redux": "4.2.1",

already exist as peerDependencies, meaning running npm install --omit=dev on a consuming MFE will already result in those being installed.

The other 2 being moved are

    "@edx/brand": "npm:@openedx/brand-openedx@^1.2.2",
    "@edx/browserslist-config": "1.5.0",

If those are the problematic ones then I could see either adding them as peerDependencies if we need some version flexibility, or possibly adding them as direct dependencies.

[regisb]

I was completely ignorant of peerDependencies before reading your comment, so thanks for taking the time to explain :) Yet, your explanations (or the ones I found here) do not match my experience:

1. npm ci: installs @openedx/frontend-build) (OK)
2. npm ci --omit=dev: does not install @openedx/frontend-build) (KO)
3. npm ci --omit=dev --include=peer: does not install @openedx/frontend-build) (KO)

To reproduce, run:

$ git clone https://github.com/openedx/frontend-platform/
$ cd frontend-platform
$ npm ci --omit=dev
$ npm run build
...
/bin/sh: 1: ./node_modules/.bin/fedx-scripts: not found

I'm using:

$ node --version
v20.18.3
$ node --version
v20.18.3

My reasoning when I opened this PR was:

1. frontend-build is missing, so let's move it to dependencies
2. browserlist-config is missing, so let's move it to dependencies
3. I'll use the opportunity to move all the things that look like build dependencies

It's quite possible that I went overboard with step 3, in which case I'm ready to amend this PR.

[brian-smith-tcril]

@regisb why are you running npm ci on frontend-platform directly as opposed to running npm ci --omit=dev in an MFE that has frontend-platform as a dependency?

[regisb]

I believe I was testing frontend-platform, because I figured that I would have to fix frontend-platform's devDependencies before I could work on MFE apps. Isn't that a valid test case?

It looks like frontend-build is correctly installed in MFE apps, when they depend on frontend-platform. When I create a package.json with:

{
        "dependencies": {
                "@edx/frontend-platform": "^8.3.1"
        }
}

Then I install dependencies with npm install --omit=dev, @openedx/frontend-build is correctly installed, confirming your explanations. So it seems that it's not strictly necessary to move frontend-build out of devDependencies, from the perspective of MFE apps.

[brian-smith-tcril]

Isn't that a valid test case?

It is not. frontend-platform is not designed to be deployed standalone, it is designed to be packaged and published to npm, and then installed by consuming applications.

The devDependencies exist to facilitate the development, testing, build, and publishing processes - they are ignored by consuming applications.

The peerDependencies are dependencies that frontend-platform expects the consuming application to provide. This is hidden a bit because of how npm handles peerDependencies:

In npm versions 3 through 6, peerDependencies were not automatically installed, and would raise a warning if an invalid version of the peer dependency was found in the tree. As of npm v7, peerDependencies are installed by default.

When running npm install or npm ci directly in frontend-platform, there is no consuming application to provide the peer dependencies, so we specify them as devDependencies to ensure they exist for development/testing/building/publishing.

[regisb]

👌 I'll update this PR accordingly. In the same spirit, should @edx/browserslist-config be moved to dependencies or copied to peerDependencies? I'm confused, because it seems to me that frontend-platform does not actually depend on that package.

[brian-smith-tcril]

The browserslist-config question is a good one. Just removing it from frontend-platform's dependencies completely might make the most sense there, but I'm not 100% sure. I'd like to see what others think about that one.

[adamstankiewicz]

When running npm install or npm ci directly in frontend-platform, there is no consuming application to provide the peer dependencies, so we specify them as devDependencies to ensure they exist for development/testing/building/publishing.

Additionally, one related item worth explicitly mentioning here is that some dependencies might only strictly support the example app within frontend-platform. For example, I believe @edx/brand only exists as a dependency in frontend-platform in support of the example app for testing.

Other library repos have a standalone example/package.json containing the dependencies needed to run the example app, where the root package.json only includes the dependencies strictly needed for the library itself (example, example), which might help be more explicit about which devDeps, etc. are installed in support of the library for consuming applications vs. only included for testing the project's example app.

In the same spirit, should @edx/browserslist-config be moved to dependencies or copied to peerDependencies? I'm confused, because it seems to me that frontend-platform does not actually depend on that package.

The browserslist-config question is a good one. Just removing it from frontend-platform's dependencies completely might make the most sense there, but I'm not 100% sure. I'd like to see what others think about that one.

@regisb @brian-smith-tcril My understanding is the build tools such as Babel (used within frontend-platform's build, source) take into account any configured browserslist defined within the package.json, via @babel/preset-env (docs, frontend-build source usage). That said, while frontend-platform installs it (imo, correctly as a devDep), the package.json doesn't tell define the browserslist option to use it like other projects (example, example, example). I believe frontend-platform (and other libraries with it missing, e.g. frontend-app-header) should be defining the browserslist option. It is a build-time dependency, though, not runtime.

npm install --omit=dev should install everything required to build the MFE, so the status quo is clearly wrong. We need react and paragon to build the MFE.

@bradenmacdonald I'm not sure I follow the recommendation for frontend-build to be a regular dependencies vs. devDependencies, per OEP-67, mentioning that Webpack should be included as a dependencies yet Jest, eslint, etc. installed as devDependencies. In the case of frontend-build, it provides all of these, which means Jest, eslint, etc. would additionally be included in production builds when moving frontend-build into dependencies.

Webpack documentation recommends installing Webpack with --save-dev if only using Webpack for bundling, which is the case in consuming libraries/MFEs of frontend-build:

Whether to use --save-dev or not depends on your use cases. Say you're using webpack only for bundling, then it's suggested that you install it with --save-dev option since you're not going to include webpack in your production build. Otherwise you can ignore --save-dev.

[brian-smith-tcril]

@adamstankiewicz I think @regisb's the overall goal here is to be able to run npm ci --omit-dev to improve build times for MFEs by not installing devDependencies.

This goes against my working understanding of direct dependencies being the dependencies that need to exist in the bundle that is served to users when using an MFE in the browser, and devDependencies being any extra dependencies that are required to develop/test/build.

As for frontend-build specifically, the fact that it is currently in frontend-platform's peerDependencies means that MFEs adding frontend-platform as a direct dependency end up with a non-dev dependency on frontend-build. In frontend-app-communications for example, @testing-library/react is in the package lock with "dev": true, while frontend-build (which is in package.json as a dev dep) is not in package-lock.json as dev.

[regisb]

I'm learning a lot from your conversation -- I'll just take a back seat and implement whatever decision you make. For instance, I could open PRs to move browserlists-config to the prod dependencies of MFE apps.

[brian-smith-tcril]

I know we had quite a few conversations around this when openedx/open-edx-proposals#615 was being discussed.

I think when I approved it I was focused on the "make sure things that shouldn't be direct deps are dev deps" aspect of it, and not fully grokking the "some things that are currently dev deps should move to be direct deps" aspect of it.

One thing that I don't see mentioned on that PR is impact on bundle size. I know keeping something out of direct deps is a pretty surefire way to keep it out of the bundle, but I know it's far from the only way to do so.

[regisb]

One thing that I don't see mentioned on that PR is impact on bundle size.

I can't answer with confidence on that. As I explained here, I'm not too sure how to test the changes in dependencies locally.

[adamstankiewicz]

This goes against my working understanding of direct dependencies being the dependencies that need to exist in the bundle that is served to users when using an MFE in the browser, and devDependencies being any extra dependencies that are required to develop/test/build.

@brian-smith-tcril Yes, I agree and have the same understanding as you regarding direct dependencies vs. devDependencies. I'm just calling out that some of our libraries (like frontend-platform) root package.json is polluted with dependencies only needed for the example MFE, contributing to the confusion in this PR, and that our understanding seems to contradict what's mentioned in OEP-67 (PR).

As for frontend-build specifically, the fact that it is currently in frontend-platform's peerDependencies means that MFEs adding frontend-platform as a direct dependency end up with a non-dev dependency on frontend-build. In frontend-app-communications for example, @testing-library/react is in the package lock with "dev": true, while frontend-build (which is in package.json as a dev dep) is not in package-lock.json as dev.

@brian-smith-tcril This indeed seems like a problem, and an unintended consequence from introducing support for env.config, the original rationale for why the peer dependency exists in frontend-platform as discussed here. We wanted to enforce frontend-platform as using an appropriate version of frontend-build so that env.config (and, now, the global variable PARAGON_THEME) exist, given frontend-platform (largely) assumes they will.

On some initial investigation, I'm wondering if this concern could be mitigated by adding the following to frontend-platform's package.json, where I believe frontend-build may potentially still be treated as a devDep in the consuming MFE, yet continue to enforce version compatibility within frontend-platform itself:

{
  "peerDependenciesMeta": {
      "@openedx/frontend-build": {
        "optional": true
      }
    }
}

Npm will not automatically install optional peer dependencies (docs)

[bradenmacdonald]

@bradenmacdonald I'm not sure I follow the recommendation for frontend-build to be a regular dependencies vs. devDependencies, per OEP-67, mentioning that Webpack should be included as a dependencies yet Jest, eslint, etc. installed as devDependencies. In the case of frontend-build, it provides all of these, which means Jest, eslint, etc. would additionally be included in production builds when moving frontend-build into dependencies.

That's exactly why I want us to split frontend-build into two packages: frontend-build and frontend-test. Then for most MFEs, frontend-build (webpack) would be a dependency and frontend-test (jest, eslint, etc.) would be a devDependency. Until then, we can't achieve the goal of reducing the install size/time for non-dev situations.

I'm just calling out that some of our libraries (like frontend-platform) root package.json is polluted with dependencies only needed for the example MFE

Let's move those into example/package.json. That makes way more sense.

[brian-smith-tcril]

I think the underlying issue here is that dependency categorization in package.json files doesn't seem to be designed for our use case.

For a node application running on a server, splitting between dependencies and devDependencies is very straightforward. Do we need it to run the node application on the server? If so, it's a direct dependency. If not, it's a dev dependency.

For a client side application it is a bit more complex.

It seems the status quo has more or less been:

Need it in the bundle when served to users in the browser	dependencies
Need it to build the bundle	Sometimes dependencies, sometimes devDependencies
Need it for testing/development only	devDependencies (except for a few cases where people have forgotten to use --save-dev)

and as of openedx/open-edx-proposals#615, the plan is to move to

Need it in the bundle when served to users in the browser	dependencies
Need it to build the bundle	dependencies
Need it for testing/development only	devDependencies

I really wish package.json supported another type of dependency, so we could have something like

Need it in the bundle when served to users in the browser	dependencies
Need it to build the bundle	buildDependencies (doesn't exist, I wish it did)
Need it for testing/development only	devDependencies

---

@bradenmacdonald re:

That's exactly why I want us to split frontend-build into two packages: frontend-build and frontend-test. Then for most MFEs, frontend-build (webpack) would be a dependency and frontend-test (jest, eslint, etc.) would be a devDependency. Until then, we can't achieve the goal of reducing the install size/time for non-dev situations.

Considering frontend-build is already making it in to MFEs as a direct dependency, I think this is a reasonable path forward. I also think we can start moving towards this without creating a frontend-test package.

If someone makes a breaking PR to frontend-build that moves all of the packages that are only needed for testing/development out of the direct dependencies and into devDependencies, then we could land that upgrade on MFEs by directly adding any of the packages needed to develop/test those MFEs as devDependencies on the MFEs themselves.

---

I do think we should bring some more people into this discussion. openedx/open-edx-proposals#615 landed without any input from @arbrandes or @adamstankiewicz, and I didn't think through the implications of it as fully as I should have before approving it.

Splitting up dependencies as

Need it in the bundle when served to users in the browser	dependencies
Need it for building/testing/development	devDependencies

still feels semantically correct to me, but I understand the downsides of that strategy when taking build times into account.

I think for implementing the strategy outlined in openedx/open-edx-proposals#615 to happen smoothly we'll need to clearly communicate the implications out to the greater Open edX frontend development community.

[adamstankiewicz]

I agree that continuing to include build-related dependencies in devDependencies also feels semantically correct and is more in line with recommended best practices that I've seen. For instance, I'd like reiterate my note above regarding Webpack's own documentation:

Webpack documentation recommends installing Webpack with --save-dev if only using Webpack for bundling, which is the case in consuming libraries/MFEs of frontend-build:

Whether to use --save-dev or not depends on your use cases. Say you're using webpack only for bundling, then it's suggested that you install it with --save-dev option since you're not going to include webpack in your production build. Otherwise you can ignore --save-dev.

[adamstankiewicz]

FWIW, I did a bit of discovery around my earlier suggestion to treat @openedx/frontend-build as an optional peer dependency to mitigate the concern @brian-smith-tcril raised above, where consuming MFEs end up with @openedx/frontend-build as a direct dependency vs. devDependency, by adding the following to @edx/frontend-platform (PR):

{
  "peerDependenciesMeta": {
      "@openedx/frontend-build": {
        "optional": true
      }
    }
}

More details in the draft PR , but when installed into frontend-app-communications with the optional peer dependency, @openedx/frontend-build is listed with devOptional: true instead of a direct dependency in the MFE's package-lock.json.

While npm ci --omit=dev still installs @openedx/frontend-build in this case, npm ci --omit=dev --omit=optional does not.

[regisb]

Maybe I've got the wrong idea since the beginning: I'm now thinking that webpack, frontend-build et al. could remain in devDependencies, and then Tutor would provide its own build system, shared across MFEs, in a separate node_modules/ (spoiler: Tutor would then probably not use webpack at all).

[bradenmacdonald]

@regisb That sounds like a lot of work, and I don't think it's good to have two different build systems around for the same MFEs. Much as I'm a fan of ditching webpack and other goals like that.

---

@brian-smith-tcril @adamstankiewicz

Splitting up dependencies as

Need it in the bundle when served to users in the browser dependencies
Need it for building/testing/development devDependencies
still feels semantically correct to me

I agree that this feels correct to me too. However, because of how we build our MFEs, we don't "need" anything in the bundle when serving to users. The result of our build process is a bunch of static files in dist and they have no dependencies at all. So it's more accurate to say:

"Ideal world"
Imported by the MFE source code (statically or dynamically)	dependencies
Need it for building/testing/development	devDependencies

But even though this makes sense and is fairly standard and familiar, it's not particularly useful. In this scenario, we never need just the dependencies without the devDependencies, so there's no practical advantage to splitting them up. This is made clear by the fact that many people have regularly mis-categorized our dependencies (regardless of what definition you use, there have been things that are clearly wrong in both directions), and there's been no consequences to that at all.

---

As I understand, there are two potential problems affecting MFE builds: image size and slow dependency installation. The "normal" way to build a containerized version of an app like an MFE is: install all the dependencies, build the MFE, delete node_modules, then install just the production dependencies with npm ci --omit=dev and create your image. But in our case we have something radically simpler: install whatever dependencies are required to build, run webpack, then save the dist folder as your result - it doesn't even have to be a docker image and it has no need for any dependencies at all. So because our dependencies are only used during the build and not part of any final image, I think disk space is not really a concern. The big problem is just the time it takes to install dependencies.

If npm could install our huge dependency tree in under 2 seconds, I don't think this would be a concern at all. So one fix is just to figure out a better way to install the enormous dependency tree. Another is to aggressively prune our dependencies. But I think the most tractable one is to use the dependencies vs. devDependencies split to install only what we need. It turns out that tools like jest and eslint are some of the worst offenders for slow install speed. Because they're not part of production bundles, they don't really try to be lean and mean. I did some quick testing now with frontend-build, and removing the *jest* and *eslint* dependencies reduced the time by 25% (npm install from 40s to 30s and npm ci from 8s to 6s). Multiply that by the number of MFEs and the number of times they're built, and I think it could justify the work to split those up. But as I've tried to mention, this is not the only way to improve the build times and perhaps not even the lowest hanging fruit.

[regisb]

As I understand, there are two potential problems affecting MFE builds: image size and slow dependency installation.

According to my experiments, there is a third one, which is: slow build. And that's actually the most important one.

Even after we trim dependencies, building the learning or the authoring apps still take more than one or two minutes, which seems totally bonkers to me.

Here are the low-hanging fruit I've found:

1. Installation optimization:
   a. Replace npm by yarn or pnpm.
   b. Install packages across multiple workspaces at once with yarn or pnpm
2. Build optimizations:
   a. Replace babel by esbuild both as a loader and minimizer.
   c. Get rid of BundleAnalyzer
   d. Trim lodash/fortawesome dependencies

Even after these build optimizations, I'm still getting terrible results:

• frontend-app-learning: 35s
• frontend-app-authoring: 144s

This SUCKS. So I'm now looking at replacing webpack by rspack. Preliminary results are very promising, but I'm still facing issues with the build result.

[brian-smith-tcril]

Even after we trim dependencies, building the learning or the authoring apps still take more than one or two minutes, which seems totally bonkers to me.

@regisb I think the main problem is that fast build time simply isn't something most React SPA developers optimize for at all. For example, I did a quick web search for "optimize react single page app for fast build" and found zero results related to build time - every result was about optimizing performance for users in the browser.

Most React SPAs are designed to deploy to one production instance, meaning they only ever need to be built a couple times per change that lands on the branch that deploys to production. Once with any staging specific build time configuration, and once with production specific build time configuration.

I'd argue the issue isn't one of build times, but one of how many times the MFEs need to be built. The design tokens project will at least remove the need to rebuild for theme changes.

I'm completely in favor of finding ways to decrease build times so the status quo of needing to rebuild often is less painful, but I don't think we'll be able to get build times to a point where they can be considered good in a world where rebuilding MFEs is a common occurance.

[regisb]

Here are a few resources related to improving build time:

https://stackoverflow.com/questions/67615038/how-to-reduce-react-app-build-time-and-understanding-behaviour-of-webpack-when-b
https://www.dhiwise.com/post/how-to-optimize-react-app-performance-with-webpack-5
https://dev.to/slashgear_/how-to-boost-the-speed-of-your-webpack-build-16h0
https://medium.com/@iMayank.Shekhar/keep-webpack-fast-a-tried-and-tested-guide-for-better-build-performance-fee466526233
https://michalzalecki.com/optimize-react-build-for-production-with-webpack/
https://rudolfolah.com/profiling-webpack-node-react/
https://slack.engineering/keep-webpack-fast-a-field-guide-for-better-build-performance/
https://medium.com/@sauravtiru/how-i-decreased-our-react-app-build-time-by-96-26-webpack-esbuild-3486b3262caa

In addition, the cambrian explosion of build tools (webpack, vite, rsbuild, turobopack, farm, ...) tells us that there are many people out there who face the same problem as us.

Build time does not affect only production builds, but development builds as well. An initial run of npm run dev takes (on my machine):

• frontend-app-learning: 44s
• frontend-app-authoring: 51s

MFEs need to be built locally, in CI, on production servers. This happens for every change in MFEs, and every change in an MFE plugin, for every user out there. We are building often because we are a community of many users with many different use cases. There is simply no way around it -- and design tokens are not going to resolve that at all. So we build often. And we can't put a 20 minute step with 400% CPU on the path of every build.

This is a situation that has affected everyone in the community for years. We are now learning that the OEP-65 improvements are postponed from Teak to Ulmo. In the meantime, more MFEs are added to the community distribution. So I'm now doing the same thing that I did when I created Tutor, which is to rewrite the build configuration from scratch. I'm only ~50% confident this is going to be successful, but I'm pretty sure that in the process I'm going to learn some thing that can have at least a medium impact on build time.

[brian-smith-tcril]

https://stackoverflow.com/questions/67615038/how-to-reduce-react-app-build-time-and-understanding-behaviour-of-webpack-when-b

This one describes an issue with build time increasing significantly after adding react-loadable (seemingly improperly) with what I assume is a goal of reducing bundle size by integrating component level code splitting. The only user other then the original poster that replied recommended module federation as a solution to the bundle size problem, that is a major part of the frontend-build effort.

---

https://www.dhiwise.com/post/how-to-optimize-react-app-performance-with-webpack-5

This one is mostly about performance for the user in the browser, with a small section about hot module replacement.

---

https://dev.to/slashgear_/how-to-boost-the-speed-of-your-webpack-build-16h0

This one is about build times. It also links to https://webpack.js.org/guides/build-performance/, which has a note in the production section

Warning

Don't sacrifice the quality of your application for small performance gains! Keep in mind that optimization quality is, in most cases, more important than build performance.

---

https://medium.com/@iMayank.Shekhar/keep-webpack-fast-a-tried-and-tested-guide-for-better-build-performance-fee466526233

This one is interesting. Definitely focused on better build times, and suggests a few ways to dig into what is slowing down builds.

---

https://michalzalecki.com/optimize-react-build-for-production-with-webpack/

This one is about optimizing for smaller bundles so users don't need to download as much. Definitely a good thing to do, but it's also possible that the configuration that leads to the smallest bundle isn't the configuration that leads to the fastest build.

---

https://rudolfolah.com/profiling-webpack-node-react/

This one is interesting. The section about build times links to a repo with a couple good examples of how to use some build time profiling tools, which is great!

---

https://slack.engineering/keep-webpack-fast-a-field-guide-for-better-build-performance/

This one has some reasonable suggestions. One thing that stood out to me was

DllPlugin will let you carve off prebuilt bundles for consumption by webpack at a later stage and is well suited to large, slow-moving dependencies like vendor libraries. While it has traditionally been a plugin that required an enormous amount of configuration, autodll-webpack-plugin is paving the way to a simpler implementation and is well worth a look.

but when looking into autodll-webpack-plugin I found the README began with

Important Note

Now, that webpack 5 planning to support caching out-of-the-box, AutoDllPlugin will soon be obsolete.

In the meantime, I would like to recommend Michael Goddard's hard-source-webpack-plugin,
which seems like webpack 5 is going to use internally.

---

https://medium.com/@sauravtiru/how-i-decreased-our-react-app-build-time-by-96-26-webpack-esbuild-3486b3262caa

The linked part (Part 1) is basically just saying "exclude node_modules from babel", which is something that we already do.

Part 2 is about moving away from babel/terser to esbuild. I'm not sure what exactly happened with openedx/frontend-build#566 (comment), but it seems that moving from babel to swc-loader didn't provide a huge improvement when it was tried.

That seems odd to me, and is probably worth investigating again.

---

Build time does not affect only production builds, but development builds as well.

Looking at the comparisons on https://rspack.dev/ shows decently large improvements for initial build times, but the hot module replacement times (how long it takes for a hot reload in the browser after making a change when running a dev server) are already quite low for webpack.

---

MFEs need to be built locally, in CI, on production servers. This happens for every change in MFEs, and every change in an MFE plugin, for every user out there. We are building often because we are a community of many users with many different use cases. There is simply no way around it -- and design tokens are not going to resolve that at all. So we build often. And we can't put a 20 minute step with 400% CPU on the path of every build.

I understand that the current state of build times is painful, and I agree it should be improved.

I am curious as to what a good build time would be in your mind. I don't doubt we can make improvements in this space, but I do have doubts that those improvements will be monumental.

I am also concerned about focusing too heavily on build time. I don't want build time improvements to come at the cost of in-browser performance for users of the platform (learners, course authors, course staff).

I also don't think we should rule out trying to reduce the number of situations that require site operators to rebuild MFEs. I in no way was trying to suggest that design tokens is a silver bullet, I just see it as an example of providing site operators with a way to customize the frontend without rebuilding. If we can do a similar thing for plugins (which I believe is one of the goals of frontend-base), then we'll be in a much better place.

---

in the process I'm going to learn some thing that can have at least a medium impact on build time.

I'm excited to see what you find!

As I mentioned earlier in this post, I have no doubt there are improvements we can make in this space.

Thank you for digging in and looking for ways to make build times better!

[bradenmacdonald]

Part 2 is about moving away from babel/terser to esbuild. I'm not sure what exactly happened with openedx/frontend-build#566 (comment), but it seems that moving from babel to swc-loader didn't provide a huge improvement when it was tried.

I just want to point out that I was only measuring the effects of removing babel on dependency size/count, not build speed. It seems I didn't write down the actual build times.

[brian-smith-tcril]

I just want to point out that I was only measuring the effects of removing babel on dependency size/count, not build speed. It seems I didn't write down the actual build times.

Ah. That makes a lot more sense. Definitely worth revisiting to see the impact on build speed.

[regisb]

In my experiments, replacing babel by esbuild has a considerable impact (roughly 50%), but it's not sufficient by itself (IMHO).