Webpack 3.7.0 and above slow build

[tanzl88]

Short description of the problem:

When webpack is updated to 3.7.0 and above, webpack build step takes much longer time and more memory.

What behavior are you expecting?

Webpack build step should be almost identical after update.

Debug mode ionic serve log. Identical project codes and npm packages except webpack.

Webpack 3.7.1

Webpack step is 1 minute slower and use ~200MB more memory compared to webpack 3.6.0

[22:52:22]  watch started ... MEM: 158.6MB 
[22:52:22]  build dev started ... MEM: 158.7MB 
[22:52:22]  clean started ... MEM: 158.5MB 
[22:52:22]  clean finished in 25 ms MEM: 159.1MB 
[22:52:22]  copy started ... MEM: 159.1MB 
[22:52:23]  deeplinks started ... MEM: 174.3MB 
[22:52:23]  deeplinks finished in 608 ms MEM: 209.4MB 
[22:52:23]  transpile started ... MEM: 209.4MB 
[22:52:29]  transpile finished in 5.86 s MEM: 438.2MB 
[22:52:29]  preprocess started ... MEM: 438.2MB
[22:52:29]  copy finished in 6.81 s MEM: 439.9MB 
[22:52:29]  preprocess finished in 125 ms MEM: 443.7MB 
[22:52:29]  webpack started ... MEM: 443.7MB 
[22:53:48]  webpack finished in 79.33 s MEM: 1206.0MB 
[22:53:48]  sass started ... MEM: 1206.0MB 
[22:53:51]  sass finished in 2.11 s MEM: 1205.2MB 
[22:53:51]  postprocess started ... MEM: 1205.2MB 
[22:53:51]  postprocess finished in 46 ms MEM: 1209.8MB 
[22:53:51]  lint started ... MEM: 1209.8MB 
[22:53:51]  build dev finished in 88.51 s MEM: 1209.9MB ```

Webpack 3.6.0

[22:47:08]  watch started ... MEM: 159.4MB 
[22:47:08]  build dev started ... MEM: 159.5MB  
[22:47:08]  clean started ... MEM: 160.4MB 
[22:47:08]  clean finished in 35 ms MEM: 160.9MB 
[22:47:08]  copy started ... MEM: 160.9MB 
[22:47:09]  deeplinks started ... MEM: 175.4MB 
[22:47:09]  deeplinks finished in 578 ms MEM: 188.9MB 
[22:47:09]  transpile started ... MEM: 188.9MB 
[22:47:15]  transpile finished in 5.50 s MEM: 429.2MB 
[22:47:15]  preprocess started ... MEM: 429.2MB 
[22:47:15]  copy finished in 6.42 s MEM: 429.4MB 
[22:47:15]  preprocess finished in 126 ms MEM: 434.3MB 
[22:47:15]  webpack started ... MEM: 434.3MB 
[22:47:30]  webpack finished in 15.24 s MEM: 1042.8MB 
[22:47:30]  sass started ... MEM: 1042.8MB 
[22:47:32]  sass finished in 1.97 s MEM: 1041.7MB 
[22:47:32]  postprocess started ... MEM: 1041.7MB 
[22:47:32]  postprocess finished in 47 ms MEM: 1045.7MB 
[22:47:32]  lint started ... MEM: 1045.7MB 
[22:47:32]  build dev finished in 23.90 s MEM: 1045.7MB 

Which @ionic/app-scripts version are you using?
3.0.0

[dmastag]

I have the same problem. My production build went from 12 minutes to 26 minutes after upgrading to the new webpack that I got from [email protected]

[moberwasserlechner]

I have the same problem, with a 70+ module ionic app my build time is around 70s.

But I tried to use Webpack 3.8.1 directly in my dev dependencies, which cuts the build time in half using ionic serve. This is at least better, than before.

Webpack 3.7.1 as transitive dependency.

$ ionic serve
Starting app-scripts server: --address 0.0.0.0 --port 8100 --livereload-port 35729 --dev-logger-port 53703 - Ctrl+C to cancel
[18:52:02]  watch started ...
[18:52:02]  build dev started ...
[18:52:03]  clean started ...
[18:52:03]  clean finished in 3 ms
[18:52:03]  copy started ...
[18:52:03]  deeplinks started ...
[18:52:03]  deeplinks finished in 393 ms
[18:52:03]  transpile started ...
[18:52:08]  transpile finished in 4.94 s
[18:52:08]  preprocess started ...
[18:52:08]  preprocess finished in 1 ms
[18:52:08]  webpack started ...
[18:52:08]  copy finished in 5.59 s
[18:53:14]  webpack finished in 66.34 s
[18:53:14]  sass started ...
[18:53:15]  sass finished in 995 ms
[18:53:15]  postprocess started ...
[18:53:15]  postprocess finished in 104 ms
[18:53:15]  lint started ...
[18:53:15]  build dev finished in 73.01 s
[18:53:16]  watch ready in 73.58 s
[18:53:16]  dev server running: http://localhost:8100/

3.8.1

Starting app-scripts server: --address 0.0.0.0 --port 8100 --livereload-port 35729 --dev-logger-port 53703 - Ctrl+C to cancel
[19:22:28]  watch started ...
[19:22:28]  build dev started ...
[19:22:29]  clean started ...
[19:22:29]  clean finished in 4 ms
[19:22:29]  copy started ...
[19:22:29]  deeplinks started ...
[19:22:29]  deeplinks finished in 395 ms
[19:22:29]  transpile started ...
[19:22:34]  transpile finished in 4.78 s
[19:22:34]  preprocess started ...
[19:22:34]  preprocess finished in 1 ms
[19:22:34]  webpack started ...
[19:22:34]  copy finished in 5.43 s
[19:22:58]  webpack finished in 24.40 s
[19:22:58]  sass started ...
[19:22:59]  sass finished in 946 ms
[19:22:59]  postprocess started ...
[19:22:59]  postprocess finished in 82 ms
[19:22:59]  lint started ...
[19:22:59]  build dev finished in 30.83 s
[19:23:00]  watch ready in 31.30 s
[19:23:00]  dev server running: http://localhost:8100/

ionic info

cli packages: (C:\Users\moberwasserlechner\Development\git\team-conductor-go\node_modules)

    @ionic/cli-utils  : 1.14.0
    ionic (Ionic CLI) : 3.14.0

local packages:

    @ionic/app-scripts : 3.0.1
    Ionic Framework    : ionic-angular 3.7.1

System:

    Node : v6.9.5
    npm  : 3.10.10
    OS   : Windows 10

[dmastag]

A fix for me was using [email protected]
But since today when updating to ionic-app-script 3.1.0 this is not working anymore, and production build time went from 12 minutes to 30 minutes.

Also I can't do the ionic serve

[dmastag]

I see, since [email protected] is dependent on webpack 3.8.1 so adding webpack 3.6.0 separate will not help anymore

[tanzl88]

Hi @danbucholtz, would like to check whether this issue has to wait to be resolved at webpack's end since it is not resolved in the recent app script update? The long build time makes app script unusable.

[dmastag]

@tanzl88 I also opened an issue at the webpack repo

webpack/webpack#5876

They are asking for a repo that has the problem for them to check, but I am sadly not able to reproduce on a new ionic app and can't send my company's app. Do you have a sample repo that you can give them to check?

[danbucholtz]

@tanzl88,

We'll take a look. This is sort of strange. I wonder why that would be. Worst case, I don't see any reason we couldn't go back to 3.6.0 potentially (not sure if that support ModuleConatenationPlugin or not).

Thanks,
Dan

[masimplo]

@dmastag are you usingmax_old_space_size? How much memory do you see used during the build process?
I am only asking because, I was using max_old_space_size in my package.json scripts for ionic:build and server but then was building using ionic cordova build, to which max_old_space_size was not being applied (I realised after looking to mem usage during build) and was having the same behaviour. Running ionic cordova build with a larger max_old_space_size than the used memory, returned my build process to its previous acceptable performance levels.

[mlynch]

@dmastag your build was 12 minutes? How?!

[dmastag]

@masimplo yes, actually I am
Here is an excerpt from webpack/webpack#5876 (comment)

ionic serve takes a long time and in windows Task manager the amount of memory does not go above 750MB

But on 3.6.0 it can go up to 1.3GB and is working in 2 minutes to load my app.

So are you suggesting that I set the --max-old-space-size=4096 to a higher value?
But this would only be for production builds, any advice on ionic serve?

@mlynch correct

The following are the builds using gitlab-ci

Using [email protected]

Using [email protected]

[mlynch]

That's crazy. Are you doing anything special? What does your app look like (i.e. anything that would cause builds to be that slow)?

[dmastag]

Not sure, I have 238 lazy loaded IonicPages
and the packages and app.modules that I use have nothing special
Gist to app.module.ts and package.json

Though this app has been using ionic since angular 2.0.0-rc.4 and ionic-angular 2.0.0-beta.11
But I am sure to upgrade every time based on the release notes

[mlynch]

@dmastag okay that explains it 😅 Why so many pages? Could probably cut down on it if you generalize them somehow, but I don't know your app

[dmastag]

Yeahh, maintenance has become quite difficult since its an enterprise app with 10-15 developers working on it 😃
So am trying to really slim it down now

But still, this doesn't explain why webpack 3.6.0 to 3.8.1 is such a huge jump in build time 😢

[masimplo]

@dmastag do you have any entries in your package.json scripts looking like these?

"ionic:build": "node --max_old_space_size=4096 ./node_modules/.bin/ionic-app-scripts build",
"ionic:serve": "node --max_old_space_size=4096 ./node_modules/.bin/ionic-app-scripts serve",

these should address both build and ionic serve.

[dmastag]

@masimplo Yes I do, because I get a exceed memory error if I don't use that

[masimplo]

I had 2048 on these two and was getting a really slow build and sometimes out of memory exceptions. I raised them to 4096 and builds are back to normal build times. Did you try setting it to something like 8GB? You have a large project so it wouldn't be the weirdest thing to use a lot of memory.

[danbucholtz]

@masimplo, @dmastag,

If you want to use those commands to increase memory, make sure you call the npm scripts directly. So npm run ionic:build or npm run ionic:serve. The ionic cli doesn't call the npm scripts anymore.

Thanks,
Dan

[sharonswli]

@dmastag For our app, I needed to allocate 8GBs for node to get a successful build after updating to 3.0.1, and that still took 30 mins to build with ngc. We also have a pretty large app, but we never encountered any issues until the update (We were using app-scripts 1.3.7 previously). Maybe increasing your node memory limit would also help for you?

But either way, upgrading webpack/ionic-app-scripts shouldn't yield such drastic build time difference, and I believe many others with similarly non-trivial size apps have been facing the same issue #1247 #1076 #1036. I can't help but think upping the node memory limit as the app size inevitably increases is just putting a band-aid to the problem. We are currently stuck on @ionic/[email protected] that uses [email protected] because the new version of ionic-app-script is practically unusable for our app unless we allocate like 10GBs to node. Reverting app-scripts back to an older version immediately reduces our AOT build time back to a reasonable amount (2GBs memory limit on node. Build time takes about 2-4 mins on AOT)

[GuoLunHao]

I have the same problem with @dmastag , my app have nearly 300 lazy loaded IonicPages, after upgrading to [email protected] is dependent on webpack 3.8.1, and production build time went from 10 minutes to 30 minutes.

[lfxsantos]

Do you have any performance improvement ?

[ZacharyGarcia305]

Hello everybody. I know I am late to the convo... BUT, my slow webpacks came from upgrading @ionic/[email protected] to @3.1.7-201801172029.... So, by uninstalling app-scripts and reinstalling version 2.1.4, I was able to go from 70 second build times to 13 second build times and, best of all, 7 second incremental build times. So, try the following and see if it helps!!!

npm uninstall @ionic/app-scripts
npm install @ionic/[email protected]

Hope it helps!

[cvaliere]

I totally confirm that downgrading from [email protected] to [email protected] makes incremental builds go from ~30 sec to ~5 sec

(you need to install @angular/tsc-wrapped to make it work, by the way)

would be good to understand why :)

[mlynch]

Hey all. We're aware of this, but we don't have a solution right now. If 2.1.4 works for you, awesome. With Ionic 4's angular support, we don't need app scripts anymore, we will use the ng cli toolchain. I'm not saying that will solve these problems because some are inherent to the Angular build process, but it should help.

Stay tuned for that, we're getting close to the alpha release this Spring.

[cvaliere]

@mlynch do you know what we lose in using [email protected]? I must admit I don't exactly understand what these scripts do, and how they relate with webpack. I'm currently using [email protected] with latest [email protected], it seems to work, but maybe I miss something.

I'm so glad that you decided to use ng cli in next version. Good choice, IMHO 👍

Good luck for Ionic 4. We really wait for it, as Ionic 3 is stuck with annoying bugs :(

[mlynch]

On the choice: yea, and it's much less for us to have to maintain, so we can focus more on the components!

You shouldn't lose much, at this point ng cli has overlapped with app scripts pretty well

[pjorquera]

I confirm that with version 3.1.2 of @ionic/app-scripts also works correctly. With that version webpack phase runs for 12 seconds (my project has ~50 IonicPage() lazy pages with its own @NgModule).

If I try one of [3.1.3 - 3.1.8] then webpack runs for almost 70 seconds (in both first start and each update on live reload).

[ayaka209]

I have a 8000+ lines template and webpack used more than 3 hours…… (with --prod)

[jlbeard84]

I've got about 170 modules and moving to version 3.1.2 of @ionic/app-scripts cut my build time down from about 5 minutes to 3 and a half minutes. Still not great but better...

[kevryan2]

Any news on when app-scripts will look into this (or if it's confirmed that it is instead webpack's issue)? I can confirm that using app-scripts 3.1.2 makes webpack work approx. 30x faster for both builds and live reloads (live reload average time of 60s down to 2s). Will not be upgrading app-scripts beyond 3.1.2 until that is fixed, obviously an insane downgrade in performance to upgrade to 3.1.7.

[Sroose]

Same here, 3.1.2 works while 3.2.0 takes forever.

[zheng17]

I upgraded ionic cli to 4.3.0 yesterday, and use ionic-app-scripts 3.2.0, webpack 3.8.1. Webpack took 180 seconds in --prod build mode, that took about 20 minutes before. I am pretty happy with the latest version.

Make sure change the memory setting after updating.
node_modules/@ionic/app-scripts/bin/ionic-app-scripts.js
#!/usr/bin/env node --max-old-space-size=8192

[dmastag]

@zheng17 awesome
Could you explain step by step how you managed to get ionic-app-scripts 3.2.0 to work with webpack 4.24.0?
I want to try it out too

[zheng17]

@zheng17 awesome
Could you explain step by step how you managed to get ionic-app-scripts 3.2.0 to work with webpack 4.24.0?
I want to try it out too

Hi @dmastag , sorry for confusing you. I check the package version of webpack again on my local, the version is 3.8.1. I used wrong commnad 'npm view webpack version'.

But it did faster than before.

[dmastag]

Thanks @zheng17
Sadly not working for me. My build usually take 35 minutes using 3.1.2 and when upgrading to 3.2.0 haven't finished after 120 minutes.

[dielangsea]

i toke 30min using --prod, build scripts version is: 3.2.0

[tomavic]

It's nearly 2020 and the problem still show up. I have an app with more than 100 pages, and I am not using lazyloading. I've managed to solve this issue by running

ionic cordova build android --aot --minifyjs --minifycss

For more information #1426