Precache manifest missing entries when recompiled via webpack --watch

[lakesare]

Library Affected:
workbox-webpack-plugin

Browser & Platform:
Webpack 2.3.3, workbox-webpack-plugin 3.6.3

Issue Description:
In the webpack --watch mode, precache-manifest.hash.js file at first generates a perfect list of files, but on subsequent updates it misses some files.
For example, files generated with CopyWebpackPlugin are missing (but including copyUnmodified: true in options helps).
Another kind of files that are missing, is fontawesome-webfont.eot?hash files (importes via this process https://medium.com/@chanonroy/webpack-2-and-font-awesome-icon-importing-59df3364f35c), this I don't yet know how to fix.

Here is a similar issue for another manifest plugin: shellscape/webpack-manifest-plugin#144.

[arackaf]

Just adding to this, for completeness, the navigate fallback file is also ejected on re-build.

[jeffposnick]

Sorry for letting this drop until now.

workboox-webpack-plugin background

workbox-webpack-plugin generates its precache manifest based on the contents of the current compilation's assets, as determined in a compiler hook into the emit event. I think that's fairly standard for a webpack plugin.

webpack --watch mode behavior

After running some local experiments with --watch mode, I've found that only brand new assets, and modifications to existing assets, show up in the compilation's assets available during each subsequent emit hook invocation.

I believe some other plugins (like webpack-manifest-plugin, as referenced in shellscape/webpack-manifest-plugin#144) maintain state information about all of the previous compilation's assets and make additions or changes to hashes to that state information upon each subsequent compilation. They then create their output based on the cumulative total of all the assets that have been created across multiple compilations. (This seems to be the purpose of the seed parameter that can be passed to webpack-manifest-plugin.)

Problems with cumulative state

workbox-webpack-plugin could do something similar, and generate its precache manifest based on a cumulative list of assets across multiple compilations.

The concern that I have, though, is that there is no way for a webpack plugin to detect that a given asset has been removed following a recompilation—the absence of an asset from any particular compilation can't be used as a signal that it was removed, since that scenario is identical to what would happen if it was still present but unchanged.

Unfortunately, it's pretty easy to trigger that scenario even without bringing any third-party plugins into the mix; if you're following the official guide for dynamic imports with output.chunkFilename: '[name].bundle.js', and you have code like:

// Assume this is in your main chunk:
async function load() {
  const b = await import(/* webpackChunkName: "b" */ './b.js');
  const c = await import(/* webpackChunkName: "c" */ './c.js');
}

If you remove the dynamic import for c in watch mode, that will trigger a recompilation, but the only entries provided in assets will be b and the main chunk. There's no indication that c is no longer emitted.

Possible solutions

Given that, it sounds like there's a tradeoff when in --watch mode between potentially leaving out manifest entries that should be there (the current behavior) and including entries that correspond to assets that have been deleted (if we maintained asset info across compilations, and cumulatively added to it).

Neither solution is ideal, but I think from a service worker functionality perspective, the current behavior might be preferable—if you include a URL in the precache manifest list that doesn't exist, the installation process of the updated service worker will end up failing entirely. That sounds worse than the current behavior, in which the updated service worker installs but precaches less than expected. I'd love to hear folks' thoughts, though.

Additional warnings

What I plan on doing regardless is to detect when workbox-webpack-plugin has been called across multiple compilations and add a warning to the webpack compilation letting developers know not to trust the output for anything that should be considered production-ready. I will include a link that points to this (hopefully complete) description of the problem and trade-offs to give developers more context. The ultimate recommendation is that developers should only deploy a service worker that was generated by a one-shot webpack compilation.

[arackaf]

Ugh Rock and a Hard place, Jeff. Sorry webpack leaves you so few good options.

Would plugging this specific bug make sense? Ie, on watch, manually, forcibly check for the navigateFallback, and just add that? Not perfect, but it would solve one of the worse manifestations of this bug.

[jeffposnick]

So what I'd like to do for the v5 timeframe is add in the webpack compilation warning when we detect that re-compilation has occurred, letting folks know that the contents of the precache manifest might be inconsistent.

I'm going to explore a bit more to see if there's anything in the webpack internals that could give us the info we need about the complete set of assets, raise issues with the webpack maintainers if not, and in all likelihood switch things over to the cumulative asset manifest for the time being.

I don't want to block the v5 release on that work, as it's likely going to take some time. I feel like adding in the warning about inconsistently gives us some leeway to make this sort of change post-v5, since developers will have a heads-up about not relying on --watch for anything production-ready.