Skip to content

feat: etag support #746

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 12 commits into from
33 changes: 30 additions & 3 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
"webpack": "^4.0.0 || ^5.0.0"
},
"dependencies": {
"etag": "^1.8.1",
"mem": "^8.0.0",
"memfs": "^3.2.0",
"mime-types": "^2.1.27",
Expand Down
39 changes: 39 additions & 0 deletions src/middleware.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,35 @@ import path from 'path';

import mime from 'mime-types';

import getETag from 'etag';

import getFilenameFromUrl from './utils/getFilenameFromUrl';
import handleRangeHeaders from './utils/handleRangeHeaders';
import ready from './utils/ready';

const { hasOwnProperty } = Object.prototype;

export default function wrapper(context) {
let etagRegistry;

if (context.options.etags) {
etagRegistry = new Map();
context.compiler.hooks.done.tap('webpack-dev-middleware', (stats) => {
etagRegistry.clear();

try {
const { assets } = stats.compilation;
for (const assetId in assets) {
if (hasOwnProperty.call(assets, assetId)) {
const { existsAt: fsPath } = assets[assetId];
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems existsAt doesn't exist in Webpack 5 either

const etag = getETag(assets[assetId].source());
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems this is no longer possible in Webpack 5, due to error:

[webpack-dev-middleware] Error: Content and Map of this Source is not available (only size() is supported)
at SizeOnlySource._error (/node_modules/webpack-sources/lib/SizeOnlySource.js:16:10)
at SizeOnlySource.source (/node_modules/webpack-sources/lib/SizeOnlySource.js:26:14)

@alexander-akait Do you have a recommended work around for this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up leveraging chunkHashs, which is actually more convenient because I can just use the hash as the etag instead of generating one from the file source:

const { chunkHashs: assetHashes } = stats.compilation.records;
for (const assetId in assetHashes) {
	if (hasOwnProp(assetHashes, assetId)) {
		etagRegistry.set(assetId, JSON.stringify(assetHashes[assetId]));
	}
}

If there's an alternative you recommend, LMK.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/cc @sokra What do you think?

etagRegistry.set(fsPath, etag);
}
}
} catch (_ignoreError) {} // eslint-disable-line no-empty
});
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No hooks here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already can load assets here, please look at logic

Copy link
Author

@privatenumber privatenumber Oct 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw that there's access to the memfs instance, but is there a manifest object (eg. build stats) I can iterate over to get all the asset paths?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why you need iterate?

Copy link
Member

@alexander-akait alexander-akait Oct 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anyway if you need this, you have context, put this logic inside done hooks and collect them context.etags

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good.

I want to iterate a list of distribution files to efficiently calculate etags. If I only have the memfs instance, I'd have to traverse the entire fs, which isn't optimal.


return async function middleware(req, res, next) {
const acceptedMethods = context.options.methods || ['GET', 'HEAD'];
// fixes #282. credit @cexoso. in certain edge situations res.locals is undefined.
Expand Down Expand Up @@ -49,6 +73,21 @@ export default function wrapper(context) {
return;
}

if (etagRegistry) {
const assetEtag = etagRegistry.get(filename);
if (assetEtag) {
const { 'if-none-match': ifNoneMatch } = req.headers;
if (ifNoneMatch) {
if (assetEtag === ifNoneMatch) {
res.status(304).end();
return;
}
} else {
res.set('ETag', assetEtag);
}
}
}

try {
content = context.outputFileSystem.readFileSync(filename);
} catch (_ignoreError) {
Expand Down
3 changes: 3 additions & 0 deletions src/options.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@
"headers": {
"type": "object"
},
"etags": {
"type": "boolean"
},
"publicPath": {
"type": "string"
},
Expand Down