Serve precomputed checksums from sibling file

[UiP9AV6Y]

I am using SWS to serve a lot of small files. An external requirement is that each file is served with a header depicting its checksum like X-Checksum-Sha256, X-Checksum-Sha1, X-Checksum-Md5 or Content-MD5 (i provide all of them even only one is ever consumed). My current solution is to re-render the config file anytime a file is added/changed and restart SWS.

[advanced]

[[advanced.headers]]
source = "foo/bar.txt"
[advanced.headers.headers]
Content-MD5 = "md5-checksum"
X-Checksum-Md5 = "md5-checksum"
X-Checksum-Sha256 = "sha256-checksum"
X-Checksum-Sha1 = "sha1-checksum"

This has 2 downsides:

• the config file becomes quite large
• any file change requires a server restart to read the updated file

Apache supports calculating the checksum on the fly, altough they warn that this might cause performance issues as the value is calculated for each request and is not cached. #173 discusses a similar requirement for the ETag header which also touches on the topic of performance.

One potential solution i would like to seen in SWS, is to serve the checksum header(s) by reading their precomputed value from a file; e.g. a sibling file with a file extension depicting the resulting header. Example:

/docroot
\_ foo
 |_ bar.txt
 |_ bar.txt.md5sum
 |_ bar.txt.sha1sum
 \_ bar.txt.etag

A request for /foo/bar.txt would yield a response with 3 additional headers

HTTP/1.1 200 OK
Content-Type: text/plain
Last-Modified: Mon, 13 Jan 2025 20:11:20 GMT
ETag: <content of /docroot/foo/bar.txt.etag>
Content-MD5: <content of /docroot/foo/bar.txt.m5sum>
X-Checksum-Md5: <content of /docroot/foo/bar.txt.m5sum>
X-Checksum-Sha1: <content of /docroot/foo/bar.txt.sha1sum>

The files itself are nothing special and can still be accessed normally (i.e. a request for /foo/bar.txt.etag would simply yield the content of the file as response and if a file /docroot/foo/bar.txt.etag.sha1sum exists the response would contain its value in the X-Checksum-Sha1 header).

How those files are generated and if their content are actually proper checksums is not the concern of SWS. what remains to be decided is if SWS does are least some form of validation (e.g. what happens if /docroot/foo/bar.txt.md5sum is a binary, large, or multiline file?)

This feature should be disabled by default and configurable to only serve required headers regardless of which sibling files are present

[advanced]
[external_headers]
# no ETag header is provided unless a non-empty value is defined
#etag_extension = ""
md5_extension = ".md5sum"
sha1_extension = ".sha1sum"
# if a sibling file with this extension does not exist, no
# header is provided either
sha256_extension = ".sha256sum"

Those are my requirements but this feature can even be taken a step further by adding support for serving arbitrary headers from sibling files
(e.g. read a TOML file as map and use it as header input)

# /docroot/foo/bar.txt.hdrs
Content-Disposition = "burn-after-reading"
Content-Md5 = "some-checksum"
X-Forwared-For = "what?"

I am not a Rust developer but from looking at the code, a potential solution would be to extend the FileMetaData struct to include the retrieved values

[joseluisq]

The feature idea seems interesting. Here are my observations:

• For now, I will take out the Etag from this idea. If the checksums work as expected, then we could think about applying the same logic for ETag if it makes sense.
• This feature should be disabled by default and behind an advanced configuration entry.
• We are talking about working with single-line plain text files. Therefore, SWS should take care of validating them accordingly.
• The checksum file extensions should be clearly defined so SWS knows where to look.

Those are my requirements but this feature can even be taken a step further by adding support for serving arbitrary headers from sibling files (e.g. read a TOML file as map and use it as header input)

# /docroot/foo/bar.txt.hdrs
Content-Disposition = "burn-after-reading"
Content-Md5 = "some-checksum"
X-Forwared-For = "what?"

As I said, if this works fine, then why not think about using a similar approach for setting up dynamic headers.

--

I would suggest creating a feature issue defining clearly what this feature should do, then starting a simple prototype and picking up one of the checksum types first (e.g., MD5) to see how it will behave. Then, if that works as expected, decide to add the other checksum hashes gradually.