Repository navigation
Add rewrite_media_url_host command for repairing media URLs after a server address change - #1153
Conversation
Media URLs are stored with an absolute scheme and host. When a self-hosted server's domain changes, previously stored timelapse, prediction, poster and event snapshot URLs keep pointing at the old host, and resign_media_urls could not repair them because new_signed_url() only replaces the query string. With --rewrite-host, absolute URLs are rewritten to the given host while being re-signed, with the scheme taken from SITE_USES_HTTPS (matching how the server builds all other full URLs). Relative URLs are left untouched. The digest is calculated from the path only, so rewriting the host never invalidates the signature. Without the flag, behavior is unchanged.
|
Thanks for the thorough report and tests. The problem and need for a repair command make sense. Could this be implemented as a separate I’d also prefer requiring both the old and new origins, and rewriting only URLs matching the old origin. Rewriting every absolute URL could unintentionally modify externally hosted or provider-signed URLs. Preserving their existing query strings would also be safer. The model/field traversal can still be shared between the two commands to avoid duplication. |
|
@kennethjiang - sure thing. I was on the fence about submitting this in the first place. Glad you like the idea. I'll re-tool it later tonight when I get a chance and admmend the PR. |
|
@kennethjiang - reverted Since these functions share some database traversal code, I extracted the shared helpers to |
|
Thanks for reworking this so thoroughly. The standalone command, explicit old and new origins, dry-run support, and shared traversal make the repair much safer. Merged—thank you for the contribution! |
Fixes #1152 and the issue referenced in #1129
What changed
A new management command:
It rewrites the scheme and host of every stored media URL that matches
--old-originso it points at--new-origininstead. Only URLs on the old origin are touched; URLs on any other origin (for example externally hosted or provider-signed URLs), and relative URLs, are left as they are. The rest of the URL, including the query string and digest, is preserved.Both origins are required.
--dry-runreports how many URLs would be rewritten per model without saving anything. The command prints per-model counts either way.resign_media_urlsis unchanged in behavior. The loop over media URL fields (and the progress bar it uses) is now shared between the two commands inlib/media_urls.py, next to the other shared helpers. Happy to move it if you'd prefer another location.The server guide gets a short section under "Django Site" explaining when to run the command.
Why
Media URLs are stored in the database as full URLs, including scheme and host. When a self-hosted server's address changes, all previously stored timelapse, detective, poster and event snapshot URLs keep pointing at the old address and break.
resign_media_urlscannot repair this: it is forDJANGO_SECRET_KEYrotation and only replaces the query string.The HMAC digest covers the path only, so rewriting the host never invalidates the signature. See the linked issue.
Review history
The first version of this PR added a
--rewrite-hostflag toresign_media_urls. Per review it is now a standalone command that requires both origins and preserves query strings.How I tested it
rewrite_url_origin()inbackend/lib/tests.py: matching origin rewritten with the rest of the URL preserved and the digest still valid, other origins untouched, relative URLs untouched, case-insensitive host and trailing-slash tolerance, invalid origins rejected.backend/app/tests.pyviacall_command(): matching rows rewritten and still valid, other-origin and relative rows untouched, invalid origin rejected before anything is changed, dry run changes nothing.resign_media_urlsstill keeps the host and produces a valid digest after the shared-helper refactor.docker compose -f docker-compose.yml -f docker-compose-dev.yml run --rm web python manage.py test. The 33 errors on cleanmaster(test setup creating users without a Syndicate row) are unchanged; all new tests pass.Notes
resign_media_urlssaved every row it visited). Stored URLs are byte-for-byte the same either way; onlyupdated_attimestamps on unchanged rows are no longer bumped.