Skip to content

ref(preprod): Use regular PUTs for snapshot archive uploads - #128181

Merged
lcian merged 8 commits into
masterfrom
lcian/ref/snapshot-archive-resumable-uploads
Oct 21, 2026
Merged

lcian merged 8 commits into
masterfrom
lcian/ref/snapshot-archive-resumable-uploads

Conversation

@lcian

@lcian lcian commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Bump objectstore-client to 0.2.10 and upload snapshot ZIP archives with a simple Session.put instead of using the multipart API.

Note that this means that other code paths where we do simple PUTs will also use resumables under the hood from now on, so this change affects more than preprod, most notably debug files which are the only likely files to exceed the default threshold of 256 MiB.

Motivation: the multipart API is going away and this was the only remaining usage of it in sentry.

Note that as of this version of objectstore-client, Session.put benefits from "automatic resumable uploads", meaning that calls to Session.put are translated to the new Resumable Uploads protocol internally when the provided stream is seekable and its length is above a certain threshold (which currently defaults to 256 MiB).

The archive call sets resumable_threshold_bytes=0 to attempt resumable uploads for every archive (given that this is not a latency sensitive operation, we prefer to take the more reliable path always).

Even though we remove multipart, this makes the large snapshot ZIP uploads resilient to failures, as with resumable uploads they would pick up the upload process where it was left instead of retrying the whole thing.

Multipart also potentially made the upload faster by uploading parts in parallel, but speed should generally not be an issue, as all transfers are happening inside the same VPC, so they should max out available bandwidth practically.

Close FS-474

getsentry-bot and others added 2 commits October 20, 2026 14:02
@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Oct 20, 2026
@lcian

lcian commented Oct 20, 2026

Copy link
Copy Markdown
Member Author

bugbot run
@sentry review

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 8bbc763. Configure here.

@lcian lcian changed the title ref(preprod): Use automatic resumable uploads for snapshot archives ref(preprod): Use regular PUTs for snapshot archives Oct 20, 2026
@lcian lcian changed the title ref(preprod): Use regular PUTs for snapshot archives ref(preprod): Use regular PUTs for snapshot archive uploads Oct 20, 2026
@lcian
lcian marked this pull request as ready for review October 20, 2026 14:59
@lcian
lcian requested review from a team as code owners October 20, 2026 14:59
@linear-code

linear-code Bot commented Oct 20, 2026

Copy link
Copy Markdown

FS-474

Comment thread src/sentry/preprod/snapshots/zip_tasks.py
Comment thread src/sentry/preprod/snapshots/zip_tasks.py Outdated
@lcian
lcian requested a review from a team October 21, 2026 09:25

@jamieQ jamieQ left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall the changes lgtm. Nice that some of the logic will be handled automatically by the library! Left a couple minor comments.

Comment thread src/sentry/preprod/snapshots/zip_tasks.py
compress: Compression | None = None,
resumable_threshold_bytes: int | None = None,
) -> str:
return self._primary.put(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC the retry logic will change slightly after this change. Not sure how much it matters, but seems like 500s in the resumable upload path will now fail vs resulting in a retry attempt. This was noted by the bot review here, but unclear if that's an intentional decision or not.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving out 500s is intentional as you should get a 500 only when something goes really wrong internally.
Although the bot comment also mentions ConnectTimeoutError which we're not treating as retryable right now but I think we should, so I'm gonna make a patch release to do that, thanks for flagging.

@lcian
lcian merged commit 3be8aa4 into master Oct 21, 2026
96 checks passed
@lcian
lcian deleted the lcian/ref/snapshot-archive-resumable-uploads branch October 21, 2026 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants