fix(evals): Use account-backed Cloudflare tunnels in CI - #1918
Draft
sentry-junior[bot] wants to merge 6 commits into
Draft
sentry-junior[bot] wants to merge 6 commits into
sentry-junior[bot] wants to merge 6 commits into
Conversation
Co-Authored-By: David Cramer <david@sentry.io>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Co-Authored-By: David Cramer <david@sentry.io>
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
Co-Authored-By: David Cramer <david@sentry.io>
|
1 screenshot change — 1 changed · 0 added · 0 removed Review screenshots in Frameshift
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace anonymous CI Quick Tunnels with a separate remotely managed tunnel and DNS record for each eval invocation. One runner-side script installs the latest official cloudflared release with SHA-256 verification, runs the command, and removes its resources. Run, attempt, job, shard, and a random value determine the hostname; jobs never share a tunnel.
Use
sentry-ci-<hash>.sentry.coolso the zone's standard Universal SSL wildcard covers CI. SetCLOUDFLARE_TUNNEL_BASE_DOMAINtosentry.cool, not a deeper subdomain. No advanced certificate is required.Keep Cloudflare credentials out of the eval command and Sandboxes. Preserve proxy and fixture-control authentication. Verify public HTTPS reaches this exact proxy before starting evals, using normal system DNS. Stop child process groups on failure or cancellation, and retain exact cleanup state for an
always()fallback step. Local evals still use Quick Tunnels.Document the actual dashboard labels—Argo Tunnel (Legacy) → Edit and DNS → Edit—alongside permission IDs, resource scopes, GitHub bindings, TLS setup, token rotation, and manual cleanup after runner loss. No eval inputs, thresholds, or reply budgets change.
This remains a draft until CI proves real routing and cleanup. Set
CLOUDFLARE_ACCOUNT_IDandCLOUDFLARE_ZONE_IDto their hexadecimal IDs as repository variables, and bindCLOUDFLARE_API_TOKENas a repository secret. Confirm Universal SSL is active. Offline coverage cannot prove Cloudflare permissions, DNS, certificates, or GitHub cancellation behavior.Fixes #1914
via David Cramer.
--
View Junior Session [Sentry]