Skip to content

Prepare the Rust 1.99.0 CI move and stop discarding policy install results in tests - #557

Merged
Antnation merged 3 commits into
mainfrom
chore/rust-ci-1.99-and-test-results
Oct 16, 2026
Merged

Antnation merged 3 commits into
mainfrom
chore/rust-ci-1.99-and-test-results

Conversation

@Antnation

@Antnation Antnation commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #549. Part of #44 (the executable build-tool contract from #429).

#549 moved the production/default compiler to Rust 1.99.0 and left rust_ci on 1.98.1, so the CI compiler move could be reviewed on its own. This PR was meant to make that move. It can't, and this PR does the first required step instead. It also fixes the ten unused_must_use test warnings #549 listed as a follow-up, and adds one cheap warning gate so they cannot come back unnoticed.

Why rust_ci cannot move to 1.99.0 in this PR

The first push of this branch moved rust_ci to 1.99.0 (run 38669955458). Every Rust job passed on 1.99.0, but microbench failed before it measured anything:

error: use of deprecated method `std::sync::atomic::Atomic::<u64>::fetch_update`: renamed to `try_update` for consistency
  --> harness.rs:76:10
   = note: `-D deprecated` implied by `-D warnings`

The standalone harness scripts/benchmarks/pure_paths.rs is compiled with -D warnings, so its code has to change. The comparison only accepts harness digests listed in the base commit's microbench-budget.json. That means the harness cannot change in the same PR that approves its new digest. Relaxing -D warnings for the benchmark compile would weaken a gate, and editing the head budget cannot make the comparison pass. pure-path-benchmarks.md prescribes the path instead: two reviewed changes, where the first adds the next digest beside the current one and the second changes the harness.

So this PR is the first of those two changes, and rust_ci stays on 1.98.1 for now. The follow-up is ready and verified locally; see below.

What run 38669955458 showed about 1.99.0 (useful for the follow-up)

  • clippy (cargo clippy --workspace --locked -- -D warnings) and postgres-foundation's --no-default-features --all-targets Clippy step both passed on 1.99.0. 1.99.0 adds no lints to fix.
  • fmt, test (with the new -D warnings), security-corpus and every other job that uses the ci profile passed on 1.99.0.
  • The harness rename above is the only change the 1.99.0 move needs.

1. Pre-approve the try_update harness digest

microbench-budget.json now accepts two harness digests:

  • 0e005f79…98881f73: the current harness. It stays active and unchanged in this PR.
  • f0c6abba555cb81e8cc419f39473895eeefb3ad558537f95659dfcdfcf22285e: the same file with its one call renamed (.fetch_update( to .try_update(, same arguments and closure). It is computed the way the controller and the transport guard hash it: SHA-256 of the UTF-8 text with LF line endings.

The new harness is rustfmt --edition 2021 --check-clean on both 1.98.1 and 1.99.0. try_update is stable on both, so it compiles under the controller's flags (including -D warnings) on either compiler. Allocation budgets and thresholds are unchanged. The validator allows one or two digests, so no script changes.

2. Ten discarded policy install results in app_tests.rs

RbacState::install_revision_snapshot has returned Result<(), CompileError> since f3db892 ("Install the compiled kernel beside every RBAC snapshot"). Nine PostgreSQL-backed tests and cluster_suggestions_harness called it as a statement and dropped the result. If an install had been refused, the state would have stayed on its unkeyed startup snapshot, and the test would have gone on to check that state instead, or passed vacuously. Each call now ends in .expect("the authority's revision should install"), the same pattern the RBAC unit tests use for this call.

Line on main Caller
31171 cluster_policy_endpoints_commit_versions_and_reject_stale_preconditions
31427 a_commit_that_lands_mid_pass_moves_the_watermark_not_just_the_content
31494 admin_authorization_never_serves_under_a_stale_revision
31589 revision_gate_reconciles_to_another_replicas_commit_and_fails_closed
31997 a_tools_revision_does_not_livelock_the_security_gate
32252 a_resource_registered_after_the_watermark_advanced_is_reconciled
32339 the_background_pass_outlives_the_request_deadline
32562 a_connection_commit_reconciles_after_another_resource_moved_the_counter
32692 tools_reconciliation_installs_and_fails_closed
33394 cluster_suggestions_harness, used by the four cluster_discovery_tests suggestion tests

These tests need a database locator, so the test job skips them. They run in postgres-foundation. With the assertions in place, all 13 affected tests ran and passed there, on both CI runs of this PR. None of them was vacuous. That fits the code: install_revision_snapshot only fails for a negative revision (validate_authority), and every caller installs the revision that initialize/commit just returned.

3. Other test-only warnings in CI logs

I read the logs of the last green run before this PR (38666345068, #549's head). Rust compiler warnings:

  • test (1.98.1), production-compiler (1.99.0), postgres-foundation (1.98.1, feature on) and security-coverage (nightly) all printed exactly these 10 unused_must_use warnings, from gateway (bin "gateway" test), and no other Rust warnings: none in the tests/ha/* targets, the transport_guard example or the build script. clippy and the --no-default-features --all-targets Clippy step printed none.
  • Other warnings, none of them from the compiler on test code, so out of scope here:
    • every job: ##[warning]Node.js 20 is deprecated for the pinned actions/setup-node@49933ea… (v4) in the build-tools action;
    • security-coverage: cargo-llvm-cov's --branch option is unstable, and rustup's note that RUSTUP_TOOLCHAIN overrides the default toolchain;
    • postgres-foundation: the PostgreSQL container's own trust authentication and there is no transaction in progress server log lines.

On both CI runs of this PR (CI compiler 1.99.0 first, then 1.98.1), test, production-compiler and postgres-foundation printed no Rust warnings at all, and neither did security-coverage on the second run.

4. One new gate: RUSTFLAGS=-D warnings on the test job

Why these warnings were never caught: cargo clippy --workspace --locked -- -D warnings lints only non-test targets, and the postgres-foundation lint uses --all-targets --no-default-features, which leaves out every postgres-gated test. Nothing failed on warnings in test code built with default features.

The test job already compiles every test target with default features using the CI compiler, so RUSTFLAGS: -D warnings (set at job level) closes that gap without another build.

  • Cheap. It adds no job and no build. The variable is job-level, so Swatinem/rust-cache includes it in the cache key. The only cost is one cold build of the test job. On this PR's first run, which was cold anyway because the compiler changed, the job took 4m31s.
  • Warning-free. Once the 10 warnings are fixed, the job has nothing left to deny on 1.98.1 or 1.99.0 (see section 3 and the CI evidence below). Registry dependencies still build with --cap-lints, so only workspace code can fail the gate.
  • Placement. It goes on the job that uses the CI compiler, which already owns the enforced lint set, not on production-compiler. That way a production-image bump still does not change which lints are enforced. That separation is the reason the two pins are kept apart.
  • Tradeoff. It only covers rustc warnings, not Clippy lints. Clippy with --all-targets and default features would be stricter, but nobody has measured how many lints it would raise in the test code, so it does not qualify as warning-free. I left it out. Any rustc warning a future rust_ci bump introduces in test code now has to be fixed in that bump PR, the same as Clippy lints already must be. AGENTS.md shows the matching local command.

Follow-up: the actual rust_ci move, once this merges

I have prepared this change locally on top of this PR's head and checked it there; it is not pushed. It contains:

  • build-tools.json: rust_ci 1.99.0. rust_production and the dated coverage nightly are unchanged.
  • scripts/benchmarks/pure_paths.rs: the try_update rename. The digest is f0c6abba…, as pre-approved here.
  • transport-ownership.json: the harness's standalone_benchmarks sha256 becomes f0c6abba…, keeping owner and purpose. The follow-up should regenerate it with python scripts/transport_guard.py inventory --standalone-benchmark scripts/benchmarks/pure_paths.rs, which needs a gateway build (Node 26.10.0).
  • microbench-budget.json: retire the old digest. The docs allow this in the second change.
  • scripts/test_build_tools.py: test_floating_compiler_fails replaces the literal "1.98.1". Once both Rust pins are "1.99.0", that literal would match both keys, so the fixture floats rust_ci by key and reads its current value, as the Docker drift fixture already does.
  • docs/deployment/dependency-controls.md and AGENTS.md: give the CI compiler, and note the pins can differ rather than claiming they currently do.
  • Every other consumer reads the pin from build-tools.json: the build-tools action through build_tools.py outputs, plus the microbench, security corpus and mutation runners. The 1.98.1 in pure-path-benchmarks.md and security-mutations.md records the compiler those dated measurements used, so it stays.

Local check of that follow-up: python scripts/microbench.py run --base <this PR head> --head <follow-up head> with Rust 1.99.0 passed. Allocation counts were identical on both sides, and identical to this PR's own 1.98.1 comparison and the documented initial record: request_path 0/0, path_prefix 0/0, rule_path 87,000 calls/5,286,000 bytes, egress_host 31,000/1,118,000.

Local verification (Windows, Python 3.12; no gateway compile, because gateway/build.rs requires Node 26.10.0)

  • python scripts/microbench.py run --base origin/main --head <this PR head> with Rust 1.98.1: passed.
  • python -m unittest test_microbench test_microbench_process test_build_tools (from scripts/): 63 tests OK.
  • python scripts/build_tools.py check: "Build-tool contract verified."
  • Workflow-structure suites that read ci.yml (test_security_corpus_workflows, test_security_gates, test_security_mutations_workflows, test_image_publication, test_image_evidence, test_candidate_scan) pass. The five local failures in test_alert_rules/test_microbench_workflow are Windows-only: POSIX file modes, and /bin/bash not finding python.
  • cargo fmt --all --check: clean.

CI evidence

Run 38671589558 on head d0b9707a375cbb69ee7393510f160cdf7e7fab45: every check passes. That includes all 27 required contexts: fmt, clippy, test, postgres-foundation, admin-ui, cloudflare, egress-only, audit, gitleaks, connections-acceptance, dev-traffic-smoke, ha-compose-example, all 8 ha-release-gate legs, security-coverage, supply-chain, both cargo-policy legs, npm-policy-windows, production-compiler and image-publication. microbench, security-corpus, operational-alerts and Build image also pass. The image-candidate, scan, verification and promotion jobs are skipped, as they are on every PR.

  • test (Rust 1.98.1, RUSTFLAGS: -D warnings): passed in 6m25s, with no Rust warnings.
  • postgres-foundation: passed. All 13 tests behind the ten fixed calls ran and passed. No Rust warnings.
  • production-compiler (Rust 1.99.0) and security-coverage (nightly-2026-09-01, floors unchanged): passed, with no Rust warnings.
  • microbench (Rust 1.98.1, current harness): {"errors": [], "status": "passed"}. The base budget comes from main, and the head's proposed two-digest budget validated.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-15T02:16:50.043474Z 6419d15 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Moving rust_ci to Rust 1.99.0 breaks the microbench comparison before it
measures anything. Rust 1.99.0 deprecates `AtomicU64::fetch_update` in
favour of `try_update`, and the controller compiles the standalone
harness with `-D warnings`:

    error: use of deprecated method
      `std::sync::atomic::Atomic::<u64>::fetch_update`:
      renamed to `try_update` for consistency
      --> scripts/benchmarks/pure_paths.rs:76:10

The harness has to change, and the comparison takes its accepted
harness digests from the base commit's budget, so it cannot change in
the same PR that approves it. This is the first of the two reviewed
changes docs/performance/pure-path-benchmarks.md prescribes: add the
next digest beside the current one and leave the active harness alone.

The new digest, f0c6abba555cb81e8cc419f39473895eeefb3ad558537f95659dfcdfcf22285e,
is the SHA-256 of the current harness with that one call renamed
(`.fetch_update(` to `.try_update(`, same arguments and closure). It is
rustfmt-clean, and `try_update` is stable on both Rust 1.98.1 and 1.99.0,
so the harness compiles under -D warnings on either. The follow-up
change makes the rename, moves rust_ci to 1.99.0, updates the harness's
transport-ownership record, and may retire the old digest. Budgets and
thresholds are unchanged.
`RbacState::install_revision_snapshot` returns
`Result<(), CompileError>` since the compiled kernel started being
installed beside every RBAC snapshot, and refuses a candidate it cannot
compile while the previous snapshot keeps serving. Nine PostgreSQL-backed
tests and the cluster rule-suggestion harness in app_tests.rs still
called it as a statement and discarded that result, which every Rust
test build reported as an `unused_must_use` warning.

Each of those callers installs the authority's boot revision first and
then asserts behaviour that depends on it: policy endpoint versioning,
stale-revision refusal, revision-gate reconciliation across policy, tools
and connections, and the suggestions harness's keyed snapshot. A refused
install would have left the state on its unkeyed startup snapshot, and
the test would have gone on to measure that state instead, or passed
vacuously. They now fail at the install, as the RBAC unit tests already
do for the same call.
The ten discarded install results fixed in the previous commit were
reported by every Rust test build for as long as they existed, and no
required job failed on them. The two Clippy gates do not compile that
code: `cargo clippy --workspace` lints non-test targets, and the
postgres-foundation lint passes `--all-targets` with
`--no-default-features`, which leaves out every `postgres`-gated test.
Test code built with default features had no warning gate at all.

Set `RUSTFLAGS=-D warnings` on the test job. It already compiles every
test target with default features using the CI compiler, so the gate
adds no build, and it sits with the compiler that owns the enforced lint
set rather than the production-compiler job, whose image updates are
deliberately reviewed without lint changes. Registry dependencies are
still built with `--cap-lints`, so only workspace code can fail it. On
this branch the build is warning-free; the only warnings in the main
test, production-compiler, postgres-foundation and coverage logs were
those ten.

It is job-level so Swatinem/rust-cache folds it into its key instead of
restoring artifacts built without it. AGENTS.md lists the matching local
command.
@Antnation
Antnation force-pushed the chore/rust-ci-1.99-and-test-results branch from 6419d15 to d0b9707 Compare October 15, 2026 02:33
@Antnation Antnation changed the title Run CI on Rust 1.99.0 and stop discarding policy install results in tests Prepare the Rust 1.99.0 CI move and stop discarding policy install results in tests Oct 15, 2026
Antnation added a commit that referenced this pull request Oct 16, 2026
#557 denies compiler warnings for the required test job through job-level
RUSTFLAGS, and #563's workflow contract refused any job-level env there,
so the two PRs failed supply-chain together. The contract now pins that
job's env to exactly {RUSTFLAGS: -D warnings} instead of forbidding it,
which still keeps any other variable away from the registry steps.
@Antnation
Antnation merged commit 0aa79b6 into main Oct 16, 2026
35 checks passed
@Antnation
Antnation deleted the chore/rust-ci-1.99-and-test-results branch October 16, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant