Repository navigation
Benchmark the authoritative HTTP policy decision with allocation counts - #571
Conversation
… counts Work in progress, parked when the implementing session stopped. Not yet compiled by CI or reviewed. Part of #437.
Base and head share one Cargo target directory so dependencies compile once. Cargo names a workspace member's artifacts without its source path, so the head tree was judged fresh and the head 'build' finished in 0.2 s with the base revision's binary. That binary has no benchmark test, so discovery listed zero tests and the run failed. Worse, a base that also had the harness would have been compared with itself. Each revision now runs cargo clean -p gateway before its build, so only the gateway crate recompiles, and the built test binary is copied into a per-revision work directory so the next clean cannot remove it. A process test pins the clean-then-build order and the per-revision copies.
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bcb3fc2253
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # Cargo's default test profile, without debug info: allocation behaviour is | ||
| # unchanged and both builds fit the job's time limit. | ||
| CARGO_PROFILE_ENV = {"CARGO_INCREMENTAL": "0", "CARGO_PROFILE_DEV_DEBUG": "0"} |
There was a problem hiding this comment.
Pin the actual test profile for both revisions
The archived revisions retain their own workspace profile settings, while this environment only overrides profile.dev.debug; cargo test can therefore compile base and head with different [profile.test] options. For example, a head commit adding [profile.test] opt-level = 3 is measured optimized against an unoptimized base, yet both envelopes record the same cargo_env and pass the context compatibility check, so optimization-induced allocation changes can hide a regression or create a false failure. Override and record the complete test profile, or reject differing manifest profile configuration.
Useful? React with 👍 / 👎.
Part of #437 (PR 3, harness step of the documented two-step harness/budget transition).
gateway/tests/support/policy_benchmark.rs: an ignored test that counts allocations ofCompiledPolicy::evaluateover a fixed synthetic policy and request set. The counting global allocator exists only in thecfg(test)binary (main.rswires the module under#[cfg(test)]), so production builds are unaffected.scripts/microbench.pybuilds both revisions, checks test discovery against source wiring, and compares allocation counts. The microbench job now has Node, becausebuild.rsembeds the UI in the test binary.cargo clean -p gatewaybefore building, and measures a private copy of its binary. A process test pins this.No budget numbers change in this PR. CI is green on all required checks.