Skip to content

Benchmark the authoritative HTTP policy decision with allocation counts - #571

Merged
Antnation merged 2 commits into
mainfrom
feat/437-kernel-http-benchmark
Oct 20, 2026
Merged

Antnation merged 2 commits into
mainfrom
feat/437-kernel-http-benchmark

Conversation

@Antnation

@Antnation Antnation commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Part of #437 (PR 3, harness step of the documented two-step harness/budget transition).

  • gateway/tests/support/policy_benchmark.rs: an ignored test that counts allocations of CompiledPolicy::evaluate over a fixed synthetic policy and request set. The counting global allocator exists only in the cfg(test) binary (main.rs wires the module under #[cfg(test)]), so production builds are unaffected.
  • scripts/microbench.py builds both revisions, checks test discovery against source wiring, and compares allocation counts. The microbench job now has Node, because build.rs embeds the UI in the test binary.
  • Harness bug fixed (bcb3fc2): base and head share one Cargo target directory, and Cargo names a workspace member's artifacts without its source path. The head 'build' therefore reused the base binary in 0.2 s, and the benchmark compared base with itself. Each revision now runs cargo clean -p gateway before building, and measures a private copy of its binary. A process test pins this.

No budget numbers change in this PR. CI is green on all required checks.

… counts

Work in progress, parked when the implementing session stopped. Not yet compiled by CI or reviewed. Part of #437.
Base and head share one Cargo target directory so dependencies compile
once. Cargo names a workspace member's artifacts without its source path,
so the head tree was judged fresh and the head 'build' finished in 0.2 s
with the base revision's binary. That binary has no benchmark test, so
discovery listed zero tests and the run failed. Worse, a base that also had
the harness would have been compared with itself.

Each revision now runs cargo clean -p gateway before its build, so only
the gateway crate recompiles, and the built test binary is copied into a
per-revision work directory so the next clean cannot remove it. A process
test pins the clean-then-build order and the per-revision copies.
@Antnation Antnation changed the title WIP: Benchmark the authoritative HTTP policy decision with allocation counts Benchmark the authoritative HTTP policy decision with allocation counts Oct 20, 2026
@Antnation
Antnation marked this pull request as ready for review October 20, 2026 15:01
Copilot AI balanced review requested due to automatic review settings October 20, 2026 15:01

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-20T15:05:44.018113Z bcb3fc2 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@Antnation Antnation mentioned this pull request Oct 20, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bcb3fc2253

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/microbench.py
Comment on lines +81 to +83
# Cargo's default test profile, without debug info: allocation behaviour is
# unchanged and both builds fit the job's time limit.
CARGO_PROFILE_ENV = {"CARGO_INCREMENTAL": "0", "CARGO_PROFILE_DEV_DEBUG": "0"}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pin the actual test profile for both revisions

The archived revisions retain their own workspace profile settings, while this environment only overrides profile.dev.debug; cargo test can therefore compile base and head with different [profile.test] options. For example, a head commit adding [profile.test] opt-level = 3 is measured optimized against an unoptimized base, yet both envelopes record the same cargo_env and pass the context compatibility check, so optimization-induced allocation changes can hide a regression or create a false failure. Override and record the complete test profile, or reject differing manifest profile configuration.

Useful? React with 👍 / 👎.

Antnation added a commit that referenced this pull request Oct 20, 2026
@Antnation
Antnation merged commit 1202a00 into main Oct 20, 2026
35 checks passed
@Antnation
Antnation deleted the feat/437-kernel-http-benchmark branch October 20, 2026 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants