Repository navigation
[Fix] dspark: Draw the simulated-acceptance bonus at the simulated row - #46404
Merged
kpham-sgl merged 1 commit intoOct 27, 2026
Merged
Conversation
Root cause: SGLANG_SIMULATE_ACC_LEN replaces only correct_len after the accept. The bonus stays the one the accept computed for its own length: the argmax (or the rejection-sampled token) at the real first-mismatch row. It is committed after drafts[:c], at a position the target never predicted it for, and anchors the next draft block, so the committed text degenerates. Affected paths: the eager accept (every batch unless SGLANG_SIMULATE_ACC_GREEDY=1 with a temperature-only batch), the folded greedy accept and the fused static accept tail without SGLANG_DSPARK_SIMULATE_FOLD, and the sampling-accept region graph. Fix: - Worker: under simulation every sampled batch passes its sampling info for the bonus; SGLANG_SIMULATE_ACC_GREEDY only switches draft/accept to greedy. - build_accept_outs: without a sampled bonus, the bonus is the argmax of the simulated row. - sample_simulated_bonus: top-k/top-p batches sample the simulated row from build_speculative_verify_target_probs, the sampling accept's target distribution; temperature-only batches keep the Gumbel kernel. - Folded greedy accept and the fused tail read the simulated row. - Sampling-accept region graph: stays eligible under simulation for the same batches as real serving; the worker stages temperatures and seed with begin_sim_bonus and the region draws the bonus at the simulated row with the epilogue's graph-safe sampler (sim_bonus, was _sim_bonus). - A sampled simulated bonus uses the DSPARK_ACCEPT_SAMPLE TP sync site. Validation: see PR body.
weireweire
requested review from
BBuf,
DarkSharpness,
HaiShaw,
HydraQYH,
Qiaolin-Yu,
Ying1123,
celve,
hnyls2002,
merrymercy and
yuan-luo
as code owners
October 25, 2026 09:02
4 of 5 tasks
Contributor
|
[by Claude Code] Thanks for this. It matches what we saw: under simulated acceptance our DSpark sanity replies were degenerate, while real-acceptance replies were coherent. Two small suggestions:
|
Collaborator
|
/rerun-test test_dspark_accept_tail.py test_dspark_sampling_accept_graph.py test_dspark_sim_draw_ahead.py test_dspark_kernel_parity.py |
Contributor
|
🚀 🚀 |
kpham-sgl
approved these changes
Oct 27, 2026
weireweire
added a commit
to weireweire/sglang
that referenced
this pull request
Oct 27, 2026
… pass Root cause: sgl-project#41593 keeps waiting requests' device prefixes warm by touching them in the radix LRU, but on a PD prefill worker it walks only `waiting_queue`. Requests still in `disagg_prefill_bootstrap_queue`, waiting for decode to pre-allocate their slots, are never touched. When decode is full, the bootstrap queue is the deep one, and those prefixes age out while the requests wait. With DEP2 prefill (9 workers) + DEP16 decode serving an agentic multi-turn trace at 7480 concurrent sessions, the bootstrap queue held ~300 requests per rank for ~100 s. Device-radix hits fell from ~65 % to ~22 % of prompt tokens, and the extra external (Mooncake L3) reloads slowed prefill further, so prefill collapsed after the client's turn burst and did not recover. Fix: `calc_priority` takes the PD prefill bootstrap queue, and on the same refresh pass `_touch_waiting_prefixes` touches up to WAITING_PREFIX_REFRESH_BOOTSTRAP_MAX (512, a module constant next to the existing waiting-queue bounds) bootstrap entries before the waiting queue, in reverse order. SGLANG_ENABLE_WAITING_PREFIX_REFRESH=0 still turns the whole refresh off. The existing gate is unchanged: cache-agnostic policy, LRU eviction, not decode, no fast match, prefix sharing supported. Bootstrap entries are refreshed only on passes that reach `calc_priority`. The caller returns earlier when the waiting queue is empty or no request rows are free, so the bootstrap queue is not refreshed on its own. With more than 128 waiting entries, the entries past the first 128 are still not touched and can end up older than the refreshed bootstrap entries. Validation: GB300, DSv4.1-Flash, 9 DEP2 prefill + DEP16 decode, 7480 concurrent sessions, 1200 s window, SGLang main at 6563312 plus open PRs sgl-project#46497/sgl-project#46593/sgl-project#46592/sgl-project#46404 on both arms: without: 361,815 tok/s/GPU @ p90 interactivity 145.0, TTFT p90 135.9 s; L3 73 % / device 22 % of prompt tokens in minutes 8-20. with: 560,412 tok/s/GPU @ p90 interactivity 161.0, TTFT p90 58.9 s; L3 21 % / device 74 %. test_scheduler_held_row_admission's Scheduler stub gains `disagg_prefill_bootstrap_queue = None` for the new argument.
4 of 5 tasks
weireweire
added a commit
to SemiAnalysisAI/InferenceX
that referenced
this pull request
Oct 28, 2026
…ned low-latency settings Use nightly-dev-cu13-20261028-586355a8, which includes sgl-project/sglang#46404 and #46891. TP4 c1 runs as the batch-1 NVIDIA megakernel point of the Dynamo + SGLang aggregated row, without SIM_DRAW_AHEAD so the simulated step does the same work as a real one. TP4 c4/c16 and TP2/EP2 c16/c32 pin the flashmla attention backend instead of auto (MegaAttention pads per-rank heads to 64).
weireweire
added a commit
to SemiAnalysisAI/InferenceX
that referenced
this pull request
Oct 28, 2026
…ightly and drop TP2/EP2 c16 nightly-dev-cu13-20261028-586355a8 includes sgl-project/sglang#46404. Raise the Dynamo 1.7 backend admission limits as on the disagg rows. TP2/EP2 c16 sits far below the line from TP4 c16 to the disaggregated points, so it is removed with its override.
4 of 5 tasks
weireweire
added a commit
to SemiAnalysisAI/InferenceX
that referenced
this pull request
Oct 28, 2026
… 20261028 nightly and retire the Mooncake-linker disagg key The hicache key now runs on nightly-dev-cu13-20261028-586355a8, which includes sgl-project/sglang#46404. On that image the KV pool sizes at 890 B/token, so hicache-ratio drops to 1.2 (1.8 exceeds host memory) and decode mem-fraction-static to 0.75 (0.85 runs out of memory at graph capture). Points: 5P c2884 (bs40), 8P c4096 (bs64) and 10P c7480 (bs160). The 9P and 10P bs128 c7480 points are dropped; bs160 is higher at the same interactivity. The dspark-disagg key on 65633129 and its recipe are retired.
cquil11
added a commit
to SemiAnalysisAI/InferenceX
that referenced
this pull request
Oct 30, 2026
…-fixed nightlies with prefill-HiCache disagg points and TP4 c1 megakernel / [NVIDIA][AgentX] GB300 DSV4.1-Flash Dynamo + SGLang 升级到含 bonus 修复的 nightly,分离式改用 prefill HiCache,TP4 c1 启用 megakernel (#4595) * Add GB300 DSV4.1-Flash SGLang DEP16 disagg c7480 points with prefill HiCache Root cause: at c7480 the Mooncake external linker reads L3 hits layer by layer on the forward path; after the AgentX turn burst, prefill stalls on those reads (about 40 % of wall time), device prefixes age out, and the deployment does not recover. Fix: a new config key on nightly-dev-cu13-20261027-5884e4fe (Dynamo 1.7.0.dev20261026) with 9 and 10 DEP2 prefill workers and one DEP16 decode. Prefill keeps evicted prefixes in a local host-DRAM HiCache tier (ratio 1.8, write-back, direct I/O); decode keeps SGLang's default polling interval of 1. Dynamo 1.7 backend admission limits are raised so turns queue in the scheduler, and NVIDIA/srt-slurm#629 is backported so the readiness wait asks /health?instances=true when /health is status-only. The agg and dspark-disagg keys are listed so they rerun in the same sweep. Validation: on GB300 with the same recipe and AgentX client, 10 x DEP2 batch 128 gave 924,652 tok/s/GPU at p90 117.2 tok/s/user over 3600 s. * Set perf-changelog pr-link to #4595 * Restore historical perf-changelog entries; set only this entry's pr-link * Move the GB300 DSV4.1-Flash Dynamo + SGLang agg row to the 20261028 nightly and drop TP2/EP2 c16 nightly-dev-cu13-20261028-586355a8 includes sgl-project/sglang#46404. Raise the Dynamo 1.7 backend admission limits as on the disagg rows. TP2/EP2 c16 sits far below the line from TP4 c16 to the disaggregated points, so it is removed with its override. * Move the GB300 DSV4.1-Flash Dynamo + SGLang HiCache disagg key to the 20261028 nightly and retire the Mooncake-linker disagg key The hicache key now runs on nightly-dev-cu13-20261028-586355a8, which includes sgl-project/sglang#46404. On that image the KV pool sizes at 890 B/token, so hicache-ratio drops to 1.2 (1.8 exceeds host memory) and decode mem-fraction-static to 0.75 (0.85 runs out of memory at graph capture). Points: 5P c2884 (bs40), 8P c4096 (bs64) and 10P c7480 (bs160). The 9P and 10P bs128 c7480 points are dropped; bs160 is higher at the same interactivity. The dspark-disagg key on 65633129 and its recipe are retired. * Use 9 prefill workers for the c7480 HiCache point At decode batch 160, 9 x DEP2 reaches the same p90 interactivity as 10 x DEP2 with two fewer GPUs, so its throughput per GPU is higher. * Name the KV layout, precision and Engram changes against the retired recipe in the changelog * Move the GB300 DSV4.1-Flash Dynamo + SGLang agg row to the 20261029 nightly and use the megakernel runtime-optimization switch at TP4 c1 nightly-dev-cu13-20261029-c10e0e17 bundles sgl-nvidia-mega 54b428921749 (megakernel API 2, sgl-project/sglang#47145), so TP4 c1 starts with --enable-nvidia-megakernel again. SGLANG_ENABLE_NV_MEGA_RUNTIME_OPTIMIZATION (sgl-project/sglang#47105) replaces the 16 runtime-optimization variables the recipe set one by one. * Drop the srt-slurm #629 backport patch now that main pins srt-slurm v2.65.2 Main bumped inferencex-e2e/utils/srt-slurm to v2.65.2 (#4601), which includes NVIDIA/srt-slurm#629, so runners/srt-slurm/patches/629-dynamo-health-instances.patch and its README row are removed and the changelog entry no longer mentions it. * chore: refresh PR #4595 for sweep reuse [skip-sweep] --------- Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com> Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com>
cquil11
added a commit
to SemiAnalysisAI/InferenceX
that referenced
this pull request
Oct 30, 2026
…6 and 3P1D c1228 points (append-only, follows #4595) / [NVIDIA][AgentX] 追加 GB300 DSV4.1-Flash Dynamo + SGLang TP2/EP2 c16 和 3P1D c1228 点(仅追加,接续 #4595) (#4679) * Add GB300 DSV4.1-Flash SGLang DEP16 disagg c7480 points with prefill HiCache Root cause: at c7480 the Mooncake external linker reads L3 hits layer by layer on the forward path; after the AgentX turn burst, prefill stalls on those reads (about 40 % of wall time), device prefixes age out, and the deployment does not recover. Fix: a new config key on nightly-dev-cu13-20261027-5884e4fe (Dynamo 1.7.0.dev20261026) with 9 and 10 DEP2 prefill workers and one DEP16 decode. Prefill keeps evicted prefixes in a local host-DRAM HiCache tier (ratio 1.8, write-back, direct I/O); decode keeps SGLang's default polling interval of 1. Dynamo 1.7 backend admission limits are raised so turns queue in the scheduler, and NVIDIA/srt-slurm#629 is backported so the readiness wait asks /health?instances=true when /health is status-only. The agg and dspark-disagg keys are listed so they rerun in the same sweep. Validation: on GB300 with the same recipe and AgentX client, 10 x DEP2 batch 128 gave 924,652 tok/s/GPU at p90 117.2 tok/s/user over 3600 s. * Set perf-changelog pr-link to #4595 * Restore historical perf-changelog entries; set only this entry's pr-link * Move the GB300 DSV4.1-Flash Dynamo + SGLang agg row to the 20261028 nightly and drop TP2/EP2 c16 nightly-dev-cu13-20261028-586355a8 includes sgl-project/sglang#46404. Raise the Dynamo 1.7 backend admission limits as on the disagg rows. TP2/EP2 c16 sits far below the line from TP4 c16 to the disaggregated points, so it is removed with its override. * Move the GB300 DSV4.1-Flash Dynamo + SGLang HiCache disagg key to the 20261028 nightly and retire the Mooncake-linker disagg key The hicache key now runs on nightly-dev-cu13-20261028-586355a8, which includes sgl-project/sglang#46404. On that image the KV pool sizes at 890 B/token, so hicache-ratio drops to 1.2 (1.8 exceeds host memory) and decode mem-fraction-static to 0.75 (0.85 runs out of memory at graph capture). Points: 5P c2884 (bs40), 8P c4096 (bs64) and 10P c7480 (bs160). The 9P and 10P bs128 c7480 points are dropped; bs160 is higher at the same interactivity. The dspark-disagg key on 65633129 and its recipe are retired. * Use 9 prefill workers for the c7480 HiCache point At decode batch 160, 9 x DEP2 reaches the same p90 interactivity as 10 x DEP2 with two fewer GPUs, so its throughput per GPU is higher. * Name the KV layout, precision and Engram changes against the retired recipe in the changelog * Move the GB300 DSV4.1-Flash Dynamo + SGLang agg row to the 20261029 nightly and use the megakernel runtime-optimization switch at TP4 c1 nightly-dev-cu13-20261029-c10e0e17 bundles sgl-nvidia-mega 54b428921749 (megakernel API 2, sgl-project/sglang#47145), so TP4 c1 starts with --enable-nvidia-megakernel again. SGLANG_ENABLE_NV_MEGA_RUNTIME_OPTIMIZATION (sgl-project/sglang#47105) replaces the 16 runtime-optimization variables the recipe set one by one. * Drop the srt-slurm #629 backport patch now that main pins srt-slurm v2.65.2 Main bumped inferencex-e2e/utils/srt-slurm to v2.65.2 (#4601), which includes NVIDIA/srt-slurm#629, so runners/srt-slurm/patches/629-dynamo-health-instances.patch and its README row are removed and the changelog entry no longer mentions it. * Add back GB300 DSV4.1-Flash Dynamo+SGLang TP2/EP2 c16 and 3P1D c1228 points (append-only) Fill the interactivity/E2EL gap between agg TP4 c16 and disagg 5P1D c2884 on each config's existing image: agg TP2/EP2 c16 with its previous per-point settings, and a 3 x DEP2 / 1 x DEP16 c1228 point on the prefill HiCache disagg recipe with the previous c1228 decode sizing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Set the append-only changelog pr-link to #4679 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore: drop history clauses from DSV4.1-Flash hicache decode comments --------- Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com>
nvpohanh
added a commit
to SemiAnalysisAI/InferenceX
that referenced
this pull request
Nov 3, 2026
…ly with the tuned low-latency settings and TP4 c1 megakernel / [NVIDIA][AgentX] GB300 DSV4.1-Flash SGLang 升级到 20261030 nightly,采用调优后的低延迟配置并在 TP4 c1 启用 megakernel (#4603) * Update GB300 DSV4.1-Flash SGLang AgentX image to nightly-dev-cu13-20261025-65633129 Move dsv41flash-fp4-gb300-sglang-agentic-dspark from nightly-dev-cu13-20261021-c1c287a4 to the digest-pinned nightly-dev-cu13-20261025-65633129, the image the Dynamo + SGLang agg key already uses. Recipes and search space are unchanged. * Set perf-changelog pr-link to #4603 * Move GB300 DSV4.1-Flash SGLang AgentX to 20261028 nightly with the tuned low-latency settings Use nightly-dev-cu13-20261028-586355a8, which includes sgl-project/sglang#46404 and #46891. TP4 c1 runs as the batch-1 NVIDIA megakernel point of the Dynamo + SGLang aggregated row, without SIM_DRAW_AHEAD so the simulated step does the same work as a real one. TP4 c4/c16 and TP2/EP2 c16/c32 pin the flashmla attention backend instead of auto (MegaAttention pads per-rank heads to 64). * Keep SIM_DRAW_AHEAD on the TP4 c1 megakernel point The gate only moves the simulated-acceptance draw, a host-side sample plus a fill that real acceptance never runs, ahead of the draft launch. It does not skip work that a real-acceptance step performs. * Keep TP4 c1 off the megakernel until the image ships a megakernel API 2 plugin nightly-dev-cu13-20261028-586355a8 bundles sgl-nvidia-mega 6fd7ac30 (megakernel API 1), while its SGLang requires API 2 since sgl-project/sglang#45266, so --enable-nvidia-megakernel fails at startup. TP4 c1 keeps its previous engine settings and pins flashmla like the other TP-sharded points. * Move the GB300 DSV4.1-Flash SGLang curve to the 20261029 nightly and run TP4 c1 on the megakernel nightly-dev-cu13-20261029-c10e0e17 bundles sgl-nvidia-mega 54b428921749 (megakernel API 2), so TP4 c1 returns to its megakernel config. The 16 runtime-optimization variables it set one by one are replaced by SGLANG_ENABLE_NV_MEGA_RUNTIME_OPTIMIZATION (sgl-project/sglang#47105). * Move the GB300 DSV4.1-Flash SGLang key to the 20261030 nightly for the simulated verify accept cost fix Root cause: nightly-dev-cu13-20261029-c10e0e17 predates sgl-project/sglang#46840, so simulated acceptance skipped the real verify accept cost and the TP4 c1 point was not measured on the methodology-conformant path. Fix: pin lmsysorg/sglang:nightly-dev-cu13-20261030-302b78cc by digest. It is 63 commits past the #46840 merge and bundles the same sgl-nvidia-mega 54b428921749 (megakernel API 2). Update this PR's changelog entry to name the image and #46840. Validation: Dynamo + SGLang TP4 c1 megakernel, c4 and c16 started and completed on 302b78cc with target and draft megakernels active on all four ranks. --------- Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com> Co-authored-by: Po-Han Huang <pohanh@nvidia.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
3 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
With
SGLANG_SIMULATE_ACC_LENset, DSpark replaces the accepted length with the simulated valuec. The bonus token, however, stays the one the real accept computed for its own length: the target argmax at the real first-mismatch row, or the rejection-sampled token at the real reject row. That token is committed afterdrafts[:c], at a position the target never predicted it for. It is also the anchor of the next draft block. The committed text degenerates as a result, and verify batches route to fewer experts than at the same acceptance length under real verification. Simulated-acceptance throughput is therefore overstated.The only configuration that was correct is
SGLANG_SIMULATE_ACC_GREEDY=1with temperature-only requests, which has been opt-in since #45433. EAGLE/MTP and DFlash already take the bonus from the target at the forced row.Modifications
Under simulation, the bonus is the target's token at verify row
c, drawn with the request's sampling law.dspark_worker_v2.py: every sampled batch now passes its sampling info assimulate_bonus_sampling_info. Before, onlySGLANG_SIMULATE_ACC_GREEDY=1with temperature-only batches did. The flag now only selects the greedy draft/accept, under the same eligibility checks as before.SGLANG_OPT_DSV41_SAMPLING_ACCEPT_GRAPH): it stays eligible under simulation for the same batches as in real serving (all-sampled, temperature-only, no logit adjustments or grammar), so a simulated run keeps the graph that real serving uses.begin_sim_bonus.cwith the epilogue's graph-safe sampler (sim_bonus, renamed from_sim_bonus), the same kernelSGLANG_DSPARK_SIMULATE_FOLDuses.build_accept_outs: when no sampled bonus is supplied, the bonus is the argmax of rowc. This covers greedy batches on the eager accept and on the folded greedy accept (DsparkVerifyEpilogue._acceptwithoutSGLANG_DSPARK_SIMULATE_FOLD).sample_simulated_bonus: top-k/top-p batches sample rowcfrombuild_speculative_verify_target_probs, the target distribution the sampling accept uses. Greedy requests in a mixed batch have top_k=1, so they stay argmax. Temperature-only batches keep the existing Gumbel kernel.static_accept_tail(Triton kernel and torch reference): the argmax-id bonus is read at the committed length rather than the matched length.DSPARK_ACCEPT_SAMPLETP sync site, so therngpreset keeps ranks consistent.test_dspark_simulated_bonus.py: the sampling-info stand-in gainsneed_top_k_sampling/need_top_p_sampling.Scope of the sampling law under simulation:
run_non_compact/run_compactapply the adjustments and the worker applies the grammar mask. The sampler itself does neither.Simulated acceptance is a benchmark-only mode.
TP: a sampled bonus is broadcast from rank 0 when the
DSPARK_ACCEPT_SAMPLEsite is enabled. Without it, ranks draw from identical seeds, the same assumption the simulated-length draw and the non-simulated rejection sampler already make.Non-simulated paths are unchanged. Every new branch is behind
simulated_correct_len is not None, and the tail kernel's non-simulated length is stillnum_correct.Accuracy / correctness
Unit-level checks on GB300 run the real kernels, with drafts rejected at row 0 and simulated
c = 3:cdraw; drafts at[:c], bonus at[c]c/ T)Existing tests, before and after:
test_dspark_accept_tail.py,test_dspark_sampling_accept_graph.pyandtest_dspark_sim_draw_ahead.pypass.test_dspark_simulated_bonus.py::test_bf16_full_vocab_low_temperaturefails identically with and without this change on NVIDIA. It is registered for AMD CI only.End to end: DeepSeek-V4.1-Flash FP4, DSpark block 5, TP2 EP2, GB300, AgentX-style agentic traces at 32 concurrent, temperature 1,
SGLANG_SIMULATE_ACC_LEN=3.51(match-expected). The committed text is re-tokenized; about 1,500 turns are shared across runs.After the fix, the remaining difference from real acceptance is in the drafted tokens. Simulation accepts them unconditionally, so repeats 2-3 tokens apart rise from 0.027 to 0.096 of tokens. Immediate repeats (0.0008 vs 0.0024) and repeats 4-6 tokens apart match real acceptance. An independent simulated-acceptance implementation that commits the target's token at the forced position measures the same statistics on this workload (0.148 / 0.182 / 0.087).
Benchmarking and Profiling
The size of the change depends on the point: it is largest where MoE is a big share of the step and the degenerate text cut the distinct experts the most.
TP2 EP2, 32 concurrent (the end-to-end setup above), 3600 s:
Both runs measure AL 3.51. The "after" run is 2.7% lower in total throughput and 11% lower in p90 interactivity (11% higher p90 ITL). This is expected: the "before" run's degenerate text made MoE about 1 ms/step cheaper at B=4-10. Each number is a single run, on different nodes. These two runs applied the same change to an earlier nightly and use the eager accept (no sampling-accept graph).
TP4 EP1, 1 concurrent, NVIDIA megakernel,
SGLANG_OPT_DSV41_SAMPLING_ACCEPT_GRAPH=1,SGLANG_SIMULATE_ACC_LEN=3.51, same traces, 1800 s, AL 3.51 in every run. At batch 1 with the experts sharded across four GPUs, MoE is a small share of the step, so the change costs about 1%. This table also shows why the sampling-accept region change is needed:Keeping the region in the graph is worth 4% / 10% at p50 / p90. The remaining ~1% to "before" is within what the text change and the shorter run can explain.
Checklist
CI States
Latest PR Test (Base): ❌ Run #40179008917
Latest PR Test (Extra): ❌ Run #40179008912
Latest PR Test (AMD ROCm 10): ❌ Run #40179008872