Repository navigation
refactor(bench): share the fixed lane primitive across KV-router benchmarks - #17900
Conversation
…hmarks The Active Sequences, approximate-LRU and Mooncake open-loop benchmarks each had their own copy of the same fixed ID lane: a preallocated AtomicU32 slot array, a Release-published count, a closed flag and a Notify wakeup, plus the same notified()-then-recheck consumer wait. Move that protocol into kv_router/common/fixed_lane.rs as FixedLane, with ConsumedLane adding the consumer-progress counter used for depth() and the lane_incomplete check. Mooncake uses FixedLane, so it still has no shared consumed counter. Each caller keeps its own overflow error, publication cadence, atomic orderings and payload loop. DYN-5268 Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
|
👋 Hi glamr-agent! Thank you for contributing to ai-dynamo/dynamo. Just a reminder: The 🚀 |
|
factory: Initial validation snapshot. Validation status: complete
No benchmark performance run was made. The change makes no performance claim. Detailed evidence — validation completeLeft out of the table because CI repeats them: none. Evidence tally: [2/2 validated] Advisory AI assessment: sound. No blocking findings. Validation result: complete — pass. Every planned build, lint, and test command passed on the change. The same test targets passed on unchanged Evidence audit: complete [2/2 validated]. The command report comes from recorded runs. Commands and results [2/2 validated]Generated from the commands recorded during this run. Check 1Checks the changed Rust crates with Result: Passed ( Command: Not shown because the exact command contained private run data. Check 2Inspects the changed code when the claim cannot be tested with a local command. Result: Passed ( Command: Not shown because the exact command contained private run data. |
|
No description provided. |
|
factory: @dynamo-ops please run full CI for 33e5620 |
|
/ok to test 33e5620 |
|
No actionable comments were generated in the recent review. WalkthroughThe change adds shared fixed-capacity lane types and replaces local lane implementations in the active-sequences, approximate-LRU, and Mooncake open-loop benchmarks. The shared API handles slot publication, consumer progress, overflow, closure, and waiting. ChangesOpen-loop lane sharing
Estimated code review effort: 3 (Moderate) | ~25 minutes Merge Risk:
|
✅ Dynamo PR CI passed — run 41561563002 (attempt 1) on
|
jthomson04
left a comment
There was a problem hiding this comment.
The protocol moves over unchanged (orderings, notified() before the recheck, closed-and-drained exit). A few small suggestions inline.
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
Head branch was pushed to by a user without write access
|
nursery: Changed nothing for the docstring coverage warning in CodeRabbit’s summary. |
|
nursery: @dynamo-ops please run full CI for 49cbcd4 |
|
/ok to test 49cbcd4 |
jthomson04
left a comment
There was a problem hiding this comment.
All four addressed in 49cbcd4. LGTM.
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
|
nursery: @dynamo-ops please run full CI for 1b651ec |
|
/ok to test 1b651ec |
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
Head branch was pushed to by a user without write access
|
nursery: @dynamo-ops please run full CI for 57ee286 |
|
/ok to test 57ee286 |
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
|
nursery: @dynamo-ops please run full CI for 82ed715 |
|
/ok to test 82ed715 |
Third upstream merge for the split v2 PR: the five commits past 53015f9 that landed while the second merge was validated. Four conflicts, each placed in the owner crate, upstream behavior whole, the stack's moves intact, no shims. - ai-dynamo#18008 probe worker availability with an Fx set: WorkerIdSet and the per-set AvailableWorkerIds cache in the kernel's types.rs, the listed- worker walk in the kernel's filter.rs, the actor's cached availability in queue.rs (auto-merged); WorkerIdSet exported to hosts through dynamo_scheduler::host (lib/llm's LoRA availability filter) and to plugins through dynamo_kv_router::plugins::prelude (the builtin default_policy test stays prelude-only). - ai-dynamo#18084 CRTC stale-leaf sweep trigger scaled with tree size: lib/router/kv-indexer concurrent_radix_tree_compressed (auto-merged). - ai-dynamo#17900 shared fixed-lane bench primitive: lib/bench (auto-merged). - ai-dynamo#17947 planner priority pools, ai-dynamo#17625 beam tool-call finalization: outside the router, auto-merged. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Overview
The Active Sequences, approximate LRU, and Mooncake KV-router benchmarks each kept their own copy of one lane protocol. The protocol has fixed atomic ID slots, a release-published count, a close flag, and a wakeup. A fix to its ordering or wakeups had to be made three times. This change moves the protocol into one shared module. It does not change benchmark behavior.
Key Changes
lib/bench/kv_router/common/fixed_lane.rs, holds the lane and its close-aware wait. The sameFixedLanealso holds the consumed counter used for depth reporting.FixedLane. Active Sequences and approximate LRU record consumed progress; Mooncake never callsrecord_consumed. The approximate LRU inline producer now writes and publishes through the shared lane.PeaBrane (Yan Ru Pei) should review this change.
Validation
The three benchmark test targets pass before and after the change, including both approximate LRU end-to-end trials, which CI does not build. New unit tests for the shared lane fail when the wakeup or the closed-and-drained exit is removed. No performance run was made; the change makes no performance claim.
Issues and Ongoing Work
Closes DYN-5268.
Summary by CodeRabbit