Skip to content

refactor(bench): share the fixed lane primitive across KV-router benchmarks - #17900

Merged
jthomson04 merged 5 commits into
ai-dynamo:mainfrom
glamr-agent:dyn-5268-share-fixed-lane-primitive-kv-router-benches-66f09136db29
Nov 5, 2026
Merged

jthomson04 merged 5 commits into
ai-dynamo:mainfrom
glamr-agent:dyn-5268-share-fixed-lane-primitive-kv-router-benches-66f09136db29

Conversation

@glamr-agent

@glamr-agent glamr-agent commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Overview

The Active Sequences, approximate LRU, and Mooncake KV-router benchmarks each kept their own copy of one lane protocol. The protocol has fixed atomic ID slots, a release-published count, a close flag, and a wakeup. A fix to its ordering or wakeups had to be made three times. This change moves the protocol into one shared module. It does not change benchmark behavior.

Key Changes

  • A new shared module, lib/bench/kv_router/common/fixed_lane.rs, holds the lane and its close-aware wait. The same FixedLane also holds the consumed counter used for depth reporting.
  • All three benchmarks use FixedLane. Active Sequences and approximate LRU record consumed progress; Mooncake never calls record_consumed. The approximate LRU inline producer now writes and publishes through the shared lane.
  • Unchanged: atomic orderings, wakeup timing, publication cadence (per operation in approximate LRU, per deadline group in the other two), each benchmark's overflow error, and feature gates.
  • A repo-wide search found no other copies of this protocol.
  • Deadline-clock consolidation (DYN-5211) landed separately in refactor(bench): share the KV-router replay deadline clock #17887. This PR keeps that shared clock and only shares the lane protocol.

PeaBrane (Yan Ru Pei) should review this change.

Validation

The three benchmark test targets pass before and after the change, including both approximate LRU end-to-end trials, which CI does not build. New unit tests for the shared lane fail when the wakeup or the closed-and-drained exit is removed. No performance run was made; the change makes no performance claim.

Issues and Ongoing Work

Closes DYN-5268.

Summary by CodeRabbit

  • Refactor
    • Open-loop benchmarks now use shared lane handling for query and operation processing, including publication, consumption tracking, and lane closure.
    • Lane capacity limits are handled consistently, with overflow reported when a lane is full.
  • Tests
    • Added coverage for lane capacity, publication and closure behavior, and consumption-depth tracking.

…hmarks

The Active Sequences, approximate-LRU and Mooncake open-loop benchmarks each
had their own copy of the same fixed ID lane: a preallocated AtomicU32 slot
array, a Release-published count, a closed flag and a Notify wakeup, plus the
same notified()-then-recheck consumer wait. Move that protocol into
kv_router/common/fixed_lane.rs as FixedLane, with ConsumedLane adding the
consumer-progress counter used for depth() and the lane_incomplete check.

Mooncake uses FixedLane, so it still has no shared consumed counter. Each
caller keeps its own overflow error, publication cadence, atomic orderings
and payload loop.

DYN-5268

Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
@glamr-agent
glamr-agent requested review from a team as code owners November 3, 2026 01:40
@copy-pr-bot

copy-pr-bot Bot commented Nov 3, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@glamr-agent
glamr-agent temporarily deployed to external_collaborator November 3, 2026 01:40 — with GitHub Actions Inactive
@glamr-agent
glamr-agent temporarily deployed to external_collaborator November 3, 2026 01:40 — with GitHub Actions Inactive
@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

👋 Hi glamr-agent! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

@github-actions github-actions Bot added external-contribution Pull request is from an external contributor refactor labels Nov 3, 2026
@glamr-agent

Copy link
Copy Markdown
Collaborator Author

factory: Initial validation snapshot.

Validation status: complete

Check Result Scope or reason
Checks the changed Rust crates with cargo check and Clippy. Passed (exit 0) Workspace check and Clippy, the five feature-gated bench builds, and the fixed_lane, active_sequences_trace, mooncake_trace, and approximate_lru tests on 33e56204bb; the same three test targets also passed on unchanged main 9fd7b0fcf2
Inspects the changed code when the claim cannot be tested with a local command. Passed (exit 0) Repo-wide search for other lane copies; atomic orderings, wakeups, publication cadence, overflow errors, and the absence of a Mooncake consumed counter compared with main

No benchmark performance run was made. The change makes no performance claim.

Detailed evidence — validation complete

Left out of the table because CI repeats them: none.

Evidence tally: [2/2 validated]

Advisory AI assessment: sound. No blocking findings.

Validation result: complete — pass. Every planned build, lint, and test command passed on the change. The same test targets passed on unchanged main. Two temporary edits made the new tests fail as intended. Code inspection found the lane behavior unchanged.

Evidence audit: complete [2/2 validated]. The command report comes from recorded runs.

Commands and results [2/2 validated]

Generated from the commands recorded during this run.

Check 1

Checks the changed Rust crates with cargo check and Clippy.

Result: Passed (exit 0)

Command:

Not shown because the exact command contained private run data.

Check 2

Inspects the changed code when the claim cannot be tested with a local command.

Result: Passed (exit 0)

Command:

Not shown because the exact command contained private run data.

@glamr-agent

Copy link
Copy Markdown
Collaborator Author

No description provided.

@glamr-agent

Copy link
Copy Markdown
Collaborator Author

factory: @dynamo-ops please run full CI for 33e5620

@dynamo-ops

Copy link
Copy Markdown
Contributor

/ok to test 33e5620

@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Walkthrough

The change adds shared fixed-capacity lane types and replaces local lane implementations in the active-sequences, approximate-LRU, and Mooncake open-loop benchmarks. The shared API handles slot publication, consumer progress, overflow, closure, and waiting.

Changes

Open-loop lane sharing

Layer / File(s) Summary
Shared lane API
lib/bench/kv_router/common/fixed_lane.rs, lib/bench/kv_router/common/mod.rs
Adds FixedLane and ConsumedLane, including bounded writes, publication, closure, waiting, consumer progress, and tests. The module is available under the listed benchmark and test-support features.
Active-sequences and approximate-LRU migration
lib/bench/kv_router/active_sequences_open_loop.rs, lib/bench/kv_router/approximate_lru_bench.rs
Replaces each benchmark’s local operation lane with ConsumedLane. Workers use the shared read, progress, and wait methods. Issuers handle lane overflow through the existing failure paths.
Mooncake migration
lib/bench/kv_router/mooncake_open_loop.rs
Replaces the local query lane with FixedLane. Workers use the shared read and wait methods, and write overflow maps to IssuerFailure::QueryLaneOverflow.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: Minimal · up to 33e56

No blocking behavior change is identified; the PR is mergeable after normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title names the main change: moving the lane primitive into one shared module used by the KV-router benchmarks. It matches the raw summary (new fixed_lane module, benchmarks switched to FixedLane/…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description states the problem (three benchmarks each duplicate the same lane protocol), the change (a shared fixed_lane module used by all three benchmarks), what is unchanged (atomic orderings, …
Full details: Title check

Explanation

The title names the main change: moving the lane primitive into one shared module used by the KV-router benchmarks. It matches the raw summary (new fixed_lane module, benchmarks switched to FixedLane/ConsumedLane) and the stated objective. It is a short, single sentence in conventional-commit form and is specific enough for a teammate scanning history.

Full details: Description check

Explanation

The description states the problem (three benchmarks each duplicate the same lane protocol), the change (a shared fixed_lane module used by all three benchmarks), what is unchanged (atomic orderings, wakeup timing, publication cadence, overflow errors, feature gates), and validation (test targets pass before and after, new unit tests, no performance run). It closes DYN-5268 and notes that the deadline-clock work landed separately. It covers the Overview, Key Changes, Validation, and Issues and Ongoing Work sections of the template. The Pull Request Checklist item ("I used Dynamo's /pr-checklist skill") does not appear in the description, and the Closes line uses a Linear link rather than the template's GitHub issue format ("Closes #XXXX"). These are minor gaps, and the description is otherwise complete and specific.

Full details: Docstring Coverage

Explanation

Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 12 functions across 5 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

ast-grep (0.51.1)

ast-grep could not parse rule config: /ast-grep-rules/ast-grep-essentials/rules/typescript/security/local-storage-sensitive-data-typescript.yml

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 319ad3b and 33e5620.

Files selected for processing: 5
  • lib/bench/kv_router/active_sequences_open_loop.rs
  • lib/bench/kv_router/approximate_lru_bench.rs
  • lib/bench/kv_router/common/fixed_lane.rs
  • lib/bench/kv_router/common/mod.rs
  • lib/bench/kv_router/mooncake_open_loop.rs

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 8 remain after this review.

@devin-ai-integration

Copy link
Copy Markdown
Contributor

✅ Dynamo PR CI passed — run 41561563002 (attempt 1) on 33e56204bb

Gate checks: ✅ backend-status-check · ✅ deploy-status-check · ✅ dynamo-status-check

Posted automatically by Devin for run 41561563002. Updated on every full-CI run of this PR.

@MatejKosec
MatejKosec enabled auto-merge (squash) November 3, 2026 22:04

@jthomson04 jthomson04 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The protocol moves over unchanged (orderings, notified() before the recheck, closed-and-drained exit). A few small suggestions inline.

Signed-off-by: GLAMR <svc-glamr@nvidia.com>
auto-merge was automatically disabled November 4, 2026 19:02

Head branch was pushed to by a user without write access

@glamr-agent
glamr-agent temporarily deployed to external_collaborator November 4, 2026 19:02 — with GitHub Actions Inactive
@glamr-agent

Copy link
Copy Markdown
Collaborator Author

nursery: Changed nothing for the docstring coverage warning in CodeRabbit’s summary. fixed_lane.rs already explains how IDs become visible and why the wait checks again before sleeping. The write, publish, read, and wait methods document the rules callers must follow. Extra comments on the migrated benchmark loops would repeat the code. The repository branch rules do not require CodeRabbit’s coverage threshold.

@glamr-agent

Copy link
Copy Markdown
Collaborator Author

nursery: @dynamo-ops please run full CI for 49cbcd4

@dynamo-ops

Copy link
Copy Markdown
Contributor

/ok to test 49cbcd4

@jthomson04 jthomson04 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All four addressed in 49cbcd4. LGTM.

@jthomson04
jthomson04 enabled auto-merge (squash) November 4, 2026 19:07
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
@glamr-agent
glamr-agent temporarily deployed to external_collaborator November 4, 2026 19:55 — with GitHub Actions Inactive
@glamr-agent

Copy link
Copy Markdown
Collaborator Author

nursery: @dynamo-ops please run full CI for 1b651ec

@dynamo-ops

Copy link
Copy Markdown
Contributor

/ok to test 1b651ec

Signed-off-by: GLAMR <svc-glamr@nvidia.com>
auto-merge was automatically disabled November 4, 2026 21:02

Head branch was pushed to by a user without write access

@glamr-agent
glamr-agent temporarily deployed to external_collaborator November 4, 2026 21:02 — with GitHub Actions Inactive
@glamr-agent

Copy link
Copy Markdown
Collaborator Author

nursery: @dynamo-ops please run full CI for 57ee286

@dynamo-ops

Copy link
Copy Markdown
Contributor

/ok to test 57ee286

@jthomson04
jthomson04 enabled auto-merge (squash) November 4, 2026 21:04
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
@glamr-agent
glamr-agent temporarily deployed to external_collaborator November 5, 2026 00:02 — with GitHub Actions Inactive
@glamr-agent

Copy link
Copy Markdown
Collaborator Author

nursery: @dynamo-ops please run full CI for 82ed715

@dynamo-ops

Copy link
Copy Markdown
Contributor

/ok to test 82ed715

@jthomson04
jthomson04 merged commit 02331f0 into ai-dynamo:main Nov 5, 2026
265 of 270 checks passed
ishandhanani added a commit to ishandhanani/dynamo that referenced this pull request Nov 5, 2026
Third upstream merge for the split v2 PR: the five commits past
53015f9 that landed while the second merge was validated. Four
conflicts, each placed in the owner crate, upstream behavior whole, the
stack's moves intact, no shims.

- ai-dynamo#18008 probe worker availability with an Fx set: WorkerIdSet and the
  per-set AvailableWorkerIds cache in the kernel's types.rs, the listed-
  worker walk in the kernel's filter.rs, the actor's cached availability in
  queue.rs (auto-merged); WorkerIdSet exported to hosts through
  dynamo_scheduler::host (lib/llm's LoRA availability filter) and to
  plugins through dynamo_kv_router::plugins::prelude (the builtin
  default_policy test stays prelude-only).
- ai-dynamo#18084 CRTC stale-leaf sweep trigger scaled with tree size:
  lib/router/kv-indexer concurrent_radix_tree_compressed (auto-merged).
- ai-dynamo#17900 shared fixed-lane bench primitive: lib/bench (auto-merged).
- ai-dynamo#17947 planner priority pools, ai-dynamo#17625 beam tool-call finalization:
  outside the router, auto-merged.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

This branch was previously deployed

1 inactive deployment
external_collaborator — 82ed715c Deployed Nov 5, 2026 by glamr-agent via ok-to-test #46108
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external-contribution Pull request is from an external contributor refactor size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants