Skip to content

refactor(router): add dynamo-kv-events for KV event types, the engine ZMQ codec, and the cursor - #176

Closed
ishandhanani wants to merge 1 commit into
idhanani/rs2-02-kv-hashingfrom
idhanani/rs2-03-kv-events
Closed

ishandhanani wants to merge 1 commit into
idhanani/rs2-02-kv-hashingfrom
idhanani/rs2-03-kv-events

Conversation

@ishandhanani

Copy link
Copy Markdown
Owner

Router split v2, unit 03 of 17. Stacked on #175 (unit 02).

Summary

dynamo-kv-events (lib/kv-events) is everything about KV events, for both ends of the wire: the KvCacheEvent and RouterEvent types (event), the engine ZMQ msgpack codec with its filters and extra-key parsing (zmq_wire, moved as a directory with its tests), the replay cursor a subscriber keeps in step with a publisher (cursor), and the publisher placement helpers (placement). It depends on protocol and kv-hashing plus serde, rmp-serde, rustc-hash, thiserror, tracing; no sockets, tasks, or servers (the socket is dynamo-zmq, next; the listener is a kv-router adapter). Rule applied: the producer owns its output types, so the event vocabulary leaves the monolith together with its codec. One cross-crate effect handled: KvCacheEventError is #[non_exhaustive], which binds only across crates, so kv-router's two exhaustive matches on it gain a wildcard arm and an unknown counter. kv-router keeps shims at every old path, including dynamo_kv_router::zmq_wire and ::recovery. Replaces fork PR #113.

What moves

  • refactor(router): add dynamo-kv-events for KV event types and the engine codec #113 acd8bf4226 refactor(router): add dynamo-kv-events for KV event types and the engine codec: event, zmq_wire (directory move), cursor (from recovery/cursor.rs), placement. Move-diff (its body): 4,782 normalized lines in common; 4 new (two pad_value_for_mm_hash paths re-spelled to dynamo_kv_hashing::routing, the mod tests wrapper in event.rs); 2 dropped (the old spellings); every moved file at fraction 1.000 except convert.rs at 0.998; the indexer/metrics.rs wildcard arms and unknown counter are a deliberate change outside the move set.

Dependency edges

  • dynamo-kv-events -> dynamo-router-protocol, dynamo-kv-hashing (router deps); direct: dynamo-kv-hashing, dynamo-router-protocol, rmp-serde, rustc-hash, serde, serde_json, thiserror, tracing

Validation

Unit-level move-diff, run from the worktree at the unit head acd8bf4226: python3 -I scripts/router_move_diff.py e28c4d0fc2 acd8bf4226 --from <4 paths> --to <3 paths>, the union of the replaced fork PRs' --from/--to lists filtered to the paths present at each end (full command in the artifact /tmp/rr-split/unit-prs/03-move-diff.txt).

summary from_normalized=4794 to_normalized=4796 common=4792 new=4 dropped=2
result DIFFERS

4 new / 2 dropped, as the fork PR recorded: two pad_value_for_mm_hash call paths re-spelled from crate::protocols:: to dynamo_kv_hashing::routing::, and the mod tests { } wrapper in event.rs.

New and dropped normalized lines (4 new, 2 dropped).
new 1x let pad = dynamo_kv_hashing::routing::pad_value_for_mm_hash(mm_objects[group_index]);
new 1x mod tests {
new 1x run_pad = dynamo_kv_hashing::routing::pad_value_for_mm_hash(mm_hash);
new 1x }
dropped 1x let pad = crate::protocols::pad_value_for_mm_hash(mm_objects[group_index]);
dropped 1x run_pad = crate::protocols::pad_value_for_mm_hash(mm_hash);

Chain v2, full mode, on the rebased stack head ebd31cdf4d (every unit's commits are ancestors of it; the chain ran once on the head, not per unit), log /tmp/rr-split/srebase-validation.log: stage 1, the 13 router crates (clippy under default and all features, cargo test 1,247 passed / 5 ignored, rustdoc with all features), 1 min 50 s; stage 2, kv-router (clippy, bench check, 229 tests including the prelude test), 1 min 20 s; stage 3, workspace clippy, 4 min 58 s; stage 4, hosts (dynamo-llm lib alone, its kv_router 492 / discovery 275 / pool_relay 261 tests, 13 host crates clippy, bindings clippy; the pre-existing kvbm-consolidator manifest failure is the one non-zero exit), 29 min 21 s; stage 5, gates (every kv-router feature alone and together, 6 public-API snapshots unchanged with kv-router at 918 items, layering gate 47 checks and --host 66 checks with 0 failures, codeowners 7590/7590, fmt, 3 lockfiles --locked), 3 min 06 s; 40 min 35 s total, exit 0.

Each replaced fork PR carried its own validation chain; see its body.

Tickets

Part of DYNO-172

…ine codec

Move everything about KV cache events out of lib/kv-router into
lib/kv-events: the event types (KvCacheEvents, KvCacheEvent,
KvCacheEventData, store, remove, and stored-block data, KvCacheEventError),
the RouterEvent envelope and its builders, KV_EVENT_SUBJECT, the engine ZMQ
msgpack codec (zmq_wire), the replay cursor (recovery/cursor.rs), and the
publisher placement helpers (Placement, PlacementOwner, PlacementEvent).
The RouterEvent legacy-wire tests and the router event constructor test
move with the types; the codec's tests move with the codec.

Both ends of the KV-event wire depend on this crate: workers, sidecars,
and the C bindings publish with it, the router's indexer ingests with it.
It depends on dynamo-router-protocol and dynamo-kv-hashing plus serde,
rmp-serde, rustc-hash, thiserror, and tracing; no sockets, tasks, or
servers. lib/kv-router keeps pub use shims at every old path, including the
zmq_wire and recovery module paths, so no other crate's imports change.

KvCacheEventError is non-exhaustive, which binds only across crates, so
kv-router's two exhaustive matches on it (behind the metrics feature) gain a
wildcard arm and an "unknown" status counter.

The layering gate gains a dynamo-kv-events row; CODEOWNERS claims
lib/kv-events and follows the moved extra_keys override.

Step 2 of the router split v2 plan. DYNO-172.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani

Copy link
Copy Markdown
Owner Author

Landed upstream as ai-dynamo#18055 (squash commit 79692cf on main, 2026-11-11). Closing this fork review unit.

This branch had an error being deployed

1 failed deployment
external_collaborator — acd8bf42 Deployed Nov 2, 2026 by ishandhanani via ok-to-test #515
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant