You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 7096a57
Browse filesBrowse the repository at this point in the historyBrowse files
Add `--kv-state-agent` so a GPU-free deployment exercises the full KVCR
router path: frontend KV router + `python -m dynamo.kv_state_agent` +
mocker workers.
- Each worker DP rank attaches its raw ZMQ stream to the namespace's state
agent through a KvStateAttachmentOwner, with a unique CacheOwner per
worker x rank, and advertises `kv_event_source_mode = state_agent_v2`.
The in-process legacy relay is skipped so events are published once.
- Host-pinned (G2) events are tagged `ownership: "kvcr"`; G1 events stay
framework-owned.
- The ZMQ sink now recovers G2 token IDs from the blocks' earlier G1
stores. AISimulate's G2 residency carries none, so ZMQ mode previously
dropped every G2 event and no lower tier reached the router.
- `--kv-state-agent-router-hints` advertises router-hint capability with
placeholder source endpoints so the router runs its hinted lower-tier
lookup; the mocker logs and ignores the hints.
- The ZMQ PUB socket uses vLLM's send high-water mark (100k messages)
instead of libzmq's default of 1000, so a lagging subscriber sees
vLLM-like buffering.
- examples/backends/mocker/launch/agg_kvcr_state_agent.sh launches the
state agent, mockers and frontend.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
0 commit comments