fix(kvcache): decode lora_id/medium/lora_name/group_idx in BlockStored (#2285)

[pjdurden]

Problem

The ZMQ decoder reads only the first four positional fields of vLLM's BlockStored (block_hashes, parent_block_hash, token_ids, block_size) and drops the rest. Newer vLLM (vllm/distributed/kv_events.py) also carries lora_id, medium, lora_name, extra_keys, and group_idx. Dropping group_idx/medium/lora_name is what causes the false prefix matches described in #2285:

• group_idx: hybrid-attention models (sliding-window + full layers) hash the same token range into different KV-cache groups. Collapsing all groups into one hash space produces false prefix matches.
• medium: with KV offloading, blocks can live on CPU/remote tiers, so counting every block as a GPU hit over-credits pods that only hold the prefix on a slower tier.
• lora_name: canonical adapter id (lora_id is deprecated upstream), needed to isolate adapters in the index.

This PR (1 of 2)

Decode the dropped fields and expose them on BlockStoredEvent. No routing/keying behavior change yet, this just stops dropping the data.

BlockStored is a msgspec array_like struct with omit_defaults, so trailing fields equal to their default may be absent from the wire array. The decoder therefore reads positions 5-9 by index with bounds checks, and tolerates shorter arrays from older vLLM builds:

[0]tag [1]block_hashes [2]parent_block_hash [3]token_ids [4]block_size
[5]lora_id [6]medium [7]lora_name [8]extra_keys [9]group_idx ...

extra_keys (position 8) is left for the follow-up that consumes it for block-hash reconstruction; group_idx is still read at its fixed position 9.

Follow-up (PR2)

Key prefix entries by (model, lora_name, group_idx) and score non-GPU mediums lower, plumbing the fields through the kvevent handler into the indexer. That touches the MatchPrefix/AddPrefix signature across the indexers, so it is kept separate per the discussion on #2285.

Tests

New decoder tests build raw vLLM-layout msgpack arrays (full field set, partial length, nil/None values, backward-compat short array, and group_idx read past a non-nil extra_keys). Verified locally:

go test ./pkg/cache/kvcache/
go build ./...
go vet ./pkg/cache/...
gofmt -l pkg/cache/kvcache/

Part 1 of 2 for #2285.

[Copilot]

Pull request overview

This pull request updates the KV-cache msgpack decoder to correctly decode newer optional BlockStored fields emitted by vLLM (notably lora_id, medium, lora_name, and group_idx) while remaining backward compatible with older, shorter event arrays.

Changes:

• Extend BlockStored decoding to conditionally parse optional fields by positional index with bounds checks.
• Add toStringPtr helper to decode nullable string fields that may arrive as string or []byte.
• Add targeted decoder tests that marshal raw positional arrays to validate the wire contract and backward compatibility.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
pkg/cache/kvcache/msgpack_decoder.go	Decodes optional BlockStored fields (positions 5–7, 9) and adds toStringPtr for nullable string decoding.
pkg/cache/kvcache/msgpack_decoder_test.go	Adds raw-wire-format tests covering full/partial/omitted optional fields and extra_keys positioning behavior.
pkg/cache/kvcache/event_types.go	Updates BlockStoredEvent documentation and struct to expose decoded optional fields to downstream consumers.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[gemini-code-assist]

Code Review

This pull request updates the BlockStoredEvent struct and its msgpack decoder to support new optional fields (LoraID, Medium, LoraName, and GroupIdx) introduced in newer vLLM builds. The decoder reads these fields by position with bounds checks to maintain backward compatibility with older vLLM builds that send shorter arrays. Additionally, comprehensive unit tests have been added to verify the decoding behavior. There are no review comments, so I have no feedback to provide.

[varungup90]

Review Notes & Suggestions

Thanks for the PR! Here are a few observations and minor suggestions. Most of these are non-blockers or notes for the follow-up PR, but they are worth a quick review.

Actionable Suggestions

• lora_id integer type may be too narrow: In some vLLM versions, lora_id can be a uint32 or larger. toInt64Ptr needs to handle uint64 input, or it will error on large adapter IDs. Since lora_id is deprecated upstream anyway, this is low-risk, but it is worth doing a quick check of the toInt64Ptr type switch.
• Note extra_keys skip explicitly in the struct: The struct comment references positions 5-9, but extra_keys (position 8) has no field. It's mentioned in the decoder comment, but adding a brief note like // position 8 (extra_keys) decoded in follow-up PR to the struct block would help future readers avoid wondering if it was accidentally forgotten.

Observations & Notes for PR2

• medium field typing: The medium values ("GPU", "cpu", "remote") are a known enum in vLLM. Using *string is fine for now (and matches this PR's stated goal of just not dropping the data), but the follow-up PR that scores these should consider defining a typed constant.
• Position 4 (block_size) is still not decoded: The struct and original code both skip block_size. This PR doesn't regress anything, but if PR2 wants to score blocks differently by tier/medium, having the block size available might be useful.
• Error paths: The group_idx error message is "invalid group_idx: %w", which is great. lora_id, medium, and lora_name also have good names. Everything is consistent here—no issues.

[pjdurden]

Thanks for the review.

• extra_keys: added a comment in the BlockStoredEvent struct marking position 8 (extra_keys) as deferred to PR2, so the gap between pos 7 and 9 is explicit.
• lora_id width: toInt64Ptr goes through parseInt64, which alread handles uint32/uint64, and the uint64 case guards against math.MaxInt64 overflow. So large adapter ids are covered.
• medium typed constant and block_size: agree, i'll fold both into PR2 where the scoring lands.