Bring KV rotation to llamarunner-served models (deepseek-r1, qwen2 family)

[dogkeeper886]

Goal

Bring the KV cache rotation that landed in PR #133 to all models, regardless of which runner serves them. Today, rotation only applies to the new ollamarunner engine. Models routed through the legacy llamarunner engine (`deepseek-r1:14b`, `deepseek-r1:32b`, others on `qwen2` arch) get the q8_0 KV memory savings from PR #137 but eat the corresponding quality cost without the rotation cushion.

Why this matters

Two flagship K80 models (`deepseek-r1:14b`, `deepseek-r1:32b`) are routed through llamarunner per the engine-coverage audit (`docs/research/k80-fa-model-coverage.md`). They're affected by every per-token quant noise event in the KV cache without rotation help.

Constraint discovered during investigation

Upstream llama.cpp implements this as commit `744c0c73` (PR ggml-org/llama.cpp#21038). Direct cherry-pick is not viable — our vendored llama.cpp is pinned to commit `7049736b` (2025-10-13), upstream's rotation commit is dated 2026-04-01, putting it ~6 months ahead. The intermediate commits substantially refactored the affected files:

• `llama-graph.cpp`: `build_kq_mask` was renamed to `build_attn_inp_kq_mask`; both names absent from our tree
• Function landmarks have shifted by 80–520 lines
• 28 hunks in `llama-kv-cache.cpp` alone won't apply against our tree's structure

Implementation options

A. Port the rotation work to our vendored tree

Re-implement the concept of upstream's commit 744c0c73 against our 2025-10-13 vendored llama.cpp. New code, written from scratch using upstream as a reference design. Components needed:

• Hadamard generator + cache (attn_rot_hadamard map, ggml_gen_hadamard)
• Hadamard multiply in build_attn before cache write
• Reverse rotation on attention output
• Integration with build_rope_shift for context-shift safety

Effort: ~1-2 days careful C++. Risk: medium — bugs affect every llamarunner-served model, not just the deepseek-r1 family.

B. Vendor bump

Update `FETCH_HEAD` in `Makefile.sync` to upstream commit 744c0c73 (or later) and re-apply our 30 existing patches.

Effort: medium-but-cascading. The bump is one line, but 6 months of upstream changes mean some of our patches will conflict, and our K80-specific patches (FA, NVML, CUDA scheduling) may interact badly with intermediate upstream changes. Risk: very high — touches every code path in llamarunner.

C. Defer

Accept that deepseek-r1 family runs without rotation today. Revisit when there's a concrete user complaint about quality, OR when those arches migrate to the new engine.

Recommendation

Defer (C) unless a user reports quality issues. Reasoning:

• The new ollamarunner engine is the project's direction; the list of llamarunner-served models is shrinking
• Both affected models still produce coherent output post-Productize K80 perf defaults: FA on + q8_0 KV cache (closes #136) #137 — they're just not getting the additional quality recovery that rotation provides for new-engine models
• If we do invest, A is preferable to B — porting one feature is bounded scope; bumping vendor across 6 months of upstream is not

Acceptance Criteria (when revisited)

1. `OLLAMA_KV_CACHE_TYPE=q8_0 ollama run deepseek-r1:14b` produces coherent output (no quality regression vs the un-rotated path)
2. Existing llamarunner test cases pass
3. Document update in `docs/design/kv-rotation.md` to reflect parity

References

• New-engine integration (Phase 1): Add rotation-before-quant to KV cache (Phase 1) #102, PR KV rotation: always-on for compatible head dims (closes #102) #133
• Upstream commit (reference, not directly applicable): llama : rotate activations for better quantization ggml-org/llama.cpp#21038 (commit `744c0c73`)
• Engine coverage table: `docs/research/k80-fa-model-coverage.md`
• Vendored pin: `Makefile.sync:FETCH_HEAD=7049736b`

Part of #101
Related to #102

[dogkeeper886]

Closing as wontfix per analysis above.

Bottom line: porting upstream commit 744c0c73 to our vendored llama.cpp would be ~1-2 days of careful C++ work, against a ~6-month-drifted vendor base, for a benefit that goes only to two legacy models (deepseek-r1:14b/32b) which are themselves being phased out by the broader LLM ecosystem. Poor ROI.

Reopening criteria: a concrete user complaint about output quality on a deepseek-r1 / qwen2-arch model with q8_0 KV cache, OR a brand-new important llamarunner-only model arriving on K80.