sycl: add SET_ROWS support for turbo2/turbo3/turbo4 V cache

[TheTom]

Tracking SYCL backend coverage for the turbo2/turbo3/turbo4 V-cache codepath. Vulkan was fixed in #118; SYCL is the same shape of bug but a separate backend.

Background

@cclecle (Intel A380, oneAPI) hit the same SET_ROWS abort on SYCL in #50:

ggml_backend.cpp:898: pre-allocated tensor (cache_k_l3 (view)) in a buffer (SYCL0)
that cannot run the operation (SET_ROWS)

He has a working starting point with a Claude-implemented SYCL kernel for turbo3:

https://github.com/cclecle/llama-cpp-turboquant/tree/feature/turboquant-kv-cache

What's needed

Mirror the Vulkan PR #118 pattern in ggml/src/ggml-sycl/:

1. Register TURBO2_0 / TURBO3_0 / TURBO4_0 as supported targets in the SYCL SET_ROWS op
2. Wire the per-bitwidth quant kernels (cclecle has turbo3, turbo2/turbo4 still TODO)
3. Add the dispatch entry so the scheduler doesn't fall back to abort
4. Verify end-to-end with llama-bench -ctk q4_0 -ctv turboN -fa 1 on real Intel hardware (Arc A380 / A770)
5. Output coherence check: llama-cli -p \"The capital of France is\" -n 16 with -ctv turbo4

Why we can't do this in-house

We have zero Intel GPUs. SYCL is community-maintained for this fork. Best path is for someone with Intel hardware to PR.

Hardware needed

• Intel Arc A380 / A770 (or any DG2-class+) for real validation
• Intel Tiber Cloud has free A770 access if you don't have local hardware

Reference for porters

• Vulkan PR (mechanical pattern to mirror): vulkan: add SET_ROWS support for turbo2_0 and turbo4_0 (#50) #118
• Original turbo3 Vulkan PR (apollosenvy): vulkan: fix turbo3 build + coopmat FA after April upstream sync #87
• cclecle's SYCL fork starting point: https://github.com/cclecle/llama-cpp-turboquant/tree/feature/turboquant-kv-cache

cc @cclecle — if you're up for opening a PR off your fork (just turbo3 to start, or full turbo2+3+4), I'll review and merge.

[cclecle]

Hello TheTom, no problem to PR the changes, but I'll be totally honest with you: if I used Claude its because I didn't wanteed to dive into llamacpp code and probably dont have the skill to understand the Turboquant algorithm. So I probably won't be of much help in fixing it... !

[cclecle]

PR sent: #144

[Schnurzel700]

Running latest feature/turboquant-kv-cache (commit 2cbfdc6, May 19 2026), so this includes #118.

Vulkan still fails when using turbo KV, even for turbo3:

--cache-type-k q8_0
--cache-type-v turbo3

Error:

pre-allocated tensor (cache_v_l3 (view)) in a buffer (Vulkan0) that cannot run the operation (SET_ROWS)

This indicates:

• turbo3 SET_ROWS is still not usable on Vulkan in this path
• specifically affects cache_v_l3 (view) tensors

So this is not limited to turbo2/4, turbo3 is also broken on real Vulkan drivers (RADV).

CPU fallback works, so this appears to be a Vulkan backend support gap, possibly related to:

• view tensors not being accepted by supports_op, or
• incomplete SET_ROWS coverage despite vulkan: add SET_ROWS support for turbo2_0 and turbo4_0 (#50) #118

Repro is consistent with Qwen3.6-27B.

Happy to test patches.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.