Skip to content

feat: CUDA port of TurboQuant3 KV cache — 3.47x compression, 98.5% of F16 decode speed on RTX 5090#3

Closed
signalnine wants to merge 15 commits into
TheTom:feature/turboquant-kv-cachefrom
signalnine:feature/turboquant-kv-cache
Closed

feat: CUDA port of TurboQuant3 KV cache — 3.47x compression, 98.5% of F16 decode speed on RTX 5090#3
signalnine wants to merge 15 commits into
TheTom:feature/turboquant-kv-cachefrom
signalnine:feature/turboquant-kv-cache

feat: zero-pad non-128 heads for full 7-stage WHT (replaces q8_0 fall…

b74119a
Select commit
Loading
Failed to load commit list.
Sign in for the full log view

The logs for this run have expired and are no longer available.