Repository navigation

🚨Add an FP8 backend to `QuantizedCache` - #48677

Merged

vasqu merged 21 commits into

huggingface:mainfrom

jiqing-feng:fp8-kv-cache

Oct 22, 2026

Merged

🚨Add an FP8 backend to `QuantizedCache`#48677
vasqu merged 21 commits into
huggingface:mainfrom
jiqing-feng:fp8-kv-cache

Commits on Sep 10, 2026
- Add an FP8 backend to `QuantizedCache`
  jiqing-feng
  committed
  9789b12
- Reorder the FP8 states for beam search
  jiqing-feng
  committed
  64cfc5d
- Use an early return in `Fp8QuantizedLayer.reorder_cache`
  jiqing-feng
  committed
  87ab14a
Commits on Sep 14, 2026
- Measure the quantized cache backends accurately
  jiqing-feng
  committed
  a7cc8e3
Commits on Sep 17, 2026
- Merge branch 'main' into fp8-kv-cache
  jiqing-feng
  authored
  8882b42
Commits on Oct 6, 2026
- Merge branch 'main' into fp8-kv-cache
  jiqing-feng
  authored
  2b7a9eb
- Reset the FP8 scales, and cover the FP8 backend in the quantized layer tests
  jiqing-feng
  committed
  bc126c1
- Merge branch 'main' into fp8-kv-cache
  jiqing-feng
  authored
  5eaeb32
- Merge branch 'main' into fp8-kv-cache
  jiqing-feng
  authored
  4fc4f00
Commits on Oct 16, 2026
- Address review on the FP8 quantized cache layer
  jiqing-feng
  committed
  4743c39
- Trim the comments down to one line
  jiqing-feng
  committed
  e9e889d
- Store the FP8 scale alongside the states it quantizes
  jiqing-feng
  committed
  05ef372
- Say what the `.contiguous()` in `lazy_initialization` is actually for
  jiqing-feng
  committed
  bc31eda
Commits on Oct 19, 2026
- Address @vasqu's review
  jiqing-feng
  committed
  ed19267
- One-line the docstrings and comments that fit on one line
  jiqing-feng
  committed
  7018cc4
- Merge branch 'main' into fp8-kv-cache
  jiqing-feng
  authored
  862a643
- Keep the FP8 state initialization out of the shared quantized layer
  jiqing-feng
  committed
  b4e172b
Commits on Oct 20, 2026
- Split the quantized layer into a base and a group-wise class
  jiqing-feng
  committed
  02dc2e8
- Note which subtree a future fp8 variation belongs to
  jiqing-feng
  committed
  b138c83
Commits on Oct 21, 2026
- Keep the group-wise signature on QuantizedLayer and name the new base
  jiqing-feng
  committed
  780cc9d
Commits on Oct 22, 2026
- Declare the quantized states in the base layer, and group the quantized tests
  jiqing-feng
  committed
  08c5bc7

Commits

Commits on Sep 10, 2026

Commits on Sep 14, 2026

Commits on Sep 17, 2026

Commits on Oct 6, 2026

Commits on Oct 16, 2026

Commits on Oct 19, 2026

Commits on Oct 20, 2026

Commits on Oct 21, 2026

Commits on Oct 22, 2026