Repository navigation

[CB] Changes for long generation - #45530

Merged

ArthurZucker merged 17 commits into

mainfrom

cb-very-long-gen

Apr 23, 2026

Merged

[CB] Changes for long generation#45530
ArthurZucker merged 17 commits into
mainfrom
cb-very-long-gen

Commits on Apr 20, 2026
- Fix KV dedup for decode batches
  remi-or
  committed
  1c076c6
- Fix memory estimation
  remi-or
  committed
  7eec987
- Change default
  remi-or
  committed
  b4b74ff
- Added write-only fast path
  remi-or
  committed
  88287d1
- Take both peaks into account
  remi-or
  committed
  dd00e9b
- Revert unused config field
  remi-or
  committed
  1599e24
Commits on Apr 21, 2026
- Review 1
  remi-or
  committed
  7cd08da
- Fix p1s
  remi-or
  committed
  8640626
- Fix p2s and p3s that needed it
  remi-or
  committed
  b853c85
- Added a TODO
  remi-or
  committed
  f39b68f
- Fix test, lower max cached graph, add TODO
  remi-or
  committed
  4f814b2
- Fix fragmentation with big warmup
  remi-or
  committed
  28ca9ed
- Add more space for logits processors
  remi-or
  committed
  878f469
- Merge branch 'main' into cb-very-long-gen
  remi-or
  authored
  5854ad1
Commits on Apr 23, 2026
- Merge branch 'main' into cb-very-long-gen
  remi-or
  authored
  7bd12b3
- Fix
  remi-or
  committed
  1695d37
- Merge branch 'main' into cb-very-long-gen
  remi-or
  authored
  6b45f17