Repository navigation

Quantized KV Cache - #30483

Merged

zucchini-nlp merged 21 commits into

huggingface:mainfrom

zucchini-nlp:quant

May 23, 2024

Merged

Quantized KV Cache#30483
zucchini-nlp merged 21 commits into
huggingface:mainfrom
zucchini-nlp:quant

Commits on May 10, 2024
- clean-up
  zucchini-nlp
  committed
  16731be
- Update src/transformers/cache_utils.py
  
  zucchini-nlp
  and
  ArthurZucker
  authored
  cf00de6
- Update src/transformers/cache_utils.py
  
  zucchini-nlp
  and
  ArthurZucker
  authored
  6de0d8a
- Update src/transformers/cache_utils.py
  
  zucchini-nlp
  and
  ArthurZucker
  authored
  5a87cbb
- fixup
  zucchini-nlp
  committed
  bfe0804
Commits on May 13, 2024
- Update tests/quantization/quanto_integration/test_quanto.py
  
  zucchini-nlp
  and
  younesbelkada
  authored
  62abc33
- Update src/transformers/generation/configuration_utils.py
  
  zucchini-nlp
  and
  ArthurZucker
  authored
  519682c
- more suggestions
  zucchini-nlp
  committed
  6f19cee
- mapping if torch available
  zucchini-nlp
  committed
  2b8e042
Commits on May 14, 2024
- Merge branch 'main' into quant
  zucchini-nlp
  authored
  4c65b5b
- run tests & add 'support_quantized' flag
  zucchini-nlp
  committed
  1d9cf15
- Merge branch 'main' into quant
  zucchini-nlp
  authored
  a58aa9d
- fix jamba test
  zucchini-nlp
  committed
  d181339
- revert, will be fixed by another PR
  zucchini-nlp
  committed
  e658c9f
Commits on May 18, 2024
- Merge branch 'main' into quant
  zucchini-nlp
  authored
  9397dda
- codestyle
  zucchini-nlp
  committed
  045e406
Commits on May 20, 2024
- HQQ and versatile cache classes
  zucchini-nlp
  committed
  3193b43
Commits on May 21, 2024
- final update
  zucchini-nlp
  committed
  126ce84
Commits on May 23, 2024
- Merge "main"
  zucchini-nlp
  committed
  0d1df5f
- typo
  zucchini-nlp
  committed
  89413d3
- make tests happy
  zucchini-nlp
  committed
  9d0f668

Commits

Commits on May 10, 2024

Commits on May 13, 2024

Commits on May 14, 2024

Commits on May 18, 2024

Commits on May 20, 2024

Commits on May 21, 2024

Commits on May 23, 2024