Cache class working with generate#1

Merged

gante merged 15 commits intofeat/kv_cache_classfrom

Nov 21, 2023

Commits on Nov 21, 2023

Draft version of new KV Caching

tomaarsen
authored and
gante
committed
Address numerous PR suggestions

tomaarsen
authored and
gante
committed
Integrate (Sink)Cache with Llama FA2

tomaarsen
authored and
gante
committed
Move from/to_legacy_cache to ...Model class

tomaarsen
authored and
gante
committed
Undo unnecessary newline change

tomaarsen
authored and
gante
committed
Match import style

tomaarsen
authored and
gante
committed
working generate
gante
committed
Add tests; Simplify code; Apply changes to Mistral and Persimmon
gante
committed
fix rebase mess
gante
committed
a few more manual fixes
gante
committed
last manual fix
gante
committed
propagate changes to phi
gante
committed
upgrade test
gante
committed
add use_legacy_cache docstring; beef up tests
gante
committed
reintroduce unwanted deletes
gante
committed