Exllama v2#1211

Merged

Narsil merged 11 commits intomainfrom

Nov 25, 2023

Commits on Nov 25, 2023

draft exllamav2, first commit

flozi00
authored and
Narsil
committed
build in docker

flozi00
authored and
Narsil
committed
use exllamav2QuantLinear instead of exllama1

flozi00
authored and
Narsil
committed
Update exllamav2 (illegal address issued)
Narsil
committed
Adding scratch space.
Narsil
committed
Fixing exllamav2.

Ubuntu
authored and
Narsil
committed
Exllamav2 functional.
Narsil
committed
Fix imports.
Narsil
committed
Deactivating v2 for sharded.
Narsil
committed
Rebased.
Narsil
committed
Keep exllamav1 for sharded flows.
Narsil
committed