IQ4_K: SOTA 4-bit quantization#6

Merged

ikawrakow merged 7 commits into

mainfrom

Jul 28, 2024

Commits on Jul 27, 2024

iq4_k: basics
Iwan Kawrakow
committed
iq4_k: TG now works on CUDA
Iwan Kawrakow
committed
iq4_k: AVX512 implementation
Iwan Kawrakow
committed
iq4_k: AVX2 implementation
Iwan Kawrakow
committed

Commits on Jul 28, 2024

iq4_k: NEON implementation
Iwan Kawrakow
committed
iq4_k: Metal implementation
Iwan Kawrakow
committed
iq4_k: scalar dot product
Iwan Kawrakow
committed