Adding IQ4_KSS: 4.0 bpw quants#89

Merged

ikawrakow merged 11 commits into

mainfrom

Oct 16, 2024

Commits on Oct 16, 2024

iq4_kss: WIP
Iwan Kawrakow
committed
iq4_kss: CUDA dequantize works
Iwan Kawrakow
committed
iq4_kss: slightly better quantization
Iwan Kawrakow
committed
iq4_kss: another small quantization improvement
Iwan Kawrakow
committed
iq4_kss: CUDA works
Iwan Kawrakow
committed
iq4_kss: new bit arrangement - CUDA and Zen4 work
Iwan Kawrakow
committed
iq4_kss: ARM_NEON. Predictably very slow
Iwan Kawrakow
committed
iq4_kss: Metal
Iwan Kawrakow
committed
iq4_kss: somewhat faster Metal dot product
Iwan Kawrakow
committed
iq4_kss: AVX2
Iwan Kawrakow
committed
iq4_kss: very slightly faster Metal dot product
Iwan Kawrakow
committed