Use Q8_K_128 for IQ1_S_R4 and IQ1_M_R4 matrix multiplications#194

Merged

ikawrakow merged 6 commits into

mainfrom

ik/iq1_s_r4_k128

Feb 9, 2025

Commits on Feb 8, 2025

iq1_s_r4: Use Q8_K_128 instead of Q8_1_X4 for gemm (AVX2/Zen4)
Iwan Kawrakow
committed
iq1_m_r4: Use Q8_K_128 instead of Q8_1_X4 for gemm (AVX2/Zen4)
Iwan Kawrakow
committed

Commits on Feb 9, 2025

iq1_s_r4: Use Q8_K_128 instead of Q8_1_X4 for gemm (Neon)
Iwan Kawrakow
committed
iq1_m_r4: Use Q8_K_128 instead of Q8_0_X4 for gemm (Neon)
Iwan Kawrakow
committed
Simdify q8_K128 quantization also on Neon
Iwan Kawrakow
committed
Cleanup
Iwan Kawrakow
committed