CUDA: small PP performance improvement for MoE models#589

Merged

ikawrakow merged 4 commits into

mainfrom

ik/cuda_quantized_fmoe

Jul 7, 2025

Commits on Jul 5, 2025

Trying to implement quantized fmoe - not working yet
Iwan Kawrakow
committed
This works, but is slower than the non-working version
Iwan Kawrakow
committed

Commits on Jul 6, 2025

quantize_mmq_q8_1_id
Iwan Kawrakow
committed
Minor
Iwan Kawrakow
committed