More accurate Q4_0 and Q4_1 quantizations#896

Closed

ikawrakow wants to merge 12 commits intomasterfrom

quantize_experiments

Commits on Apr 11, 2023

Use better conversion to ints
Iwan Kawrakow
committed
Adding new functions for Q4_0 and Q4_1 quantization
Iwan Kawrakow
committed
Remove forgotten remnant from a discarded change to ggml.c
Iwan Kawrakow
committed
Add ability to use new quantization in quantize-stats
Iwan Kawrakow
committed
Add new quantization to quantize
Iwan Kawrakow
committed
Reverting round() change so we can pass tests
Iwan Kawrakow
committed
Improve Q4_0 MSE
Iwan Kawrakow
committed

Commits on Apr 12, 2023

Further improve Q4_0 MSE
Iwan Kawrakow
committed
Various experiments, including 5-bit qunatization
Iwan Kawrakow
committed
POC: Even lower rmse 4-bit Q4_0 quantization
Iwan Kawrakow
committed

Commits on Apr 13, 2023

POC: Q4_1 for groups of 16 weight
Iwan Kawrakow
committed
POC: Measure rmse of 8 bit quantization
Iwan Kawrakow
committed