Weight Protective Quantization Range -119, 119

[Jzz24]

https://github.com/mit-han-lab/omniserve/blob/main/omniserve/modeling/layers/quantized_linear/w4a8_linear.py#L176, it seems we do not quantize the int8 w to range [-119, 119]? And how to caculate the s1_scale? just like the int8 quantization? but use qmin=-119, qmax=119?

[ys-2020]

Hi. Thanks for your interests in QServe. The protective range 119 has already be considered and utilized during the model quantization process. When computing the s1_scale for int8 quantization, we use 119 for the scaling factor computation.

[Jzz24]

get it, thx