Repository navigation
[1/3] fix dsv3 awq issue - #4556
Conversation
|
@zhyncs This kernel supports GPUs with |
|
FMA operations in
|
Thx for detailed benchmark data! |
Co-authored-by: leoneo <1320612015@qq.com>

Motivation
related issue: #4462
Currently dsv3 awq suffers accuracy problem that may due to the overflow of fp16.
Modifications
co-author: @Hongbosherlock
We add support int4 -> bf16 in awq_dequantize to inpelement int4-bf16 quantization.
Checklist