Feature request
Support register customize quantization method out-of-tree.
The usage would be as follows:
from transformers.quantizers import HfQuantizer
from transformers.quantizers import regsiter_quantization_config, register_quantizer
from transformers.utils.quantization_config import QuantizationConfigMixin
@regsiter_quantization_config("custom")
class CustomFakeQuantizationConfig(QuantizationConfigMixin):
"""The custom fake quantization config."""
@register_quantizer("custom")
class CustomFakeQuantizer(HfQuantizer):
"""The custom fake quantizer."""Motivation
We would greatly appreciate it if HuggingFace could support registering custom quantization schemes externally. This would allow us to integrate the schemes of any LLM quantization tools and evaluate fake quantization models using the powerful combination of lm_eval + huggingface. Thank you for considering this!
Similar features have already been supported by vLLM, see:
Your contribution
If this feature request is considered, I'd happily submit a PR to implement it.
Feature request
Support register customize quantization method out-of-tree.
The usage would be as follows:
Motivation
We would greatly appreciate it if HuggingFace could support registering custom quantization schemes externally. This would allow us to integrate the schemes of any LLM quantization tools and evaluate fake quantization models using the powerful combination of
lm_eval+huggingface. Thank you for considering this!Similar features have already been supported by vLLM, see:
Your contribution
If this feature request is considered, I'd happily submit a PR to implement it.