IlyasMoutawwakil · Aug 13, 2026 · Aug 15, 2026 · Aug 16, 2026 · Aug 18, 2026 · Aug 18, 2026
diff --git a/docs/source/en/experts_interface.md b/docs/source/en/experts_interface.md
@@ -117,7 +117,7 @@ The kernel is loaded lazily on the first forward.
 
 ### FP8 and FP4 quantized experts
 
-DeepSeek-style checkpoints are usually pre-quantized and carry their own quantization config, so you don't need to pass a [`FineGrainedFP8Config`]. The `"deepgemm"` backend automatically picks the FP8 (or FP4 on Blackwell) grouped-GEMM kernel. DeepGEMM requires dynamic per-row activation scales (`activation_scheme="dynamic"`) and rejects static (per-tensor) activation quantization.
+DeepSeek-style checkpoints are usually pre-quantized and carry their own quantization config, so you don't need to pass a [`FineGrainedConfig`]. The `"deepgemm"` backend automatically picks the FP8 (or FP4 on Blackwell) grouped-GEMM kernel. DeepGEMM requires dynamic per-row activation scales (`activation_scheme="dynamic"`) and rejects static (per-tensor) activation quantization.
 
 ```py
 from transformers import AutoModelForCausalLM
@@ -133,14 +133,14 @@ For FP4-packed expert weights (DeepSeek V4-style), the GPU must be SM100+ (Black
 > [!NOTE]
 > On Blackwell (SM100+), the `"deepgemm"` and `"deepgemm_megamoe"` experts kernels require power-of-two UE8M0 expert scales. A checkpoint quantized with plain `float32` scales (`scale_fmt="float"`) raises a `ValueError` on the first forward instead of silently corrupting the output. Load a checkpoint quantized with `scale_fmt="ue8m0"`, or switch to `grouped_mm` or `batched_mm`, which consume `float32` block scales directly. Hopper (SM90+) consumes `float32` scales on the DeepGEMM path without conversion.
 
-The main reason to pass a [`FineGrainedFP8Config`] for a pre-quantized checkpoint is to dequantize it back to `bfloat16`, in which case the experts run in `bfloat16` rather than on the FP8/FP4 DeepGEMM path.
+The main reason to pass a [`FineGrainedConfig`] for a pre-quantized checkpoint is to dequantize it back to `bfloat16`, in which case the experts run in `bfloat16` rather than on the FP8/FP4 DeepGEMM path.
 
 ```py
-from transformers import AutoModelForCausalLM, FineGrainedFP8Config
+from transformers import AutoModelForCausalLM, FineGrainedConfig
 
 model = AutoModelForCausalLM.from_pretrained(
     "deepseek-ai/DeepSeek-V3",
-    quantization_config=FineGrainedFP8Config(dequantize=True),
+    quantization_config=FineGrainedConfig(dequantize=True),
     experts_implementation="deepgemm",
 )
 ```

diff --git a/docs/source/en/main_classes/quantization.md b/docs/source/en/main_classes/quantization.md
@@ -98,6 +98,19 @@ Learn how to quantize models in the [Quantization](../quantization/overview) gui
 
 [[autodoc]] SpQRConfig
 
+## FineGrainedConfig
+
+> [!WARNING]
+> `FineGrainedConfig` supersedes `FineGrainedFP8Config`, `Mxfp4Config` and `NVFP4Config`, which are
+> frozen and will be removed in a future release. The `fp8`, `mxfp8`, `mxfp4` and `nvfp4`
+> `quant_method` keys already route here, so existing checkpoints need no change.
+
+[[autodoc]] FineGrainedConfig
+
+## FineGrainedGroup
+
+[[autodoc]] FineGrainedGroup
+
 ## FineGrainedFP8Config
 
 [[autodoc]] FineGrainedFP8Config

diff --git a/docs/source/en/model_doc/deepseek_v32.md b/docs/source/en/model_doc/deepseek_v32.md
@@ -58,11 +58,11 @@ In DeepSeek-V3.2 **every layer runs its own indexer** — there is no cross-laye
 DeepSeek-V3.2-Exp is distributed as an FP8 checkpoint. The indexer projections are kept out of FP8 quantization, since the checkpoint stores them in bf16/fp32:
 
 ```python
-from transformers import FineGrainedFP8Config, AutoModelForCausalLM, AutoTokenizer
+from transformers import FineGrainedConfig, AutoModelForCausalLM, AutoTokenizer
 import torch
 
 model_name = "deepseek-ai/DeepSeek-V3.2-Exp"
-quantization_config = FineGrainedFP8Config(
+quantization_config = FineGrainedConfig(
     modules_to_not_convert=["model.layers.*.mlp.gate.*", "*.self_attn.indexer.weights_proj.*"],
     weight_block_size=(128, 128),
 )

diff --git a/docs/source/en/model_doc/minimax_m3_vl.md b/docs/source/en/model_doc/minimax_m3_vl.md
@@ -153,7 +153,7 @@ from transformers import (
     AutoModelForImageTextToText,
     AutoTokenizer,
     CompileConfig,
-    FineGrainedFP8Config,
+    FineGrainedConfig,
 )
 from transformers.distributed import DistributedConfig
 
@@ -164,7 +164,7 @@ model = AutoModelForImageTextToText.from_pretrained(
     "MiniMaxAI/MiniMax-M3-preview",
     dtype=torch.bfloat16,
     # Dequantize the native MXFP8 weights to bf16 at load (the speed win); needs even TP/EP sharding.
-    quantization_config=FineGrainedFP8Config(dequantize=True),
+    quantization_config=FineGrainedConfig(dequantize=True),
     distributed_config=DistributedConfig(
         tp_size=int(os.environ["WORLD_SIZE"]),
         ep_size=int(os.environ["WORLD_SIZE"]),

diff --git a/docs/source/en/quantization/finegrained_fp8.md b/docs/source/en/quantization/finegrained_fp8.md
@@ -16,6 +16,13 @@ rendered properly in your Markdown viewer.
 
 # Fine-grained FP8
 
+> [!WARNING]
+> `FineGrainedFP8Config` is frozen and receives no new recipes. [`FineGrainedConfig`] supersedes it, serving
+> block-FP8, MXFP8, MXFP4 and NVFP4 through one config for dense linears, embeddings and MoE
+> experts alike. Existing checkpoints keep loading — the `fp8`, `mxfp8`, `mxfp4` and `nvfp4`
+> `quant_method` keys now route to the fine-grained integration — so this page is here for
+> reference while the frozen classes remain. They will be removed in a future release.
+
 Fine-grained FP8 quantization quantizes the weights and activations to fp8.
 
 - The weights are quantized to 8-bits for each 2D block (`weight_block_size=(128, 128)`).

diff --git a/docs/source/en/quantization/mxfp4.md b/docs/source/en/quantization/mxfp4.md
@@ -16,6 +16,13 @@ rendered properly in your Markdown viewer.
 
 # MXFP4
 
+> [!WARNING]
+> `Mxfp4Config` is frozen and receives no new recipes. [`FineGrainedConfig`] supersedes it, serving
+> block-FP8, MXFP8, MXFP4 and NVFP4 through one config for dense linears, embeddings and MoE
+> experts alike. Existing checkpoints keep loading — the `fp8`, `mxfp8`, `mxfp4` and `nvfp4`
+> `quant_method` keys now route to the fine-grained integration — so this page is here for
+> reference while the frozen classes remain. They will be removed in a future release.
+
 Note: MXFP4 quantization currently only works for OpenAI GPT-OSS 120b and 20b.
 
 MXFP4 is a 4-bit floating point format that dramatically reduces the memory requirements of large models. Large models (GPT-OSS-120B) can fit on a single 80GB GPU and smaller models (GPT-OSS-20B) only require 16GB of memory. It uses blockwise scaling to preserve its range and accuracy, which typically becomes degraded at lower precisions.

diff --git a/docs/source/en/quantization/nvfp4.md b/docs/source/en/quantization/nvfp4.md
@@ -16,6 +16,13 @@ rendered properly in your Markdown viewer.
 
 # NVFP4
 
+> [!WARNING]
+> `NVFP4Config` is frozen and receives no new recipes. [`FineGrainedConfig`] supersedes it, serving
+> block-FP8, MXFP8, MXFP4 and NVFP4 through one config for dense linears, embeddings and MoE
+> experts alike. Existing checkpoints keep loading — the `fp8`, `mxfp8`, `mxfp4` and `nvfp4`
+> `quant_method` keys now route to the fine-grained integration — so this page is here for
+> reference while the frozen classes remain. They will be removed in a future release.
+
 NVFP4 quantization packs full-precision linear weights into NVIDIA's 4-bit floating-point format while a model is
 loaded. [`NVFP4Config`] replaces eligible bias-free `torch.nn.Linear` modules, whose `in_features` and `out_features` are both divisible by 16, with an NVFP4 linear implementation from
 the [NVFP4 Hub kernel](https://huggingface.co/kernels-community/nvfp4-gemm). The model's attention and MLP interfaces are

diff --git a/src/transformers/__init__.py b/src/transformers/__init__.py
@@ -236,7 +236,9 @@
         "CompressedTensorsConfig",
         "EetqConfig",
         "FbgemmFp8Config",
+        "FineGrainedConfig",
         "FineGrainedFP8Config",
+        "FineGrainedGroup",
         "FourOverSixConfig",
         "FPQuantConfig",
         "GemmaQuantizationConfig",
@@ -737,7 +739,9 @@
     from .utils.quantization_config import CompressedTensorsConfig as CompressedTensorsConfig
     from .utils.quantization_config import EetqConfig as EetqConfig
     from .utils.quantization_config import FbgemmFp8Config as FbgemmFp8Config
+    from .utils.quantization_config import FineGrainedConfig as FineGrainedConfig
     from .utils.quantization_config import FineGrainedFP8Config as FineGrainedFP8Config
+    from .utils.quantization_config import FineGrainedGroup as FineGrainedGroup
     from .utils.quantization_config import FourOverSixConfig as FourOverSixConfig
     from .utils.quantization_config import FPQuantConfig as FPQuantConfig
     from .utils.quantization_config import GemmaQuantizationConfig as GemmaQuantizationConfig

diff --git a/src/transformers/conversion_mapping.py b/src/transformers/conversion_mapping.py
@@ -546,16 +546,21 @@ def _build_checkpoint_conversion_mapping():
             ),
         ],
         "deepseek_v4": [
+            # checkpoint-form renames the general `.attn.` one below would otherwise take first
+            WeightRenaming(source_patterns=r"\.attn\.wq_b\.", target_patterns=r".attn.q_b_proj."),
             # General mid-key renames acting on several keys
             WeightRenaming(source_patterns=r"\.attn\.", target_patterns=r".self_attn."),
             WeightRenaming(source_patterns=r"\.ffn\.", target_patterns=r".mlp."),
             WeightRenaming(source_patterns=r"\.indexer\.compressor\.", target_patterns=r"\.compressor\.indexer\."),
             # Leaf patterns
-            WeightRenaming(source_patterns=r"^embed\.weight$", target_patterns="embed_tokens.weight"),
+            # The checkpoint has no `model.` prefix: named here, so a save takes it back off
+            WeightRenaming(source_patterns=r"^layers\.", target_patterns="model.layers."),
+            WeightRenaming(source_patterns=r"^norm\.weight$", target_patterns="model.norm.weight"),
+            WeightRenaming(source_patterns=r"^embed\.weight$", target_patterns="model.embed_tokens.weight"),
             WeightRenaming(source_patterns=r"^head\.weight$", target_patterns="lm_head.weight"),
-            WeightRenaming(source_patterns=r"^hc_head_fn$", target_patterns="hc_head.hc_fn"),
-            WeightRenaming(source_patterns=r"^hc_head_base$", target_patterns="hc_head.hc_base"),
-            WeightRenaming(source_patterns=r"^hc_head_scale$", target_patterns="hc_head.hc_scale"),
+            WeightRenaming(source_patterns=r"^hc_head_fn$", target_patterns="model.hc_head.hc_fn"),
+            WeightRenaming(source_patterns=r"^hc_head_base$", target_patterns="model.hc_head.hc_base"),
+            WeightRenaming(source_patterns=r"^hc_head_scale$", target_patterns="model.hc_head.hc_scale"),
             WeightRenaming(source_patterns=r"\.attn_norm\.", target_patterns=r".input_layernorm."),
             WeightRenaming(source_patterns=r"\.ffn_norm\.", target_patterns=r".post_attention_layernorm."),
             WeightRenaming(source_patterns=r"\.hc_attn_fn$", target_patterns=r".attn_hc.fn"),
@@ -570,10 +575,13 @@ def _build_checkpoint_conversion_mapping():
                 target_patterns=r"\.compressor\.indexer\.scorer\.weights_proj\.",
             ),
             WeightRenaming(source_patterns=r"\.indexer\.wq_b\.", target_patterns=r"\.compressor\.indexer\.q_b_proj\."),
-            WeightRenaming(source_patterns=r"\.norm\.", target_patterns=r"\.kv_norm\."),
+            # the compressors' norms only: the attention's own is `kv_norm` already, and a save reversing a
+            # bare `.kv_norm.` would rename it too (`[.]`: the reverse unescapes a group's `\.`)
+            WeightRenaming(
+                source_patterns=r"\.compressor\.(|indexer[.])norm\.", target_patterns=r"\.compressor\.\1kv_norm\."
+            ),
             WeightRenaming(source_patterns=r"\.ape$", target_patterns=r"\.position_bias"),
             WeightRenaming(source_patterns=r"\.wq_a\.", target_patterns=r".q_a_proj."),
-            WeightRenaming(source_patterns=r"\.self_attn\.wq_b\.", target_patterns=r"\.self_attn\.q_b_proj\."),
             WeightRenaming(source_patterns=r"\.wkv\.", target_patterns=r".kv_proj."),
             WeightRenaming(source_patterns=r"\.wgate\.", target_patterns=r".gate_proj."),
             WeightRenaming(source_patterns=r"\.wo_a\.", target_patterns=r".o_a_proj."),
@@ -683,7 +691,12 @@ def _build_checkpoint_conversion_mapping():
                 source_patterns=r"^patch_merge_mlp\.linear_2\.",
                 target_patterns="model.multi_modal_projector.merge_linear_2.",
             ),
-            WeightRenaming(source_patterns=r"\.block_sparse_moe\.", target_patterns=r"\.mlp\."),
+            # the MoE block only, so a save does not reverse the dense and vision `.mlp.` too (`[.]`: the
+            # reverse unescapes a group's `\.`)
+            WeightRenaming(
+                source_patterns=r"\.block_sparse_moe\.(experts[.]|gate[.]|shared_experts[.]|e_score_correction_bias$)",
+                target_patterns=r"\.mlp\.\1",
+            ),
             WeightRenaming(
                 source_patterns=r"\.e_score_correction_bias", target_patterns=r"\.gate\.e_score_correction_bias"
             ),