[qgallouedec]

WeightConverter source patterns are compiled with re.search, so a pattern ending in an identifier (e.g. experts.gate_up_proj) substring-matches sibling keys like experts.gate_up_proj_scale_inv. On save_pretrained, the reverse converter then routes the FP8 scale companion through [Chunk, SplitModulelist]. With block-quantized scales of shape (n_experts, 1, 1), torch.chunk(tensor, 2, dim=1) returns 1 piece instead of 2, and SplitModulelist.get_target_patterns raises:

ValueError: Undefined Operation encountered!

Scope. Only save_pretrained is affected! from_pretrained and inference round-trip cleanly. I.e. loading and using DeepSeek-v4 works.
The bug breaks any user flow that writes the model back to disk: re-exporting a loaded FP8 checkpoint, pushing a fork to the Hub, or checkpointing during fine-tuning.

It is not DeepSeek-V4-specific: any model whose conversion mapping has a multi-source WeightConverter with a leaf-style target name and a sibling key sharing that prefix (e.g. an FP8 *_scale_inv companion) hits the same substring-match. DeepSeek-V4 is the first in-tree model to combine those ingredients.

Fix

In WeightTransform.__init__, auto-anchor leaf-style source patterns at a key boundary when compiling the alternation regex:

if pattern and (pattern[-1].isalnum() or pattern[-1] == "_"):
    pattern = f"{pattern}(?=$|\\.)"

Patterns that already end in $ or in a separator (\. for prefix renames) are unaffected. Now experts.gate_up_proj matches ...experts.gate_up_proj but no longer matches ...experts.gate_up_proj_scale_inv.

Repro

import torch, tempfile
from torch import nn
from transformers import DeepseekV4Config, DeepseekV4ForCausalLM
from transformers.models.deepseek_v4.modeling_deepseek_v4 import DeepseekV4Experts

cfg = DeepseekV4Config(
    vocab_size=128, hidden_size=8, num_attention_heads=4, num_key_value_heads=1, num_hidden_layers=2, intermediate_size=32,
    quantization_config={"activation_scheme": "dynamic", "fmt": "e4m3", "quant_method": "fp8", "scale_fmt": "ue8m0", "weight_block_size": [128, 128]},
)
m = DeepseekV4ForCausalLM(cfg).to(dtype=torch.bfloat16)


def cdiv(a, b):
    return (a + b - 1) // b


# Attach all FP8 scale companions, matching what DeepSeek-v4 has.
for name, mod in m.named_modules():
    if isinstance(mod, nn.Linear) and not name.endswith("lm_head"):
        of, inf = mod.out_features, mod.in_features
        mod.register_parameter("weight_scale_inv", nn.Parameter(torch.ones(cdiv(of, 128), cdiv(inf, 128)), requires_grad=False))
    elif isinstance(mod, DeepseekV4Experts):
        n, gu_o, gu_i = mod.gate_up_proj.shape
        _, d_o, d_i = mod.down_proj.shape
        mod.register_parameter("gate_up_proj_scale_inv", nn.Parameter(torch.ones(n, cdiv(gu_o, 128), cdiv(gu_i, 128)), requires_grad=False))
        mod.register_parameter("down_proj_scale_inv", nn.Parameter(torch.ones(n, cdiv(d_o, 128), cdiv(d_i, 128)), requires_grad=False))

with tempfile.TemporaryDirectory() as d:
    m.save_pretrained(d)  # before fix: raises Undefined Operation; workaround: save_kwargs={"save_original_format": False}
    DeepseekV4ForCausalLM.from_pretrained(d)  # before fix (with workaround): MISSING list

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[zucchini-nlp]

imo we should allow it in case models need to replace anything that starts with "prefix", and the issue is in the regex of MoE in conversion mapping. We need to add an explicit /. in regex instead

in any case cc @Cyrilvallez to decide on it

[ArthurZucker]

yeah and probably error out? this is a good example where there's inteded vs not intended and we should probably not I think assume user wants one of the two

[Cyrilvallez]

Agreed with @zucchini-nlp! We cannot force such matching strategy without breaking matching logic in general. The specific deepseekv4 pattern should be changed instead! Let's just add a final dot to the pattern to avoid wrong matches

[qgallouedec]

closing in favour of #45930