Conversion to GGUF bug: --mistral-format failed to be applied

[FedericoFB]

Git commit

I tried converting mistral-small-4-nvfp4 to GGUF. llama.cpp did not take the --mistral-format option into account; it looked for config.json and failed to start the conversion.

Operating systems

Windows

GGML backends

CUDA

Problem description & steps to reproduce

download from HF mistralai/Mistral-Small-4-119B-2603-NVFP4 and run the conversion script convert_hf_to_gguf.py with the options highlighted in screenshot (ython convert_hf_to_gguf.py [PATH]\Mistral-Small-4-119B-2603-NVFP4 --outtype auto --outfile [PATH]\Mistral-Small-4-119B-2603-NVFP4.gguf --mistral-format

First Bad Commit

I'm not familiar with it; I tried the 11355, 11370, and 11371.

Compile command

no need to compile

Relevant log output

G:\llama.cpp\Source 13.3>python convert_hf_to_gguf.py G:\AI_Models\HF-downloaded\Mistral-Small-4-119B-2603-NVFP4 --outtype auto --outfile G:\AI_Models\FedericoFB\Mistral-Small-4-119B-2603-NVFP4.gguf --mistral-format
INFO:hf-to-gguf:Loading model: Mistral-Small-4-119B-2603-NVFP4
INFO:hf-to-gguf:gguf: indexing model part 'consolidated-00001-of-00013.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'consolidated-00002-of-00013.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'consolidated-00003-of-00013.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'consolidated-00004-of-00013.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'consolidated-00005-of-00013.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'consolidated-00006-of-00013.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'consolidated-00007-of-00013.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'consolidated-00008-of-00013.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'consolidated-00009-of-00013.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'consolidated-00010-of-00013.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'consolidated-00011-of-00013.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'consolidated-00012-of-00013.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'consolidated-00013-of-00013.safetensors'
INFO:hf-to-gguf:heuristics detected bfloat16 tensor dtype, setting --outtype bf16
INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
WARNING:hf-to-gguf:Failed to load model config from G:\AI_Models\HF-downloaded\Mistral-Small-4-119B-2603-NVFP4: Unrecognized model in G:\AI_Models\HF-downloaded\Mistral-Small-4-119B-2603-NVFP4. Should have a `model_type` key in its config.json.
WARNING:hf-to-gguf:Trying to load config.json instead
Traceback (most recent call last):
  File "G:\llama.cpp\Source 13.3\conversion\base.py", line 1280, in load_hparams
    config = AutoConfig.from_pretrained(dir_model, trust_remote_code=False).to_dict()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Federico\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\transformers\models\auto\configuration_auto.py", line 442, in from_pretrained
    raise ValueError(
ValueError: Unrecognized model in G:\AI_Models\HF-downloaded\Mistral-Small-4-119B-2603-NVFP4. Should have a `model_type` key in its config.json.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "G:\llama.cpp\Source 13.3\convert_hf_to_gguf.py", line 312, in <module>
    main()
  File "G:\llama.cpp\Source 13.3\convert_hf_to_gguf.py", line 285, in main
    model_instance = model_class(dir_model, output_type, fname_out,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\llama.cpp\Source 13.3\conversion\mistral.py", line 120, in __init__
    super().__init__(*args, **kwargs)
  File "G:\llama.cpp\Source 13.3\conversion\deepseek.py", line 246, in __init__
    hparams: dict = ModelBase.load_hparams(self.dir_model, is_mistral_format=False)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\llama.cpp\Source 13.3\conversion\base.py", line 1288, in load_hparams
    with open(dir_model / "config.json", "r", encoding="utf-8") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'G:\\AI_Models\\HF-downloaded\\Mistral-Small-4-119B-2603-NVFP4\\config.json'

[CISC]

Should be self.is_mistral_format instead of is_mistral_format=False, if that works feel free to submit a PR.

[FedericoFB]

it worked