Name and Version
Latest master, freshly cloned and compiled (June 2026). Build was compiled on Jetson Orin Nano Super 8gb (JetPack 6.2.1, aarch64) with CUDA.
Linux (Jetson Orin Nano Super, JetPack 6.2.1, aarch64) CUDA backend.
Using a custom LoRA fine-tune of Qwen/Qwen3.5-4B (dense VLM, image-text-to-text), merged in bf16 and converted to GGUF Q5_K_M with the latest convert_hf_to_gguf.py. Not yet uploaded publicly, happy to share the GGUF or a minimal repro if useful.
After fine-tuning Qwen/Qwen3.5-4B with Unsloth (LoRA, vision+language layers), merging the adapter into the base model, and converting to GGUF (bf16 first, then quantized to Q5_K_M), llama-server fails to load the resulting model with:
**missing tensor 'blk.32.attn_norm.weight'**
I verified this is not a problem with the merged model itself: I inspected the safetensors weight map directly (both for the official Qwen/Qwen3.5-4B base model and for my merged model) and confirmed both have exactly 32 transformer layers (indices 0–31), with the final layer (31) fully populated with all 11 expected tensors (input_layernorm, post_attention_layernorm, mlp.{gate,up,down}_proj, self_attn.{q,k}_norm, self_attn.{q,k,o}_proj). config.json's num_hidden_layers also correctly reports 32 on both base and merged model.
Despite this, the GGUF conversion or the loader appears to expect a 33rd block (index 32) that doesn't exist in the source model. This looks similar to other recently reported issues with Qwen3.5/3.6 hybrid architecture GGUF conversion (#24211 — missing tensor 'blk.60.attn_norm.weight' on a Qwen3.5-based MoE model with 60 real layers; #23033 — missing tensor 'blk.40.ssm_conv1d.weight'), suggesting a broader off-by-one or extra-block issue specific to how convert_hf_to_gguf.py writes block_count metadata for Qwen3.5-family hybrid models, independent of model size or dense/MoE variant.
convert_hf_to_gguf.py itself completes the conversion without error or warning, reporting tensors only up through blk.31 during the writing phase (427/427 tensors written successfully for the text backbone in my case).
Verified independently via safetensors.safe_open() on both the official Qwen/Qwen3.5-4B base model and my merged model: both report num_hidden_layers: 32 in config.json, and both have tensors only for layer indices 0–31, with layer 31 fully populated (no missing sub-tensors).
This suggests the issue is in how the GGUF header/metadata (block_count) is computed or interpreted for Qwen3.5-family models, not in the source model weights or the conversion script's tensor-writing logic itself.
Happy to provide the full GGUF, the merged safetensors, or run additional diagnostics if helpful.
good evening for everyone :D
Operating systems
Linux
GGML backends
CUDA
Hardware
Jetson Orin nano Super 8gb with jetpack 6.2.1
Models
Qwen 3.5 4B
Problem description & steps to reproduce
Steps to reproduce this particular isue:
-Fine-tune Qwen/Qwen3.5-4B with any LoRA adapter (vision+language layers enabled), or even just load the base model unmodified.
Merge adapter into base (merge_and_unload()), save as clean bf16 safetensors.
-Convert with python convert_hf_to_gguf.py --outtype bf16 --outfile model-bf16.gguf.
-Quantize with llama-quantize model-bf16.gguf model-Q5_K_M.gguf Q5_K_M.
-Attempt to load with llama-server -m model-Q5_K_M.gguf --mmproj mmproj-f16.gguf.
-Observe failure: missing tensor 'blk.32.attn_norm.weight'.
-On the first bad commit there was no response — using freshly cloned master, unable to bisect further without more time invested. Happy to help test a fix if pointed to a suspect commit/PR.
First Bad Commit
No response — using freshly cloned master, unable to bisect further without more time invested. Happy to help test a fix if pointed to a suspect commit/PR.
Relevant log output
Logs
0.00.082.864 I srv init: using 5 threads for HTTP server
0.00.083.608 I srv start: binding port with default address family
0.00.084.996 I srv llama_server: loading model
0.00.085.013 I srv load_model: loading model '/mnt/usb/models/wendy-v11.0-Q5_K_M.gguf'
0.00.690.206 I srv load_model: [mtmd] estimated worst-case memory usage of mmproj is 892.48 MiB
0.00.690.266 I common_init_result: fitting params to device memory ...
0.00.690.267 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.02.007.239 E llama_model_load: error loading model: missing tensor 'blk.32.attn_norm.weight'
0.02.007.295 E llama_model_load_from_file_impl: failed to load model
0.02.007.412 E common_fit_params: encountered an error while trying to fit params to free device memory: failed to load model
Name and Version
Latest master, freshly cloned and compiled (June 2026). Build was compiled on Jetson Orin Nano Super 8gb (JetPack 6.2.1, aarch64) with CUDA.
Linux (Jetson Orin Nano Super, JetPack 6.2.1, aarch64) CUDA backend.
Using a custom LoRA fine-tune of Qwen/Qwen3.5-4B (dense VLM, image-text-to-text), merged in bf16 and converted to GGUF Q5_K_M with the latest convert_hf_to_gguf.py. Not yet uploaded publicly, happy to share the GGUF or a minimal repro if useful.
After fine-tuning Qwen/Qwen3.5-4B with Unsloth (LoRA, vision+language layers), merging the adapter into the base model, and converting to GGUF (bf16 first, then quantized to Q5_K_M), llama-server fails to load the resulting model with:
**missing tensor 'blk.32.attn_norm.weight'**
I verified this is not a problem with the merged model itself: I inspected the safetensors weight map directly (both for the official Qwen/Qwen3.5-4B base model and for my merged model) and confirmed both have exactly 32 transformer layers (indices 0–31), with the final layer (31) fully populated with all 11 expected tensors (input_layernorm, post_attention_layernorm, mlp.{gate,up,down}_proj, self_attn.{q,k}_norm, self_attn.{q,k,o}_proj). config.json's num_hidden_layers also correctly reports 32 on both base and merged model.
Despite this, the GGUF conversion or the loader appears to expect a 33rd block (index 32) that doesn't exist in the source model. This looks similar to other recently reported issues with Qwen3.5/3.6 hybrid architecture GGUF conversion (#24211 — missing tensor 'blk.60.attn_norm.weight' on a Qwen3.5-based MoE model with 60 real layers; #23033 — missing tensor 'blk.40.ssm_conv1d.weight'), suggesting a broader off-by-one or extra-block issue specific to how convert_hf_to_gguf.py writes block_count metadata for Qwen3.5-family hybrid models, independent of model size or dense/MoE variant.
convert_hf_to_gguf.py itself completes the conversion without error or warning, reporting tensors only up through blk.31 during the writing phase (427/427 tensors written successfully for the text backbone in my case).
Verified independently via safetensors.safe_open() on both the official Qwen/Qwen3.5-4B base model and my merged model: both report num_hidden_layers: 32 in config.json, and both have tensors only for layer indices 0–31, with layer 31 fully populated (no missing sub-tensors).
This suggests the issue is in how the GGUF header/metadata (block_count) is computed or interpreted for Qwen3.5-family models, not in the source model weights or the conversion script's tensor-writing logic itself.
Happy to provide the full GGUF, the merged safetensors, or run additional diagnostics if helpful.
good evening for everyone :D
Operating systems
Linux
GGML backends
CUDA
Hardware
Jetson Orin nano Super 8gb with jetpack 6.2.1
Models
Qwen 3.5 4B
Problem description & steps to reproduce
Steps to reproduce this particular isue:
-Fine-tune Qwen/Qwen3.5-4B with any LoRA adapter (vision+language layers enabled), or even just load the base model unmodified.
Merge adapter into base (merge_and_unload()), save as clean bf16 safetensors.
-Convert with python convert_hf_to_gguf.py --outtype bf16 --outfile model-bf16.gguf.
-Quantize with llama-quantize model-bf16.gguf model-Q5_K_M.gguf Q5_K_M.
-Attempt to load with llama-server -m model-Q5_K_M.gguf --mmproj mmproj-f16.gguf.
-Observe failure: missing tensor 'blk.32.attn_norm.weight'.
-On the first bad commit there was no response — using freshly cloned master, unable to bisect further without more time invested. Happy to help test a fix if pointed to a suspect commit/PR.
First Bad Commit
No response — using freshly cloned master, unable to bisect further without more time invested. Happy to help test a fix if pointed to a suspect commit/PR.
Relevant log output
Logs
0.00.082.864 I srv init: using 5 threads for HTTP server
0.00.083.608 I srv start: binding port with default address family
0.00.084.996 I srv llama_server: loading model
0.00.085.013 I srv load_model: loading model '/mnt/usb/models/wendy-v11.0-Q5_K_M.gguf'
0.00.690.206 I srv load_model: [mtmd] estimated worst-case memory usage of mmproj is 892.48 MiB
0.00.690.266 I common_init_result: fitting params to device memory ...
0.00.690.267 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.02.007.239 E llama_model_load: error loading model: missing tensor 'blk.32.attn_norm.weight'
0.02.007.295 E llama_model_load_from_file_impl: failed to load model
0.02.007.412 E common_fit_params: encountered an error while trying to fit params to free device memory: failed to load model