Skip to content

model: add support for Mellum2 architecture#1919

Merged
ikawrakow merged 1 commit into
ikawrakow:mainfrom
joelfarthing:mellum2-support
Jun 4, 2026
Merged

model: add support for Mellum2 architecture#1919
ikawrakow merged 1 commit into
ikawrakow:mainfrom
joelfarthing:mellum2-support

Conversation

@joelfarthing

Copy link
Copy Markdown
Contributor

Summary

Adds JetBrains Mellum2 support, including:

  • mellum architecture registration and model type reporting
  • Mellum tensor mapping/loading
  • graph support for Mellum's mixed sliding-window/full-attention layer pattern
  • per-layer SWA/full RoPE handling
  • routed MoE expert tensor loading
  • Mellum2 tokenizer pre-tokenizer support
  • converter support for official JetBrains/Mellum2-12B-A2.5B-* weights
  • converter metadata for Mellum sliding-window pattern and full-attention/SWA RoPE settings

This PR does not include MTP/speculative decoding, performance tuning, custom quantization work, or chat/server UX changes.

Validation

Official JetBrains Instruct source revision:

4ee5751ef73ac6ae5a65b76b092ffc7c3b9c60e3

Public branch for reviewer inspection:

https://github.com/joelfarthing/ik_llama.cpp/tree/mellum2-support

Reviewer/community convenience GGUFs generated from official JetBrains Instruct BF16 weights:

https://huggingface.co/ji-farthing/Mellum2-12B-A2.5B-Instruct-ik-llama-GGUF

These GGUFs were converted with this PR's ik_llama.cpp converter. They embed the stock JetBrains chat_template.jinja and include the Mellum RoPE/YARN metadata emitted by this PR. They are expected to run on current llama.cpp builds with Mellum2 support as well as this ik_llama.cpp branch.

Validation highlights:

  • current origin/main rejects Mellum2 GGUFs with unknown model architecture: 'mellum'
  • converted BF16 and Q8_0 include Mellum rope.freq_base, rope.freq_base_swa, YaRN scaling metadata, sliding-window size, and sliding-window layer pattern
  • converted BF16 and Q8_0 embed the stock JetBrains Instruct chat template byte-for-byte
  • Q8_0 passes ik_llama.cpp CUDA server long-code smoke with --cpu-moe
  • Q8_0 passes current llama.cpp upstream CUDA server long-code smoke with --cpu-moe
  • BF16 passes current llama.cpp upstream CUDA server long-code smoke with --cpu-moe
  • long-code smoke responses stop normally, compile with python3 -m py_compile, and pass a functional topological-sort test including cycle detection
  • llama-server and llama-quantize CUDA build targets are up to date
  • converter/GGUF py_compile passes
  • git diff --check passes

Notes / caveats

  • The linked GGUFs are convenience artifacts for review and community testing; they are not performance or quality claims.

  • The model's normal logits path uses the standard output head. MTP/draft-head support is not required for normal Mellum2 generation and is intentionally out of scope here.

  • The uploaded BF16 is a reference conversion; Q8_0 is the practical reviewer/community download.

  • No custom chat template is used. The embedded template matches JetBrains/Mellum2-12B-A2.5B-Instruct/chat_template.jinja.

  • No performance claims are made.

  • I have read the contributing guidelines

  • Self-reported review complexity:

    • Low
    • Medium
    • High

@ikawrakow ikawrakow merged commit dc51c6f into ikawrakow:main Jun 4, 2026
@joelfarthing joelfarthing deleted the mellum2-support branch June 16, 2026 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants