Repository navigation
[Model] Add Mistral Large 4 - #64534
Merged
Merged
Conversation
Contributor
|
Documentation preview: https://vllm--64534.org.readthedocs.build/en/64534/ |
cjackal
reviewed
Nov 12, 2026
Comment on lines
+1334
to
+1336
|
|
||
| if self.is_aiter_triton_fp4_bmm_enabled: | ||
| mqa_ql_nope = rocm_aiter_ops.batched_gemm_a16wfp4( |
Contributor
Member
Author
|
/ci run |
|
✅ Triggered Buildkite CI #102097 for commit ℹ️ This PR is 1 commit behind upstream |
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
Member
Author
|
/ci run |
|
✅ Triggered Buildkite CI #102470 for commit |
Member
Author
|
/ci run |
|
✅ CI is already running for this commit: https://buildkite.com/vllm/ci/builds/102470 |
Add Mistral Large 4 (mistralai/Mistral-Large-4-1T-A52B and the NVFP4 checkpoint) on model runner V2, with sparse MLA attention, cross-layer KV sharing, the DSpark speculative decoding drafter, DCP/PCP long-context support, NIXL P/D, HiSparse host KV offload, and NVIDIA and ROCm kernels. Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Felix Marty <Felix.Marty@amd.com> Co-authored-by: Hongxia Yang <hongxia.yang@amd.com> Co-authored-by: Andy Luo <andy.luo@amd.com> Co-authored-by: Julien Denize <julien.denize@mistral.ai> Co-authored-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com> Co-authored-by: Julien Debache <jdebache@nvidia.com> Co-authored-by: vanshil shah <49670736+vanshilshah97@users.noreply.github.com> Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Move vllm/models/mistral_large4 to vllm/models/mistral/mistral_large4 so the
Mistral Large 4 model sits next to the shared Mistral building blocks
(vllm/models/mistral/{common,nvidia,amd}). The inner layout (platform
dispatch __init__, nvidia/, amd/) is unchanged.
- Update absolute imports, the two model registry entries
(MistralLarge4ForCausalLM, MistralLarge4DSparkModel), the ruff per-file
ignore for specialized_decode/mono, and the README test path.
- Move tests/models/mistral_large4/amd_specialized_decode to
tests/models/mistral/mistral_large4/amd_specialized_decode to mirror the
package, and fix imports in the ML4 DSpark, modelopt and quark tests.
- No compatibility shim at the old path: nothing in the repo or the
recipes loads the model through it.
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
The ROCm specialized decode kernel was adapted from ROCm/ATOM's GLM-5.2 mono kernel and kept its identifiers and docstrings, although every geometry value is ML4's. Rename GLM5_*/glm5_*/Glm5MonoKernel to ML4 names and reword the comments; keep the provenance lines in the README, the vendored mono/__init__.py header and the FlyDSL port note. No functional change. Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
- HY-V4: accept and forward the new topk_length argument in its _fp8_flash_mla_kernel override (the DCP decode path now passes it). - ROCm specialized decode: vLLM SPDX headers (MIT code from ROCm/ATOM keeps its attribution, as for the DeepSeek-V4 AMD mono code), regex instead of re, torch.accelerator instead of torch.cuda, typo fixes and targeted type-ignores for mypy in the vendored kernel code. - Buildkite: run tests/models/mistral on B200 and MI355. Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
test_hf_registry_coverage and test_registry_imports require an example entry for every registered architecture. Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Use file-level ruff noqa comments in the vendored FlyDSL mono kernel instead of a pyproject.toml per-file ignore, so the change no longer touches pyproject.toml. Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
fused_indexer_q_rope_quant asserts a CUDA platform, so the use_rope=True cases of test_indexer_q failed on MI355. Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Member
Author
|
/ci run |
|
✅ Triggered Buildkite CI #102482 for commit |
This was referenced Nov 13, 2026
1 task done
2 of 6 tasks
6 tasks done
6 tasks done
6 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mistral Large 4 support for vLLM, shared as an open working branch. This PR is not meant to be merged as is; generic pieces are being upstreamed in separate PRs.
Recipe: https://recipes.vllm.ai/mistralai/Mistral-Large-4-1T-A52B
AI assistance was used for this change.