Skip to content

[Cherry-pick][v0.34.0] [Model] Add Mistral Large 4 (#64534) - #67568

Merged
khluu merged 1 commit into
vllm-project:releases/v0.34.0from
NickLucche:cherry-pick-ml4-v0.34.0
Nov 13, 2026
Merged

khluu merged 1 commit into
vllm-project:releases/v0.34.0from
NickLucche:cherry-pick-ml4-v0.34.0

Conversation

@NickLucche

Copy link
Copy Markdown
Member

Cherry-pick of #64534 (Mistral Large 4) onto releases/v0.34.0, as a single commit (git cherry-pick -x 25e9f7b3f2) with the conflicts resolved against the release branch's APIs. No prerequisite PRs are pulled in (#61452, #66532, #65349, #64560, #61243, #66037 are not on the release branch).

Conflict resolutions

Behaviour matches main:

Test plan

  • pre-commit on the commit: all hooks pass.
  • pytest (GB300):
    • tests/models/mistral: 114 passed, 10 skipped.
    • ML4 config: 7 passed. modelopt: 2 passed. Registry: 11 passed.
    • indexer DCP localize: 126 passed. Sparse MLA DCP/HiSparse/top-k: 126 passed.
    • KV planning + utils: 194 passed, 1 failed. The failure (test_finalize_extensible_kv_cache_auto_max_model_len_warns_when_short[True]) fails the same way on the release head without this commit.
    • Worker attn_utils/utils/pcp_manager/dspark_utils: 252 passed. Breakable cudagraph: 27 passed. NIXL HMA + HiSparse connector: 117 passed.
    • tests/test_config.py and the HF-dependent quark test fail on this cluster's offline HF cache, the same as without this commit.
  • Serving (Mistral Large 4 NVFP4, DSpark depth 3, 2x GB300), compared with the same runs on main:
Config GSM8K-200 Acceptance length Notes
TP8 (vision, mistral parsers) 0.960 (main 0.960) 3.491 (3.491) API checks 15/18 (same as main)
TP8 DCP8 0.970 (0.970) 3.525 (3.525)
TP1 x PCP8 x DCP8 0.975 (0.970-0.975) 3.511 (3.515)
1P1D NIXL (P DP4/EP, D TP4) 0.975 (0.965-0.98) D 3.450 (3.46-3.48) chat + image 7/7, 0 NIXL failures
  • Short-context TPOT p50, TP8: c1 2.90 ms (main 2.90), c32 9.23-9.51 ms (main 9.34-9.38).
  • No tracebacks.

AI assistance was used for this change.

Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Andy Luo <andy.luo@amd.com>
Co-authored-by: Julien Denize <julien.denize@mistral.ai>
Co-authored-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Co-authored-by: Julien Debache <jdebache@nvidia.com>
Co-authored-by: vanshil shah <49670736+vanshilshah97@users.noreply.github.com>
(cherry picked from commit 25e9f7b)

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added ci/build mistral Related to Mistral models new-model Requests to new models labels Nov 13, 2026
@mergify mergify Bot added the dflash label Nov 13, 2026
@mergify mergify Bot added the mrv2 Model Runner V2 specific label Nov 13, 2026
@mergify mergify Bot added the kv-connector label Nov 13, 2026
@github-project-automation github-project-automation Bot moved this to Todo in AMD Nov 13, 2026
@mergify mergify Bot added the hisparse Related to HiSparse label Nov 13, 2026
@khluu
khluu merged commit fa87f89 into vllm-project:releases/v0.34.0 Nov 13, 2026
32 of 33 checks passed
@github-project-automation github-project-automation Bot moved this from Todo to Done in AMD Nov 13, 2026
@github-project-automation github-project-automation Bot moved this from Backlog to Done in Sprint - DFlash Nov 13, 2026
@github-project-automation github-project-automation Bot moved this to Done in NVIDIA Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/build dflash hisparse Related to HiSparse kv-connector mistral Related to Mistral models mrv2 Model Runner V2 specific new-model Requests to new models nvidia quantization rocm Related to AMD ROCm speculative-decoding

Projects

Status: Done
Status: Done
Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants