Skip to content

fix: Multi-LoRA correctness fixes across models - #3584

Merged
yueming-yuan merged 11 commits into
mainfrom
yueming/multilora-disable-mtp
Sep 25, 2026
Merged

yueming-yuan merged 11 commits into
mainfrom
yueming/multilora-disable-mtp

Conversation

@yueming-yuan

@yueming-yuan yueming-yuan commented Sep 21, 2026 •

Copy link
Copy Markdown
Collaborator

Cutoff dependency PRs: SGLang #41074 and Bridge #42. SGLang #41074 now contains expert-index bounds plus the complete fused shared-expert pool/loading fix. Both Dockerfiles pin the merged Bridge #42 commit 8cd3466d; the built PR image includes the shared-FC2 fix. Qwen full-model CI uses the exact SGLang split SHA below.

Fix Multi-LoRA loss/state isolation and preserve requested numerical settings.

  • Disable implicit MTP loss and reject explicit Multi-LoRA MTP training.
  • Freeze router expert bias so tenants cannot mutate shared base routing state.
  • Update the CUDA/ROCm Dockerfiles to Bridge 8cd3466d, containing the isolated shared-FC2 TP reduction fix (+8/-2 in Bridge).
  • Forward RoPE, activation-fusion and router flags to Bridge LoRA providers, preserving Bridge fusion compatibility checks.

Broader follow-up fixes: Megatron #104 (MoE accumulation, absorbed MLA hooks), Bridge #41 (shared/row-parallel and absorbed MLA adapter math), SGLang #40997 (router precision only; bounds and fused shared adapters are in #41074). Bridge PR overrides are not automatically installed by Miles CI.

Model results, validation limits and the v0.1.1 split by commit are tracked in #3284. The dependency PRs mix narrow fixes and larger execution changes; their full heads are not an A-only release batch.

Qwen3-MoE inference router precision and its P2P staging fix moved to #3697, paired with SGLang #40997 after cutoff.

For cutoff validation, SGLang is pinned to the bounds and fused-shared branch; Megatron #104 is not selected. Full-model Qwen CI does not exercise the Multi-LoRA shared-FC2 wrapper.

ci-sglang-pr: c4e2e3cfa881f794aa9af9d540caaecdaca6949d
ci-megatron-pr: 651dd728e3c1f763b5ba3a8de4ac0d01f70e621c

Validation: full local pre-commit passed after the split. Earlier Qwen full-model/LoRA CI passed with the router changes included; the new bounds/fused-shared dependency combination is pending CI. The P2P RDMA transfer-error logs also reproduce on unmodified main at 398dba8e34 on the same runner (main CI); final weight impact remains unverified.

@yueming-yuan
yueming-yuan marked this pull request as ready for review September 21, 2026 23:52

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@yueming-yuan yueming-yuan changed the title fix: Prevent implicit MTP loss in Multi-LoRA training fix: Isolate Multi-LoRA loss and frozen router state Sep 22, 2026
@yueming-yuan yueming-yuan changed the title fix: Isolate Multi-LoRA loss and frozen router state fix: Multi-LoRA model support Sep 22, 2026
@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

✅ tests/e2e/megatron/test_qwen3_30B_A3B_p2p.py passed on stage-c-8-gpu-h100 in 2h02m07s — workflow run

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

✅ tests/e2e/megatron/model_scripts/test_kimi_k25_2layer_ci.py passed on stage-c-4-gpu-h200 in 4h56m28s — workflow run

@yueming-yuan yueming-yuan changed the title fix: Multi-LoRA model support fix: Multi-LoRA correctness fixes across models Sep 24, 2026
@yueming-yuan yueming-yuan added the run-ci-multi-lora Run Tinker cookbook and multi-tenant gateway GPU e2e tests label Sep 25, 2026

@Zhichenzzz Zhichenzzz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@yueming-yuan
yueming-yuan merged commit a73fe58 into main Sep 25, 2026
51 of 57 checks passed
@yueming-yuan
yueming-yuan deleted the yueming/multilora-disable-mtp branch September 25, 2026 06:43
shawnghu added a commit to shawnghu/miles that referenced this pull request Oct 2, 2026
…aining is off

Since the Megatron bump in radixark#2673, Megatron derives MTP labels from input_ids whenever
labels=None and backprops the MTP loss on every training forward once the MTP block
exists. The bridge builds that block from the HF config (mtp_num_hidden_layers /
num_nextn_predict_layers), so every bridge-mode run on an MTP checkpoint trained the
decoder (and LoRA adapters) on an extra next-next-token loss over its own rollouts,
independent of the RL loss, even without --enable-mtp-training.

- full-model bridge: build the MTP block only with --enable-mtp-training
- LoRA bridge: never build it; reject --enable-mtp-training (the MTP head is frozen
  base weights and is not synced, so it could not train, while its loss still reached
  the adapters undetached). Previously only multi-LoRA did this (radixark#3584).
- raw path: keep the block for checkpoint compatibility but zero its loss scale when
  --mtp-num-layers is set without --enable-mtp-training
- test_bridge_mtp_detachment: assert the block is not built when MTP training is off

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci-megatron run-ci-multi-lora Run Tinker cookbook and multi-tenant gateway GPU e2e tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants