Repository navigation
Multi-LoRA for DeepSeek-V4.1-Flash on the raw model path - #15
Conversation
V4.1 checkpoints name modules layers.N.attn.* and layers.N.ffn.*, with the output head at head.weight. wo_a is a grouped einsum weight rather than a linear module, so it is not a target. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Maps V4.1 HF targets (attention wq_a/wkv/wq_b/wo_b and shared-expert w1/w2/w3) to the Miles model's Megatron modules, wraps them with Megatron-Bridge multi-LoRA layers, freezes the router's expert bias, and exports one slot as HF PEFT tensors: TP shards gathered, the fused shared-expert linear_fc1 split into w1/w3, trimmed to the slot's rank. V4.1 attention gathers its sequence-parallel input before its projections, so its adapters skip sequence-parallel collectives and the replicated wq_a/wkv adapters are not TP-summed. Routed experts are rejected for now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Multi-LoRA keeps --megatron-to-hf-mode raw for models with a Miles model definition (deepseek_v41) and resolves their targets from the HF layout without Megatron-Bridge; every other model is still switched to bridge. bshd with --micro-batch-size 1 and no dynamic batch size is accepted: one sample per micro-batch keeps each adapter's tokens contiguous. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In raw mode the multi-LoRA actor builds the model from the Miles model provider with adapters applied before DDP wrapping. Base weights load with every multi-LoRA wrapper swapped for its base linear, so mbridge and torch_dist see the base model's exact modules and names. Adapter export goes through the model's raw exporter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fused multi-LoRA forward multiplies raw weight shards, so under tensor parallelism the gradient reaching a TP-sharded lora_B input (and the adapter input of a column-parallel base without sequence parallelism) holds only the local rank's share. lora_A gradients of every column- and row-parallel adapter were off by roughly the TP size, and the partial input gradient leaked into upstream layers. use_tp_exact_multi_lora_forward reproduces the fused forward with the same copy-to-TP-region boundaries Megatron's parallel linears use. The DeepSeek-V4.1 multi-LoRA path enables it; on a TP=2 run its A/B gradients match a single-device reference to bf16 precision. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under sequence parallelism Megatron-Bridge asks a layernorm-fused column linear to return its gathered layernorm output, and Transformer Engine then drops the gradient flowing back through that output, so the adapter's input gradient never reaches upstream layers. None of the V4.1 targets is layernorm-fused; fail at build time if one ever is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With CUDA lazy module loading, the first use of a kernel can synchronize the CUDA context. DeepSeek-V4.1 multi-LoRA runs expert parallelism across nodes with per-rank token counts, so cuBLAS picks first-use grouped-GEMM kernels on only some ranks, mid-forward, while expert all-to-all kernels are in flight. A hung run had four ranks blocked in cuKernelSetAttribute inside cublasLtMatmul, waiting on their own GPUs, and every other rank waiting on them. Raw multi-LoRA trainers now start with CUDA_MODULE_LOADING=EAGER unless --train-env-vars sets it. Eager loading costs about 2 GiB per GPU. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| ) | ||
|
|
||
|
|
||
| def raw_multi_lora_needs_eager_module_loading(args) -> bool: |
There was a problem hiding this comment.
While testing, we hit the deadlock described in the NVIDIA lazy loading docs here:
Some programs incorrectly assume that concurrent kernel execution is guaranteed. A deadlock can occur if cross-kernel synchronization is required, but kernel execution has been serialized.
DeepSeek-V4.1 multi-LoRA runs expert parallelism across nodes, and each rank gets a different number of tokens per expert. On one rank, the CPU launched a routed-expert GEMM kernel (kernel B) for the first time. With lazy loading, the first launch loads B's code, and that load waits for kernels already running on the GPU, blocking the CPU. One of those running kernels was kernel A, a NCCL collective that can't finish until the other ranks send their data. But the other ranks were waiting on this rank: their next all-to-all needs the expert output kernel B was supposed to produce. B waits on A, A waits on the other ranks, and the other ranks wait on B, so every rank hangs.
So the stall starts within one rank, but it becomes a deadlock because kernel A is a collective that depends on the other ranks. Loading CUDA modules eagerly at startup removes the first-time load from the forward pass
…anch V4.1 raw-mode multi-LoRA lands there via modal-projects/miles#15; the build fails if it has not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-level imports Target resolution has no Megatron imports so argument parsing stays light; export needs only torch and megatron.core so its CPU tests keep running; the Bridge-dependent wrapping, TP-exact forward and unwrap helpers move out of slots.py into the V4.1 module. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| mid = reduce_from_tensor_model_parallel_region(mid) | ||
| else: | ||
| mid = gather_from_tensor_model_parallel_region(mid) | ||
| mid = copy_to_tensor_model_parallel_region(mid) |
There was a problem hiding this comment.
This line is the TP gradient fix. With TP-sharded adapters, every rank sees the full rank-r activation mid, but each rank holds only its shard of lora_B and produces only its slice of the output:
forward mid ──► B_0·mid ─► y_0 (rank 0)
└─► B_1·mid ─► y_1 (rank 1)
backward dL/dmid = B_0ᵀ·dy_0 + B_1ᵀ·dy_1 ← each rank computes one term
copy_to_tensor_model_parallel_region is the identity in forward and an all-reduce in backward, so it sums those terms before they reach lora_A. The upstream fused forward has no such op, so lora_A (and everything upstream of the adapter) trains on this rank's share alone: 72–82% gradient error at TP 2 against a single-device reference. Megatron's ColumnParallelLinear uses the same op on its input for the same reason.
There was a problem hiding this comment.
We're upstreaming a fix to Megatron bridge here. After that, we can remove the diff added in miles_plugins/models/deepseek_v4_1/lora.py in this PR.
There was a problem hiding this comment.
Added a note on _tp_exact_multi_lora_forward: remove once the pinned Megatron-Bridge includes NVIDIA-NeMo/Megatron-Bridge#6513. Miles pins the radixark fork (radixark/Megatron-Bridge@8cd3466), so the fix also needs to be cherry-picked there before this copy can go.
…anch V4.1 raw-mode multi-LoRA lands there via modal-projects/miles#15; the build fails if it has not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| mid = reduce_from_tensor_model_parallel_region(mid) | ||
| else: | ||
| mid = gather_from_tensor_model_parallel_region(mid) | ||
| mid = copy_to_tensor_model_parallel_region(mid) |
There was a problem hiding this comment.
We're upstreaming a fix to Megatron bridge here. After that, we can remove the diff added in miles_plugins/models/deepseek_v4_1/lora.py in this PR.
is_raw_multi_lora_enabled raises for models outside RAW_MULTI_LORA_MODEL_TYPES, so model building, base-weight loading and adapter export can no longer route another model into the DeepSeek-V4.1 path. Note when the copied TP-exact forward can go. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
train.py already forces CUDA_MODULE_LOADING=EAGER for raw multi-LoRA trainers. The routed-expert rejection and the fused fc1 split were each tested twice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Merging to enable the DeepSeek-V4.1-Flash path now. Upstream, radixark#4205 adds V4.1 LoRA through Miles' native path and will likely supersede the model-side pieces here; the multi-adapter wrapping and export are what it doesn't cover. We'll track it and decide from there what still needs upstreaming. The TP gradient fix is already upstream as NVIDIA-NeMo/Megatron-Bridge#6513. |
Summary
DeepSeek-V4.1-Flash LoRA, part 2 of 5. Serve adapters (sglang#80) → train them (this PR) → Miles v0.1.2 and BF16 assets (spindle#73) → LoRA on Miles v0.1.2 (spindle#74) → the recipe (spindle#75). The two fork PRs merge first.
Multi-LoRA training for DeepSeek-V4.1 on Miles' own model ("raw" mode). Megatron-Bridge has no V4.1 mapping, so the multi-LoRA path could not build, load or export it.
wq_a,wkv,wq_b,wo_band the shared expertw1/w2/w3. Routed-expert targets fail with a clear error. The router, Engram andwo_astay frozen.deepseek_v41, and every dispatch point checks it. bshd is accepted at micro-batch 1 without dynamic batching.fc1split intow1/w3.lora_Agradients 72–84% off at TP 2. A patched forward fixes it on the V4.1 path only. The upstream fix is Megatron-Bridge#6513; once it lands, the copy can go.CUDA_MODULE_LOADING=EAGER, which costs about 1.9 GiB per GPU.E2E
Tiny random-weight V4.1 (2 layers) on 2×H100, TP 2 with sequence parallelism, through the real
MultiLoRATrainRayActor:save_slot/load_slotFull-size DeepSeek-V4.1-Flash on 2×8 B300 (TP 4, EP 16), through Spindle and the stock Tinker SDK:
forward_backward.Upstream
This enables the DeepSeek-V4.1-Flash path now. Upstream, radixark/miles#4205 (with its serving counterpart sgl-project/sglang#45231) adds V4.1 LoRA through Miles' native single-adapter path and will likely supersede the model-side pieces here once it ships in a Miles release. The multi-adapter wrapping and export are the part it doesn't cover. We'll track both and decide from there what still needs upstreaming from this PR. The tensor-parallel gradient fix is already upstream as Megatron-Bridge#6513.
🤖 Generated with Claude Code