Repository navigation
Description
Miles AMD Q3 Development Roadmap
1. Model support
1.1 DeepSeek-V4-Flash
Enablement
Bring DeepSeek-V4-Flash blockwise FP8 RL end to end on MI355X.
- Core Training Enablement
- [AMD] Enable DeepSeek-V4-Flash FP8 RL training on MI355X #1607
- Fix three blockers: CUDA-only imports, TE blockwise FP8 GEMM path, and unsupported BF16-to-FP32 torch.mm
- Added scripts/amd/run_deepseek_v4.py exposing verified configs.
- [AMD] DeepSeek-V4: use the ROCm precision-parity norm path #2040
- [AMD] fix(kernels): give gfx950 its own DSA tuning rows #4426
- [Fix] [AMD] dsv4: Read preshuffled indexer keys in TileLang sgl-project/sglang#49113
- [AMD] Enable DeepSeek-V4-Flash FP8 RL training on MI355X #1607
- Training Infrastructure
- Recipe & Configuration
- [AMD] Drop inert DSv4 rollout knobs and add an MTP recipe #1733
- [AMD] retune AMD 4-node dsv4 config #2139
- docs(amd): ROCm 7.2 image, tracker install, helper patch script, and a DeepSeek-V4-Flash DSpark config for MI355X sgl-project/SpecForge#888
- Disaggregate Recipe
- Add support for Deepseek-V4-Flash-0731
Performance Optimization
- Remove redundant FP8 scale transpose-copy: [ROCm] dsv4: remove the redundant fp8 scale transpose-copy on decode sgl-project/sglang#27289
- THD packed-sequence training: [feat] Enable THD packed-sequence training for DeepSeek-V4-Flash, with context parallelism #2038
- [Triton/Gluon] [Config] [Perf] Tune gfx950 FP8 blockscale selections for DSv4 decode ROCm/aiter#7614
1.2 Kimi K3
Bring Kimi K3 RL online on MI355X, building on SGLang's ROCm 7.2 TP8 serving path.
- Enable Kimi K3 LoRA training, rollout, and reward in Miles.
- Validate weight synchronization and multi-step RL correctness on 8× MI355X.
- Extend the validated workflow to full-parameter RL training.
- [sglang-miles] Fix Kimi-K3 weight reloads after graph capture sgl-project/sglang#41925
2. Feature support
2.1 LoRA support
Make LoRA RL reliable on AMD, starting with existing Miles recipes and expanding to colocated memory-saving workflows.
- Validate LoRA training, rollout, reward, and adapter weight synchronization on representative models.
- Add reproducible MI355X recipes and multi-step correctness coverage.
- Make LoRA compatible with
torch_memory_saveron ROCm, including pause/resume and colocated rollout. - [AMD][LoRA] Add a gpt-oss-20b MoE LoRA RL launcher #3157
2.2 OPD
Enable On-Policy Distillation on AMD by validating Miles' existing teacher-student workflows on MI355X.
- Validate the Qwen3-8B OPD recipes with both SGLang-served and Megatron-loaded teachers.
- Verify teacher/student log-prob alignment, reverse-KL, reward, and multi-step training correctness.
- Extend coverage to multi-teacher routing and colocated memory-saving workflows.
2.3 Speculative decoding (MTP)
- MTP Freeze
- MTP train
- Jointly train the target model and MTP head, then synchronize both with the rollout engine.
- Validate multi-step weight updates, loss stability, and target-draft consistency on MI355X.
- Measure acceptance length and end-to-end rollout speedup against the frozen-MTP baseline.
2.4 Low precision training
blockwise FP8
Enable end-to-end blockwise FP8 training and rollout on ROCm gfx950, covering training kernels, model recipes, weight handling, and SGLang runtime correctness.
- Training Enablement
- Train / Inference Consistency
- Runtime Correctness & Compatibility
- [AMD] Fix garbled unquantized Qwen3-30B-A3B output on ROCm/aiter where the aiter CK fused-MoE falls back to Triton with pre-shuffled weights sgl-project/sglang#28244
- [AMD-miles] Make ROCm aiter fp8 weight pre-shuffle idempotent for RL weight reload sgl-project/sglang#28963
- [Fix] Make RowParallelLinear k-size tuple-aware for FP8 sgl-project/sglang#30742
- [AMD] Gate Qwen3-30B-A3B TP2 qkv/o_proj CK block-FP8 GEMM shapes to Triton (ROCm 7.0) sgl-project/sglang#31405
Other
- MXFP8: Enable end-to-end training and rollout on MI355X, and validate weight synchronization, numerical stability, and performance.
- MXFP4: Bring up native model loading and rollout on MI355X, then extend the validated workflow to training.
- Quantization-Aware Training
2.5 True On-Policy training
Eliminate training-inference mismatch and align rollout and training log probabilities.
- Integrate Rollout Routing Replay (R3) for MoE expert routing.
- Add indexer replay for sparse-attention models, supporting SGLang
attn_tp > 1or DP attention. - Develop batch-invariant ROCm kernels for RMSNorm, GEMM, and attention.
- [Fix] [AMD] attention: Default deterministic inference to Triton on HIP sgl-project/sglang#48083
- [Fix] dspark: Forward target capture outputs from verification sgl-project/sglang#49094
- [Fix] [AMD] state_capturer: Gather MoRI routed-expert capture rows sgl-project/sglang#50605
2.6 Torch Memory Saver
Enable reliable torch_memory_saver for colocated RL by fixing the ROCr VMM release regression
- [AMD] fix GPU memory release for TMS pause on ROCm 7.2 #1680
- [AMD] DSv4: bring HIP compress-state pool into the memory_saver KV_CACHE region sgl-project/sglang#31747
- [AMD] Fix AITER custom all-gather CUDA-graph capture crash under torch_memory_saver sgl-project/sglang#30557
- [AMD] Fix AITER custom reduce-scatter CUDA-graph capture crash under torch_memory_saver sgl-project/sglang#34203
- [AMD] Preserve the AITER expert mask across torch_memory_saver pause/resume sgl-project/sglang#34220
- [AMD] Align HIP VMM allocation sizes to allocation granularity (#82 counterpart) fzyzcjy/torch_memory_saver#109
- fix(docker): bump torch_memory_saver to pick up the free-while-paused fix #3192
- [AMD] build(docker): bump ROCm torch_memory_saver to 118b09b #4318
- feat(rocm): allow retaining CPU backups on ROCm 7+ fzyzcjy/torch_memory_saver#119
3. Stable daily build Docker
ROCm base images for MI350X / MI300, plus daily Miles ROCm image build automation (latest sglang-miles branch + Miles commit).
- [AMD] Merge MI300/MI350-5 Dockerfiles #1294
- [AMD-miles] add amd-miles daily docker build workflow sgl-project/sglang#27765
- [AMD] Build Miles nightly ROCm images with docker/build.py and test ROCm 7.2 sgl-project/sglang#31757
- [AMD] use prebuilt wheels for flash-attn and sgl-model-gateway #1686
- [AMD] rocm720 MI355X image: Transformer Engine wheel + build fixes #1713
- [AMD] Fix pip setup in AMD Miles nightly builds sgl-project/sglang#32469
- [AMD] Publish an undated tag for the miles nightly images sgl-project/sglang#34521
- [AMD] Point the ROCm image at the v0.5.16 wheels release with TE 2.17.0 #2660
- [AMD] fix(docker): drop the obsolete ROCm Megatron fused-kernels patch #2768
- [AMD] docker: retire the rocm700 variants, add a rocm10-mi35x variant #2854
- [AMD] ci: move the miles nightlies from rocm700 to rocm10 sgl-project/sglang#37495
- docs(AMD): Add an AMD ROCm page and fix the AMD image and hardware notes #3209
- [AMD] fix(docker): drop stale hipify byproducts before moving the sglang checkout #3588
- [AMD] replace the ROCm 7.2.0 image with ROCm 7.2.4 #3326
- [AMD] ci: move the Miles ROCm 7.2 nightly build to 7.2.4 sgl-project/sglang#40387
- [AMD] docker: build the ROCm images on the sglang v0.5.20 release images #3873
4. AMD CI Stabilization
- [AMD] Track Miles MI355X daily images and enabled tests #2256
- [AMD CI] [1/N] Support Python 3.10 on ROCm base #1339
- [AMD] fix: support Python 3.10 Megatron-Bridge#12
- Fix biased wgrad with fp32 gradient accumulation XinyuJiangCMU/TransformerEngine#5
- [AMD] Fix QuantizedTensor import for ROCm TransformerEngine 2.8 Megatron-LM#58
- [AMD CI] [2/N] Add ROCm CI support for verified MI350X e2e tests #1379
- [AMD CI] Add nightly Miles ROCm 7.2 MI350X suites sgl-project/sglang#28853
- [AMD-miles] Add a ROCm 7.0 MI35x miles nightly image sgl-project/sglang#29219
- [AMD] Enable R3 CI and add the rocm700-mi35x image variant #1556
- [AMD-miles] Bump miles rocm700-mi35x base image and migrate nightly test onto it sgl-project/sglang#30587
- [AMD] Use bridge branch for ROCm Megatron-Bridge #1475
- rocm: honor HIP_VISIBLE_DEVICES in ray gpu helpers and NOSET list #1233
- [AMD] fix mooncake object store support #2039
- [AMD] Enable amd pr ci #2347
- [AMD CI] Enable verified ROCm tests #2480
- [AMD CI] follow the miles nightly-prefixed MI350 suite names sgl-project/sglang#34877
- Compile CP softmax LSE corrections with dynamic shapes XinyuJiangCMU/TransformerEngine#6
- ci: pin the DSv4 bshd/thd parity test to the miles impl #3162
- ci: Skip the actor forward-only pass in the Megatron FSDP-align test #3166
- [AMD] ci: expand ROCm nightly coverage and refresh runtime estimates #2541
- [AMD] ci: enable MI350-verified e2e tests on the ROCm nightly #3876
- [AMD] ci: expand ROCm10 nightly coverage on MI350 #3910
- fix(ci): add the missing pytest entry to three fast-gpu tests #3889
- [AMD] ci: bump ROCm SGLang to v0.5.22 and expand MI350 nightly coverage #4202
- [AMD] ci: add ten MI350 nightly tests #4325
- [AMD] ci: enable more tests in the MI350 nightly #4510
- [AMD] ci: bump ROCm images to SGLang v0.5.23 and expand MI350 nightly coverage #4599
- [Fix] Build DeepSeek RoPE caches on the default device, not hard-coded cuda ROCm/aiter#7298
- [HIP] [Bugfix] Skip empty inputs in custom all-gather ROCm/aiter#7302
- [HIP] [Bugfix] Skip grouped top-k launches with zero tokens ROCm/aiter#7376
- [Fix] [AMD] qwen3: Guard dtype alignment when fused decode omits KV sgl-project/sglang#47926
- [CI] [AMD] miles: Pass CI and the SGLang source path to nightly suites sgl-project/sglang#48067
- [Fix] [AMD] modelopt: Support ModelOpt FP8 Nemotron models on gfx950 sgl-project/sglang#48089
- [AMD] ci: enable more MI350 nightly tests and build their ROCm image dependencies #4642
- [AMD] ci: align ROCm PR CI with CUDA #4862
Activity
HaiShaw commentedon Aug 6, 2026
@XinyuJiangCMU Thanks for putting this together!
crystalhuanxie-del commentedon Aug 11, 2026
@XinyuJiangCMU
Hello, I’m interested in the Miles AMD development project and would like to contribute to it. However, I don’t know at which stage we are currently at. Can the task 2.5 be assigned to me, if there’s no specific person designated for it? Thank you!
XinyuJiangCMU commentedon Aug 13, 2026
Hi @crystalhuanxie-del
Thanks for offering to help. AMD already has a true on-policy path using the Triton attention backend, with an open SGLang PR for it, but that work is quite old and hasn’t been updated for a while, so I’m not sure it still works as-is.
For 2.5, I think batch-invariant ROCm kernels for RMSNorm, GEMM, and attention are still open areas. Feel free to pick one up and submit a PR.
crystalhuanxie-del commentedon Aug 13, 2026
Okay! I will take it. Thanks for your explanation.