Skip to content

[AMD] retune AMD 4-node dsv4 config - #2139

Merged
guapisolo merged 2 commits into
radixark:mainfrom
JessicaJiang-123:feat/dsv4-rocm-unified-kv-rollout
Aug 4, 2026
Merged

guapisolo merged 2 commits into
radixark:mainfrom
JessicaJiang-123:feat/dsv4-rocm-unified-kv-rollout

Conversation

@JessicaJiang-123

Copy link
Copy Markdown
Contributor

No description provided.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

XinyuJiangCMU added a commit to XinyuJiangCMU/miles that referenced this pull request Aug 4, 2026
Zhiyao's retune, applied by hand because our copy of this file has drifted:
the two changes her diff makes to the response length and to mem-fraction are
already in ours at the same values, so only four land.

- TP8 -> TP4 for the 32-GPU config
- SGLANG_HACK_FLASHMLA_BACKEND triton -> unified_kv_triton, which lives in
  compressor v2 only, so SGLANG_OPT_USE_COMPRESSOR_V2 goes back in alongside it
- SGLANG_OPT_USE_JIT_NORM false -> true

That last one reverses radixark#2040. It is taken deliberately: the
logprobdiff regression that pinned it to false was in the v1 norm+rope path,
and this config forces compressor v2, so the old reasoning may no longer hold.
Nothing has measured it either way on v2 yet, so train_rollout_logprob_abs_diff
on the next run is what decides it. Healthy is around 0.045 against a noise
floor of 0.0015; the regression that produced the pin read 0.131.

Co-authored-by: Zhiyao Jiang <jessicajiang324@gmail.com>
@guapisolo
guapisolo merged commit 3507ce1 into radixark:main Aug 4, 2026
36 checks passed
@XinyuJiangCMU

Copy link
Copy Markdown
Contributor

With these changes, on 4 nodes (32x MI355X), a steady step at 8k takes ~150s for rollout and ~182s for actor training, with a full cycle time of ~7.9 min.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants