Repository navigation
feat(config): add TokenSpeed Muse Spark Open MXFP4 MI355X AgentX / feat(config): 新增 TokenSpeed Muse Spark Open MXFP4 MI355X AgentX 配置 - #4962
Conversation
…AnalysisAI#4962 Signed-off-by: Eric Feng <Eric.Feng@amd.com>
Add a TP8/EP1 TokenSpeed AgentX configuration for Muse Spark Open MXFP4 on MI355X with the DFlash2 draft, at concurrency 1, 2, 4, 8 and 16. - Recipe, README and amd-master entry; the public 20261109 TokenSpeed nightly, kernel, kernel-amd and engine wheels pinned by SHA256, and the runner image by digest. - tokenspeed-nightly-rocm.sh installs the pinned wheels and keeps the image's ROCm torch. - The golden-AL connector and spec identity accept TokenSpeed DFLASH: throughput uses golden acceptance with the greedy sampler; evals verify with the triton sampler and speculative sampling. Signed-off-by: Eric Feng <Eric.Feng@amd.com>
…AnalysisAI#4962 Signed-off-by: Eric Feng <Eric.Feng@amd.com>
Signed-off-by: Eric Feng <Eric.Feng@amd.com>
…k revisions - Pin TokenSpeed 0.1.1.post20261110, kernel and kernel-amd 0.1.4.post20261110 and engine 0.1.0.post20261110 (all from e8b98dd83) by SHA256, with scheduler 0.1.53. - Pin the checkpoint with revision and the DFlash2 draft with speculative-draft-model-revision, which this nightly resolves offline. Signed-off-by: Eric Feng <Eric.Feng@amd.com>
…eval sampling - The connector accepts TokenSpeed MTP and DFLASH; throughput removes enable-speculative-sampling under the greedy sampler. - Eval-only runs verify MTP with flashinfer and DFLASH with triton and speculative sampling. Signed-off-by: Eric Feng <Eric.Feng@amd.com>
9ad0914 to
6cf38d7
Compare
|
Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package. 中文大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要,改动通常也很小。请让你的 agent 清理 PR,使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。 |
Keep the infx diff minimal: TokenSpeed's spec_parameters builds one spec for MTP and DFLASH, the speculative-config error keeps its original wording, and the identity test for the DFlash2 curve is dropped since the DFLASH synthetic-acceptance test already selects that curve. Connector overrides are unchanged for throughput, eval-only, non-agentic and non-speculative runs. Signed-off-by: Eric Feng <Eric.Feng@amd.com>
…mi355x-agentx Signed-off-by: Eric Feng <Eric.Feng@amd.com> # Conflicts: # inferencex-e2e/perf-changelog.yaml
…mi355x-agentx Signed-off-by: Eric Feng <Eric.Feng@amd.com> # Conflicts: # inferencex-e2e/perf-changelog.yaml
…mi355x-agentx Signed-off-by: Eric Feng <Eric.Feng@amd.com> # Conflicts: # inferencex-e2e/perf-changelog.yaml
|
@functionstackx thanks for helping add the label to trigger CI! can you also help approve the workflow? also @raikonenfnu |
|
It seems like the label also needs to be removed + readded to trigger CI even after workflows have been approved after a rebase unfortuantely |
|
Dispatched trusted sweep run for approved external revision 已为获批的外部提交 |
…pes set The TokenSpeed Muse Spark MI355X recipe set PYTHONNOUSERSITE, PYTHONUNBUFFERED, PYTHONSAFEPATH and NO_COLOR in its global environment, which srt-slurm also exports into the batch script and so into the cluster's host_setup hook; no other MI355X recipe sets any of them globally. Follow the MI355X recipes: PYTHONNOUSERSITE moves to roles.agg.env, where most of them set it, and the rest are dropped. Also drop TOKENSPEED_JIT_COMPILE_CHECK=warn, which is the engine default, and TOKENSPEED_HEALTH_CHECK_TIMEOUT, which the pinned TokenSpeed build no longer reads. The TokenSpeed wheel pins stay global because the install script also runs in the SMG gateway's container. Signed-off-by: Eric Feng <Eric.Feng@amd.com>
…mi355x-agentx Signed-off-by: Eric Feng <Eric.Feng@amd.com> # Conflicts: # inferencex-e2e/perf-changelog.yaml
|
Dispatched trusted sweep run for approved external revision 已为获批的外部提交 |
…mi355x-agentx Signed-off-by: Eric Feng <Eric.Feng@amd.com> # Conflicts: # inferencex-e2e/perf-changelog.yaml
Summary
Add
musespark-fp4-mi355x-tokenspeed-agentic-dflash: Muse Spark Open MXFP4 (amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4) AgentX on 8x MI355X with TokenSpeed, attention TP8 / MoE TP8 EP1, the DFlash2 draft (7 draft tokens), FP8 KV cache, at concurrency 1, 2, 4, 8 and 16 with per-concurrency running-request caps and prefill/decode intervals (recipe, README).0.1.1.post20261110, ROCm 7.2 kernel and kernel-amd0.1.4.post20261110, engine0.1.0.post20261110(all from lightseekorg/tokenspeed@e8b98dd83), scheduler0.1.53,lightseekorg/tokenspeed-runner-amd:rocm7.2.4-pytorch-2.14.0-gfx950by digest. SHA256s are in the recipe;tokenspeed-nightly-rocm.shinstalls them and keeps the image's ROCm torch.c7bc74fd64371315a091bc15f9723882eead0dc1, DFlash2 draft46c90f412549c472941a68382d91a99e5e6372a0(--speculative-draft-model-revision). DFlash2 draft served as it ships (BF16).Additional detail
RadixArk/Muse-Spark-Open-DFlash2); effective BF16 (no draft quantization flag; TokenSpeed's default handling).AI model disclosure
PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs):full-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied.OWNER/MEMBER/COLLABORATOR) has commented/use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.中文
新增
musespark-fp4-mi355x-tokenspeed-agentic-dflash:在 8 张 MI355X 上用 TokenSpeed 运行 Muse Spark Open MXFP4(amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4)的 AgentX 配置。注意力 TP8,MoE TP8 EP1,使用 DFlash2 草稿模型(7 个草稿 token),FP8 KV cache,并发 1、2、4、8、16,每个并发点设置各自的运行请求上限和 prefill/decode 间隔。0.1.1.post20261110,ROCm 7.2 kernel 与 kernel-amd0.1.4.post20261110,engine0.1.0.post20261110(均构建自 lightseekorg/tokenspeed@e8b98dd83),scheduler0.1.53,runner 镜像按 digest 固定。SHA256 写在 recipe 中;tokenspeed-nightly-rocm.sh安装这些 wheel 并保留镜像自带的 ROCm torch。c7bc74fd64371315a091bc15f9723882eead0dc1,DFlash2 草稿模型46c90f412549c472941a68382d91a99e5e6372a0(--speculative-draft-model-revision)。DFlash2 草稿模型按发布原样(BF16)运行。