Skip to content

Commit 3041a7f

Browse files
chore: retire MiniMax-M3 and DeepSeek-V4-Pro / chore: 停用 MiniMax-M3 和 DeepSeek-V4-Pro (#4664)
* chore: retire MiniMax-M3 and DeepSeek-V4-Pro serving benchmarks * docs: link model retirement to PR 4664 * fix: preserve historical changelog links when recording retirement * fix: handle retirement-only merge ingest without result artifacts * docs: clarify retirement merge ingest ordering
1 parent 1f87891 commit 3041a7f

122 files changed

Lines changed: 566 additions & 19793 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.claude/commands/add-model-hardware.md‎

Lines changed: 9 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -9,17 +9,17 @@ closest existing sibling recipe, then ship it as one `[Klaud Cold]` PR with a fu
99
Inputs from `$ARGUMENTS` (links first, then SKU. The rest are optional and order-tolerant):
1010

1111
- **model-link** (required). HuggingFace URL of the exact checkpoint to benchmark, e.g.
12-
`https://huggingface.co/MiniMaxAI/MiniMax-M3-MXFP8`. Derive from it: the `model:` id
12+
`https://huggingface.co/moonshotai/Kimi-K3`. Derive from it: the `model:` id
1313
(`org/repo`), the **precision** (from the repo name, where `MXFP8`/`FP8`→`fp8`, `NVFP4`/`FP4`→`fp4`,
14-
`INT4`→`int4`, else `bf16`), and the **model-prefix** (e.g. `minimaxm3`).
14+
`INT4`→`int4`, else `bf16`), and the **model-prefix** (e.g. `kimik3`).
1515
- **gpu-sku** (required). Choose from `b200 | b300 | h100 | h200 | gb200 | mi300x | mi325x | mi355x`.
1616
Determines the master config (`mi*`→`amd-master.yaml`, else `nvidia-master.yaml`), the image
1717
repo (`vllm/vllm-openai` vs `vllm/vllm-openai-rocm`), and the launcher.
1818
- **recipes-link** (optional). The model's `recipes.vllm.ai` page (or a vendor recipe commit).
1919
Consult it for the authoritative serve flags (block size, parsers, attention backend,
2020
parallelism guidance). If omitted, copy the sibling recipe's flags.
2121
- **draft-model-link** (optional). HF URL of a speculative-decoding draft (e.g.
22-
`https://huggingface.co/Inferact/MiniMax-M3-EAGLE3`). Its presence means **build the MTP
22+
`https://huggingface.co/Inferact/Kimi-K3-DSpark`). Its presence means **build the MTP
2323
variant** (EAGLE3 with this draft). See the MTP appendix.
2424
- **mtp** (optional). Force the `spec-decoding: mtp` variant even without a draft link (use
2525
native MTP if the checkpoint ships `num_mtp_modules > 0`).
@@ -180,19 +180,13 @@ Finish on a clean `main`.
180180
## Appendix — MTP / EAGLE3 spec-decoding variant
181181

182182
When a **draft-model-link** is given (or `mtp` is forced), build the `spec-decoding: mtp`
183-
sibling of the base recipe. Use the provided draft id as `--speculative-config.model`. The
184-
proven setup for **MiniMax-M3** (merged for b300/b200/h100/h200/mi355x/mi300x) uses the
185-
external **`Inferact/MiniMax-M3-EAGLE3`** draft, `method: eagle3`, **3 speculative tokens**:
186-
```
187-
--speculative-config "{\"method\": \"eagle3\", \"model\": \"$DRAFT_MODEL\", \"num_speculative_tokens\": 3<CUDA: , \"attention_backend\": \"FLASH_ATTN\">}"
188-
```
189-
- **CUDA (b*/h*).** Pin the drafter to `FLASH_ATTN` because FlashInfer can't run the MHA EAGLE3 head
190-
at the mandatory page-size 128. Scale cudagraph capture to `CONC * (1 + NUM_SPEC_TOKENS)`.
191-
- **ROCm (mi*)**: no backend pin (server runs `TRITON_ATTN`). Use an image that already carries
192-
the upstream `SupportsEagle3` fix (`vllm-project/vllm#45546`); the old in-place model patch was
193-
retired with the legacy bash scripts. Copy the sibling `minimaxm3/vllm/mi*-mtp` recipe.
183+
sibling of the base recipe. Use the provided draft id as `--speculative-config.model` when
184+
the model uses an external draft. Copy a current sibling recipe for the same model,
185+
framework, and hardware; select the method, draft length, and attention backend supported
186+
by that checkpoint and pinned image. MiniMax-M3 is retired; MiniMax-M3.1-Flash uses its own
187+
DSpark recipes under `minimaxm31flash/`.
194188
- **All**: speculative fixed-sequence recipes get `USE_CHAT_TEMPLATE: 'true'` from the binder; AgentX
195189
recipes set it themselves. Raw random tokens tank spec-decode acceptance. Search space mirrors the non-MTP
196190
entry trimmed at the extreme-conc end, latency rows starting at conc 1, `tp2-ep2` dropped.
197191
- Other models may instead use **native MTP** (`method: mtp`, no external draft) when the
198-
checkpoint ships MTP modules (`num_mtp_modules > 0`), e.g. the DeepSeek-V4 recipes.
192+
checkpoint ships MTP modules (`num_mtp_modules > 0`), e.g. the GLM-5.3 recipes.

‎.claude/commands/recover-failed-ingest.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,11 @@ ETL or mapping. Missing output is not repaired by rerunning a database ingest.
7070

7171
## Verify recovery eligibility
7272

73+
Retirement-only merges have no benchmark results to ingest or recover; do not
74+
re-append their `retired: true` entries. For mixed changes, the recovery tool
75+
excludes retired entries and recovers only entries with results. Keep the
76+
planner's requirement that retired keys existed in the original base revision.
77+
7378
The target must be completed, on `main`, with event `push` and the workflow above.
7479
A green legacy dispatch is not proof of valid data; leave bogus historical rows alone.
7580
Resolve `FAILED_RUN_OR_JOB_URL`, `ORIGINAL_PR`, and `ORIGINAL_MERGE_SHA`:

‎.github/AGENT_OPERATIONS.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ Sweeps do not trigger while a PR has merge conflicts. For `inferencex-e2e/perf-c
3232

3333
Pushes to `main` never run a sweep: `.github/workflows/run-sweep.yml` is PR-only. A push that changes `inferencex-e2e/perf-changelog.yaml` runs `.github/workflows/merge-ingest.yml` (Merge Ingest). Its single `ingest` job validates the merged PR's authorized `/use <run_id>` (or legacy `/reuse-sweep-run`) command and source run, then dispatches ingest of that reused PR sweep. Without valid reuse authorization the run fails, and nothing is benchmarked or ingested. `[skip-sweep]` only skips PR benchmark setup; changelog validation and reuse authorization checks still run, and Merge Ingest ignores it.
3434

35+
Retirement-only pushes are a no-results exception: every newly appended entry must have `retired: true`, and the planner must verify its keys existed in the push's base revision and are absent from the head. Merge Ingest uploads the changelog metadata and skips reuse lookup and database dispatch. Merge these PRs directly after review and required checks, without `/use`; empty or mixed changelogs still use the normal reuse gate.
36+
3537
Reuse is rejected while the PR currently carries `evals-only` or `agentx-fast`; this is checked when `/use` is acknowledged and at merge (and by `merge_with_reuse`). On a push these labels skip the reuse gate, so a primary label starts a fresh sweep. Source-run label history is not inspected, so pin only runs produced without those labels. Staging (`/stage-results`, also triggered by `/use <run_id>`) still requires a primary label on the PR and a source run created while one was applied. See `.github/workflows/README.md` and `uv run --project inferencex-e2e --extra workflows python -m infx.workflows.merge_with_reuse` for eligibility and merge behavior.
3638

3739
## Workflow dispatch and monitoring

‎.github/workflows/README.md‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -301,15 +301,23 @@ pushes a synchronization commit, waits for checks, then merges.
301301

302302
At merge, `merge-ingest.yml` ("Merge Ingest") publishes the reused sweep. It
303303
runs on pushes to `main` that change `inferencex-e2e/perf-changelog.yaml`. Its
304-
single `ingest` job resolves the merge commit's PR, reuse command, and source
305-
run with `infx.workflows.reuse`, computes the changelog delta with
306-
`infx.matrix.plan`, and fails before uploading or dispatching anything unless
307-
reuse is validly authorized. It then uploads merge-time `changelog-metadata`
304+
single `ingest` job first computes the changelog delta with `infx.matrix.plan`.
305+
For entries with results, it resolves the merge commit's PR, reuse command, and
306+
source run with `infx.workflows.reuse`, and fails before uploading or dispatching
307+
anything unless reuse is validly authorized. It then uploads merge-time `changelog-metadata`
308308
and sends one `repository_dispatch` to InferenceX-app: `ingest-agentic-results`
309309
(with `database-target: production`) when the delta has agentic entries,
310310
otherwise `ingest-results`. The payload's `source-run-id` is the reused PR
311311
`run-sweep.yml` run and its `merge-run-id` is the Merge Ingest run.
312312

313+
For a nonempty delta consisting entirely of `retired: true` entries, the planner
314+
verifies that the selected configs existed in the base and are absent from the
315+
head. There are no result artifacts to reuse: merge these PRs directly after
316+
review and required checks, without `/use` or `merge_with_reuse`. Merge Ingest
317+
uploads the validated changelog metadata but skips reuse lookup and database
318+
dispatch. Empty or mixed deltas do not receive this exception. Ingest recovery
319+
excludes retired entries from mixed changes and rejects retirement-only changes.
320+
313321
The app downloads source artifacts, keeps the newest upload for each exact
314322
artifact name, and ingests them with changelog metadata from the merge run. The
315323
normal ingestion code skips failed benchmark rows. Benchmark rows and public

‎.github/workflows/merge-ingest.yml‎

Lines changed: 24 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -36,8 +36,29 @@ jobs:
3636

3737
- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
3838

39+
- name: Extract changelog metadata
40+
id: plan
41+
env:
42+
PUSH_BEFORE: ${{ github.event.before }}
43+
PUSH_AFTER: ${{ github.event.after }}
44+
run: |
45+
source "$GITHUB_WORKSPACE/inferencex-e2e/benchmarks/check_env.sh"
46+
check_env_vars GITHUB_WORKSPACE RUNNER_TEMP GITHUB_OUTPUT PUSH_BEFORE PUSH_AFTER
47+
cd "$GITHUB_WORKSPACE/inferencex-e2e"
48+
uv run --no-project --exclude-newer PT12H --python 3.12 --with pydantic --with pyyaml \
49+
python -m infx.matrix.plan \
50+
--changelog-file perf-changelog.yaml \
51+
--base-ref "$PUSH_BEFORE" \
52+
--head-ref "$PUSH_AFTER" > "$RUNNER_TEMP/plan.json"
53+
jq -c '.changelog_metadata' "$RUNNER_TEMP/plan.json" > "$GITHUB_WORKSPACE/changelog_metadata.json"
54+
agentic=$(jq '((.single_node.agentic // []) + (.multi_node.agentic // [])) | length > 0' "$RUNNER_TEMP/plan.json")
55+
retirement_only=$(jq '.changelog_metadata.entries | length > 0 and all(.[]; .retired == true)' "$RUNNER_TEMP/plan.json")
56+
echo "agentic=$agentic" >> "$GITHUB_OUTPUT"
57+
echo "retirement-only=$retirement_only" >> "$GITHUB_OUTPUT"
58+
3959
- name: Resolve reused sweep
4060
id: reuse
61+
if: steps.plan.outputs.retirement-only != 'true'
4162
env:
4263
GH_TOKEN: ${{ github.token }}
4364
run: |
@@ -49,12 +70,13 @@ jobs:
4970
--workflow-id "run-sweep.yml"
5071
5172
- name: Require sweep reuse
52-
if: steps.reuse.outputs.reuse-enabled != 'true'
73+
if: steps.plan.outputs.retirement-only != 'true' && steps.reuse.outputs.reuse-enabled != 'true'
5374
run: |
5475
echo "::error::Pushes to main never run sweeps. Post an authorized /use <run_id> reuse command on the PR before merging."
5576
exit 1
5677
5778
- name: Summarize reused source
79+
if: steps.plan.outputs.retirement-only != 'true'
5880
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
5981
env:
6082
SOURCE_RUN_URL: ${{ steps.reuse.outputs.reuse-source-run-url }}
@@ -68,22 +90,6 @@ jobs:
6890
['Reused benchmark source', process.env.SOURCE_SHA],
6991
]).write();
7092
71-
- name: Extract changelog metadata
72-
id: plan
73-
env:
74-
PUSH_BEFORE: ${{ github.event.before }}
75-
PUSH_AFTER: ${{ github.event.after }}
76-
run: |
77-
cd "$GITHUB_WORKSPACE/inferencex-e2e"
78-
uv run --no-project --exclude-newer PT12H --python 3.12 --with pydantic --with pyyaml \
79-
python -m infx.matrix.plan \
80-
--changelog-file perf-changelog.yaml \
81-
--base-ref "$PUSH_BEFORE" \
82-
--head-ref "$PUSH_AFTER" > "$RUNNER_TEMP/plan.json"
83-
jq -c '.changelog_metadata' "$RUNNER_TEMP/plan.json" > "$GITHUB_WORKSPACE/changelog_metadata.json"
84-
agentic=$(jq '((.single_node.agentic // []) + (.multi_node.agentic // [])) | length > 0' "$RUNNER_TEMP/plan.json")
85-
echo "agentic=$agentic" >> "$GITHUB_OUTPUT"
86-
8793
- name: Upload changelog artifact
8894
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
8995
with:
@@ -92,6 +98,7 @@ jobs:
9298
if-no-files-found: error
9399

94100
- name: Trigger database ingest
101+
if: steps.plan.outputs.retirement-only != 'true'
95102
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
96103
env:
97104
AGENTIC: ${{ steps.plan.outputs.agentic }}

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ Run the focused test path first. Use `--extra workflows` for GitHub integrations
8686
- Every priority-scheduled self-hosted job requests exactly one `nodes:N` label (physical Slurm nodes; single-node is `nodes:1`, multi-node forwards `node-count`). A queued job without the label is ineligible for priority scheduling, and labels cannot be added later; fix the branch and dispatch again.
8787
- A change that alters what any benchmark runs needs a `perf-changelog.yaml` entry for the affected config keys; the entry triggers result collection. Refactors, recipe moves, and docs need none. `python -m infx.matrix.changed --base origin/main` (from `inferencex-e2e/`) lists affected config and recipe keys; if it lists none, add no entry. It does not see script or launcher code, so judge script and launcher changes by what they run.
8888
- `perf-changelog.yaml` is append-only and byte-sensitive: preserve every existing byte and separator, append only at the tail, and resolve conflicts only with the [changelog conflict helper](.github/workflows/README.md#changelog-conflict-recovery). One PR has at most one block (listing one or more `config-keys`); revise it rather than adding another. New entries are English-only (the bilingual rules do not apply); leave historical entries unchanged.
89-
- **`main` never runs sweeps.** A PR with a changelog entry needs an accepted `/use <run_id>` before merge, or its results are never ingested. Merge with `infx.workflows.merge_with_reuse`, except external PRs (pin their `e2e-tests.yml` run explicitly) and ingest-recovery PRs (merge directly, per the [recovery command](.claude/commands/recover-failed-ingest.md)). See [`/use`](CONTRIBUTING.md#reusing-your-prs-green-sweep-at-merge-with-use).
89+
- **`main` never runs sweeps.** A PR with a changelog entry needs an accepted `/use <run_id>` before merge, or its results are never ingested. Merge with `infx.workflows.merge_with_reuse`, except external PRs (pin their `e2e-tests.yml` run explicitly), ingest-recovery PRs (merge directly, per the [recovery command](.claude/commands/recover-failed-ingest.md)), and retirement-only PRs (all new entries have `retired: true`; merge directly after review and checks, with no `/use` or database ingest). Empty or mixed changelogs are not retirement-only. See [`/use`](CONTRIBUTING.md#reusing-your-prs-green-sweep-at-merge-with-use).
9090
- Multi-node srt-slurm changes update the recipe and matching master config together. srt-slurm recipes, fixed-sequence and AgentX, are fragments: the binder supplies image, model, precision, and sequence lengths, and `inferencex-e2e/configs/srt-recipes/` supplies `benchmark.type` and `benchmark.command`; never set those in a recipe. Rows of one design share a `*-variants.yaml` ([recipe guide](inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/RECIPES.md)).
9191
- Speculative fixed-sequence benchmarks use the chat template: `bind_workload` in `infx/srt_slurm/workload.py` sets `benchmark.env.USE_CHAT_TEMPLATE: "true"` for single-node recipes that speculate, which `python3 -m infx.bench fixed-seq` (run by `srt_fixed_sequence.sh`) turns into `--use-chat-template`.
9292
- Benchmarks create no new directories under `/workspace`, and root containers leave no root-owned files in shared AMD runner workspaces.

‎CONTRIBUTING.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,10 @@ Every PR description must include an **AI model disclosure** section inside a co
2929

3030
**Performance changelog:** A PR that changes what any benchmark runs **MUST** append one new entry covering the affected config keys to the physical end of `inferencex-e2e/perf-changelog.yaml`; `python -m infx.matrix.changed --base origin/main` (run from `inferencex-e2e/`) lists them. A PR that changes no benchmark point adds none. Historical entries **MUST NOT** be edited.
3131

32+
For full config retirement, list the deleted config keys with `retired: true`. The planner verifies that every selected key existed in the base revision and is absent from the current master configs, retains the entry in changelog metadata, and schedules no benchmarks or evals for it. Retirement entries cannot use scenario filters, `append-only`, or eval-selection fields or modifiers; partial scenario retirement uses a regular entry for the surviving config.
33+
34+
A PR whose new entries are all retirements has no result artifacts to reuse. After review and required checks, merge it directly rather than through `merge_with_reuse`; no `/use` is required. Merge Ingest validates the retirement plan and uploads its changelog metadata, but skips reuse lookup and database dispatch. Mixed retirement and active entries still require normal sweep reuse. Do not re-append retirement entries for ingest recovery; they have no results to recover.
35+
3236
## Draft-model precision
3337

3438
The rule is simple: **serve the draft as it ships.** A speculative-decoding
@@ -163,6 +167,7 @@ A full benchmark sweep is expensive GPU time, and the runners are shared by ever
163167
- `/reuse-sweep-run <run_id>` remains supported with identical behavior. Bare `/reuse-sweep-run` selects automatically; bare `/use` is rejected.
164168
- The merge-to-`main` run then validates and ingests the PR sweep's artifacts; `main` never re-runs the sweep itself.
165169
- **Reuse is mandatory.** A green sweep alone is not enough. The reuse command must be on record (the sign-off verification checks for it), otherwise the `main` run fails and the PR's results are never ingested.
170+
- Retirement-only changes are the no-results exception described above: merge directly after review and required checks, without `/use`. Empty or mixed changelogs do not qualify.
166171
- Reuse does not require retaining a primary label, but it is rejected while the PR carries `evals-only` or `agentx-fast`, and `merge_with_reuse` refuses a PR with more than one primary label. The bot reacts to the command with 👍 when accepted or 👎 when rejected, with details in the Actions run summary, and posts a warning comment when the pinned run's commit is outside the PR; source artifacts are revalidated at merge.
167172
- A missing authorized reuse command produces a Check 4 **WARN**, not a rejection. The warning stays visible in the sign-off verdict; posting an authorized command is still required to reuse artifacts.
168173
- From the repository root, `uv run --project inferencex-e2e --extra workflows python -m infx.workflows.merge_with_reuse <pr-number>` is the supported merge path. It posts the command, syncs the branch with `main`, waits for checks, and squash-merges. See the [workflows README](.github/workflows/README.md#reusing-an-approved-pr-full-sweep) for eligibility details.

‎inferencex-e2e/benchmarks/multi_node/runtime_settings.sh‎

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -7,16 +7,10 @@ check_env_vars FRAMEWORK
77
case "$FRAMEWORK" in
88
sglang-disagg)
99
# Native workers inherit these engine/library settings through Slurm.
10-
check_env_vars IS_AGENTIC MODEL_PREFIX
1110
export ROCM_PATH=/opt/rocm UCX_HOME=/usr/local/ucx RIXL_HOME=/usr/local/rixl
1211
export MORI_IO_SQ_BACKOFF_TIMEOUT_US=50000 MORI_IO_QP_MAX_SEND_WR=16384
1312
export MORI_IO_QP_MAX_CQE=32768 MORI_IO_QP_MAX_SGE=2 MORI_IO_TC_DISABLE=0
1413
export UCX_IB_GID_INDEX=1 MORI_APP_LOG_LEVEL=WARNING SGLANG_ROUTER_STDOUT_LOGS=0
1514
export TORCH_NCCL_BLOCKING_WAIT=1 NCCL_BLOCKING_WAIT=1 SGLANG_OPT_USE_AITER_INDEXER=true
16-
if [[ "$IS_AGENTIC" == 1 || "$IS_AGENTIC" == true ]]; then
17-
if [[ "$MODEL_PREFIX" == dsv4 ]]; then
18-
export MORI_IO_SQ_BACKOFF_TIMEOUT_US=500000 MORI_IO_QP_MAX_SEND_WR=32768
19-
fi
20-
fi
2115
;;
2216
esac

0 commit comments

Comments
 (0)