Skip to content

[CI] suites: Unify CI suite and command names, skip instead of fail without run-ci, and share one finish-job check - #50874

Merged
hnyls2002 merged 15 commits into
mainfrom
lsyin/ci-simplify-hw-workflows
Nov 17, 2026
Merged

hnyls2002 merged 15 commits into
mainfrom
lsyin/ci-simplify-hw-workflows

Conversation

@hnyls2002

@hnyls2002 hnyls2002 commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Combines #50876, #50894, and #50933, which closed when this stack was reordered, with the finish-job cleanup that was this PR. Follows #50872.

Remove duplicate commands

  • Remove the /run-<tier>-ci commands (/run-nv-extra-ci, /run-xeon-extra-ci, /run-nv-ci, and their /rerun- forms). They duplicate /run-ci <hardware>, /run-extra-ci <hardware>, and /run-full-ci <hardware>, which now take any number of hardware arguments.
  • Remove the aliases /run-base-ci, /rerun-base-ci, /rerun-extra-ci, /rerun-full-ci, and /rerun-cpu-ci; /run-ci, /run-extra-ci, /run-full-ci, and /run-cpu-ci do the same. Removed spellings get the usual "did you mean" reply for unknown commands. Typo suggestions now include /run-ci and /run-extra-ci.
  • Update the contribution guide, .github/FOLDER_README.md, the CI workflow skill, and the opt-in tier lint's messages.

Unify CI suite names across hardware

  • CI suites follow one naming scheme: <tier>[-<stage>]-[<kind>-]test-<runner>[-<hardware>][-<variant>]. Per commit, the tier is base or extra: NVIDIA and AMD have both, every other hardware only extra. A stage letter is kept only where jobs run in stage order (NVIDIA, AMD, NPU). The rule is written down in the write-sglang-test skill.
  • Xeon: stage-a-test-cpu-intel and stage-b-test-cpu-intel were always unioned into one pool, so they merge into extra-test-cpu-intel; stage-a-tp-test-cpu-intel becomes extra-tp-test-cpu-intel.
  • XPU: stage-a-test-1-gpu-xpu and stage-b-test-1-gpu-xpu (also run as one pool) merge into extra-test-1-gpu-xpu.
  • AMD base: stage-{a,b,c}-*-amd becomes base-{a,b,c}-*-amd, in both suite= and stage= registrations; the kernel suites gain base-a-, the disaggregation suite becomes base-b-disaggregation-test-large-8-gpu-amd-mi35x, and the stage waits become wait-for-base-{a,b}-amd.
  • NPU: base-* becomes extra-* (extra-c-test-acc-* as extra-c-acc-test-*), including the coverage job and artifact names and the should_run_* / only_tests_* outputs.
  • ARM64, MLX, MPS, MLU, and PPU suites get extra-* names.
  • Nightly and NPU full suites follow the same scheme, e.g. nightly-amd-accuracy-8-gpu-mi35x-kimi-k3 -> nightly-accuracy-test-8-gpu-amd-mi35x-kimi-k3, nightly-acc-2-npu-a3 -> nightly-acc-test-2-npu-a3, nightly-xpu-kernel-main-1-gpu -> nightly-kernel-main-test-1-gpu-xpu, nightly-musa-1-gpu -> nightly-test-1-gpu-musa. AMD nightly job ids, which do not match suite names, are unchanged.
  • Workflow jobs, artifacts, and outputs named after a suite are renamed with it; scripts, monitors, and docs that name these suites are updated.
  • Remove three NPU tests registered only in per-commit-*-npu-a2/a3 suites, which no workflow runs, and a base-b-test-17-npu-a3 registration with no job (the test keeps its nightly registration).

Name NVIDIA diffusion suites by kind and pick the install by stage

  • NVIDIA diffusion suites follow the suite naming rule, with the kind before test: base-b-test-diffusion-1-gpu-h100 becomes base-b-diffusion-test-1-gpu-large, and the bcg, component, and unit lanes become base-b-diffusion-{bcg,component,unit}-test-<runner>.
  • Diffusion lanes use the generic runner keys (1-gpu-small, 1-gpu-large, 2-gpu-large, plus new 1-gpu-b200 and 1-gpu-b300); the eight diffusion-* runner configs are removed.
  • The install script is chosen by the stage's kind: scripts/ci/runner_configs.yml maps the kind's first word (diffusion) to the diffusion install through kind_installs, and scripts/ci/runner_configs.py resolves (runner_config, stage). _pr-test-stage.yml, _diffusion-pr-test-stage.yml, and /rerun-test use it.
  • _diffusion-pr-test-stage.yml takes the stage and names its job, artifacts, and suite <stage>-test-<runner_config>, since several lanes now share a runner key.
  • Update the registrations, test/run_suite.py, pr-test-multimodal-gen.yml, the PR test health check fixtures, and the CI skills.

Share one jq needs check across finish jobs

  • Finish jobs of pr-test.yml, pr-test-extra.yml, pr-test-amd.yml, pr-test-amd-extra.yml, pr-test-npu.yml, pr-test-musa.yml, pr-test-mlx.yml, pr-test-xpu.yml, and nightly-test-musa.yml share one check: toJson(needs) passes through env instead of being pasted into a shell string, jq prints each job's result and fails on failure or cancelled.
  • pr-test-musa.yml: fix workflow_dispatch input: to inputs:, so its target_stage input is accepted.
  • Remove pr-gate.yml's require-run-ci input, which every caller left at its default.
  • .github/FOLDER_README.md: link the contribution guide for command syntax instead of repeating it; keep the permission and tier-registration rules.

Skip instead of fail on a draft PR or without run-ci

  • pr-gate.yml: a draft PR, a missing run-ci, or a missing opt-in label runs one Skip tests step, and the gate passes with opted-in=false instead of failing. Every caller's test jobs, and its finish job, skip on it, so a skipped run shows no passing check; pr-test.yml, pr-test-amd.yml, pr-test-sgl-router.yml, and pr-test-sgl-processor.yml now check it too.
  • pr-test-finish, a required check, runs and fails when the tests were skipped, so it is the only red check on such a PR (down from about 19) and the PR still cannot merge without Base CI.
  • Slash commands: /rerun-failed-ci (and /tag-and-rerun-ci, which calls it) fully reruns a run whose tests the gate skipped once the PR has run-ci, is not a draft, and has the workflow's opt-in labels; "Re-run failed jobs" would rerun only pr-test-finish. /run-ci <hardware> reruns such runs in full as well.
  • pr-states.yml tells skipped runs from tested ones for the Base, Extra, and AMD rows, and refreshes when a PR leaves or enters draft.
  • scripts/lint/check_opt_in_ci_tiers.py fails when a job that needs pr-gate.yml does not skip on opted-in; only pr-test-finish is exempt.

Fail the lint for tests no workflow runs

  • scripts/lint/check_registered_tests.py: a non-CUDA registration not marked disabled fails when no workflow, action, or scripts/ci shell script names its suite and none runs its file by path. CUDA suites stay covered by the existing stage/runner_config checks. The pre-commit hook now also runs on workflow and scripts/ci changes.
  • Delete 16 tests that no workflow runs: 12 AMD nightly accuracy/perf tests and 4 NPU tests (full-test-4-npu-a2, nightly-test-17-npu-a3, nightly-test-npu-a3-merged). Tests that nightly-test-amd.yml runs by path are kept.
  • test_gdn_prefill_layout.py, added on main with stage="stage-b", moves to base-b.

Behavior

  • /run-<tier>-ci and the command aliases above no longer run CI; use /run-ci, /run-extra-ci, /run-full-ci, or /run-cpu-ci.
  • On a draft PR or one without run-ci, every workflow skips its tests and finish job, and only pr-test-finish fails; previously every workflow's gate failed.
  • Until a PR pushes after this merges, its runs come from the old gate, which has no Skip tests step, so /tag-and-rerun-ci <hardware> does not rerun their green opt-in runs; /run-extra-ci <hardware> or a push does.
  • Every suite keeps the same tests, except the never-run registrations and tests removed above, and every registration resolves to the same install, runner, artifact version, and timeout as before.
  • pr-test-musa.yml's dispatch input was declared under input:, so target_stage was never defined; it is inputs: now, and a manual MUSA dispatch honors it, and the XPU finish job now also fails when its gate job fails, as the other finish jobs do.
  • Job, artifact, and suite names change. Tools outside this repo that key on them need the new names; in particular, pr-test-mlu.yml passes the suite name as SGLANG_MLU_CI_TRIGGER_TYPE to the external MLU CI, which now receives extra-test-{1,2}-mlu. CI monitors that group runs by job name (XPU, NPU, AMD) start a new history for each renamed job. Until runs from before the rename leave its lookback window, about a week, the CI failure monitor (scripts/ci_monitor/ci_failures_analysis.py) also lists old-name jobs with the streak they had at the rename, so a job that was failing then still shows as failing.

CI States

Latest PR Test (Base): ❌ Run #44064272576
Latest PR Test (Extra): ⚠️ Not enabled -- add run-ci-nv-extra label to opt in.
Latest PR Test (AMD ROCm 10): ✅ Run #44064271858

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@github-actions github-actions Bot added documentation Improvements or additions to documentation amd npu mthreads apple-silicon labels Nov 17, 2026
@github-actions

This comment has been minimized.

@mintlify

mintlify Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
sglang-doc 🟢 Ready View Preview Nov 17, 2026, 10:03 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

@hnyls2002
hnyls2002 force-pushed the lsyin/ci-simplify-hw-workflows branch from 09c5569 to 90583b9 Compare November 17, 2026 08:23
@github-actions

This comment has been minimized.

@hnyls2002 hnyls2002 added the run-ci CI: run the baseline test suite on this PR label Nov 17, 2026
@hnyls2002
hnyls2002 merged commit 31e55a5 into main Nov 17, 2026
197 of 210 checks passed
@hnyls2002
hnyls2002 deleted the lsyin/ci-simplify-hw-workflows branch November 17, 2026 18:48
michaelzhang-ai added a commit to michaelzhang-ai/sglang that referenced this pull request Nov 17, 2026
Resolve the test/run_suite.py conflict from the suite renames in sgl-project#50874,
and rename the MMMU suite to the new scheme:
nightly-accuracy-test-8-gpu-amd-mi35x-muse-spark-mmmu.
rbrugaro-amd added a commit to rbrugaro-amd/sglang that referenced this pull request Nov 17, 2026
…MI35x suite

sgl-project#50874 renamed the CI suites from stage-* to base-*, so
stage-c-test-large-8-gpu-amd-mi35x no longer exists and no workflow ran
this test; the check-registered-tests hook now rejects it. Register it in
base-c-test-large-8-gpu-amd-mi35x, which pr-test-amd.yml runs.

Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com>
rbrugaro-amd added a commit to rbrugaro-amd/sglang that referenced this pull request Nov 17, 2026
sgl-project#50874 renamed the CI suites, and stage-b-test-1-gpu-small-amd-mi35x is
now base-b-test-1-gpu-small-amd-mi35x. It migrated every test already on
main, but this branch's test still named the old suite, so the
check-registered-tests lint hook rejected it as registered to a suite no
workflow runs.

Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com>
eppaneamd added a commit to eppaneamd/sglang that referenced this pull request Nov 17, 2026
main renamed the CI stages (sgl-project#50874); a test registered under stage-b now
maps to a suite no workflow runs.

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Rony Leppänen <rony.leppanen@amd.com>
airMeng added a commit to airMeng/sglang that referenced this pull request Nov 18, 2026
Mechanical compatibility with main CI suite renaming in sgl-project#50874.
Retain the original one-GPU extra and four-GPU nightly assignments.
Sources: sgl-project#44905, sgl-project#48374, sgl-project#43891.
kevin-mii added a commit to kevin-mii/sglang that referenced this pull request Nov 18, 2026
main renamed the CI suites (sgl-project#50874). test_verify_shared_kv.py conflicted on its
registration line: take main's base-b suite name and keep this PR's est_time.
The other registrations this PR touches move to the new names too, since a
stale suite name is never dispatched.
kevin-mii added a commit to kevin-mii/sglang that referenced this pull request Nov 18, 2026
…ites

main renamed the CI suites (sgl-project#50874); a test registered under an old name is
never dispatched.
kevin-mii added a commit to kevin-mii/sglang that referenced this pull request Nov 18, 2026
Conflicts:
- dflash_draft_ring.py: main (sgl-project#50740) added the MHA ring path ahead of the MLA
  backend check; keep it and keep this PR's dflash_draft_reads_rows_by_position
  in place of the DFLASH_MLA_ONLY_DRAFT_BACKENDS membership test.
- test_dflash_draft_kv_ring.py: keep both main's current_platform import and
  this PR's override_platform import.
- Registration lines in test_verify_shared_kv.py and
  test_fused_kda_conv_recurrent_verify.py: take main's renamed suites (sgl-project#50874)
  and keep this PR's est_time. The other AMD registrations this PR touches
  move to the new suite names too, since a stale name is never dispatched.
kevin-mii added a commit to kevin-mii/sglang that referenced this pull request Nov 18, 2026
…ites

main renamed the CI suites (sgl-project#50874); a test registered under an old name is
never dispatched.
jiaryang added a commit to jiaryang/sglang that referenced this pull request Nov 18, 2026
sgl-project#50874 renamed the AMD suite jit-kernel-unit-test-amd to
base-a-jit-kernel-unit-test-amd; no workflow runs the old name.

Co-Authored-By: Claude <noreply@anthropic.com>
Shunkangz added a commit to Shunkangz/sglang that referenced this pull request Nov 18, 2026
… suite

sgl-project#50874 renamed the Xeon suites stage-a/b-test-cpu-intel to
extra-test-cpu-intel. sgl-project#41329 merged afterwards still registering
test_quant_ops.py on stage-a-test-cpu-intel, which no workflow runs, so
check-registered-tests fails on main and on every PR based on it. Move it
to extra-test-cpu-intel, the suite pr-test-xeon.yml runs and every other
Intel CPU kernel test under test/registered/cpu uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bvolpato added a commit to bvolpato/sglang that referenced this pull request Nov 18, 2026
The previous workflows stopped at the missing run-ci label. A fresh push picks up the upstream gate change in sgl-project#50874 so optional hardware checks can skip while base CI waits for approval.
bvolpato added a commit to bvolpato/sglang that referenced this pull request Nov 18, 2026
Use the upstream CI gate from sgl-project#50874 and register the DSpark regression in the renamed AMD JIT kernel suite.
At1a8 added a commit to At1a8/sglang that referenced this pull request Nov 18, 2026
…y suite

sgl-project#50874 renamed nightly-amd-4-gpu-mi35x-deepseek-v41-flash to nightly-test-4-gpu-amd-mi35x-deepseek-v41-flash, and test/run_suite.py lists only the new name.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

amd apple-silicon blackwell SM100/SM120 deepseek documentation Improvements or additions to documentation hicache Hierarchical Caching for SGLang lora mthreads Multi-modal multi-modal language model npu quant LLM Quantization run-ci CI: run the baseline test suite on this PR speculative-decoding unified-radix-cache

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant