Skip to content
Open
Show file tree
Hide file tree
Changes from 136 commits
Commits
Show all changes
155 commits
Select commit Hold shift + click to select a range
cbdc00a
feat(qwen3-next): MoE route/combine contract with a batch-invariant g…
fusheng-ji Oct 9, 2026
fce346d
feat(qwen3-next): TP4 GDN, convolution and D=256 attention mixers
fusheng-ji Oct 9, 2026
362b80b
docs(qwen3-next): MoE route/combine contract, prior-art and TP4 evide…
fusheng-ji Oct 9, 2026
89b5212
Merge MoE route/combine evidence into the TP4 mixers
fusheng-ji Oct 9, 2026
bd13adc
test(qwen3-next): prior-art runner for the TP4 D=256 attention core
fusheng-ji Oct 9, 2026
c318687
docs(qwen3-next): TP4 mixer gates and D=256 attention prior-art evide…
fusheng-ji Oct 9, 2026
ce07836
Merge TP4 mixer evidence into the full-model chain
fusheng-ji Oct 9, 2026
92e93a4
feat(qwen3-next): full-model chain with first-drift localisation
fusheng-ji Oct 9, 2026
d6106c2
feat(qwen3-next): batch-invariance matrix for the full model
fusheng-ji Oct 9, 2026
02c576c
fix(qwen3-next): native state shapes for any prompt that fits the rol…
fusheng-ji Oct 9, 2026
193805d
Merge the evidence publisher fix into the batch-invariance matrix
fusheng-ji Oct 9, 2026
f431bad
fix(qwen3-next): evidence publisher redacts every recorded site path
fusheng-ji Oct 9, 2026
dc79953
test(qwen3-next): measure Megatron-core/TE and SGLang in the MoE prio…
fusheng-ji Oct 9, 2026
53dce6e
docs(qwen3-next): MoE prior art with Megatron-core/TE and SGLang meas…
fusheng-ji Oct 9, 2026
3c40295
Merge the MoE prior-art update into the TP4 mixers
fusheng-ji Oct 10, 2026
a97a6fe
test(qwen3-next): measure FlashAttention-4, Transformer Engine and Me…
fusheng-ji Oct 10, 2026
137d0fb
fix(qwen3-next): merge flat and per-size attention latency sections
fusheng-ji Oct 10, 2026
f01ab18
Merge test-qwennext refactor and migrate Qwen3-Next code and tests
fusheng-ji Oct 10, 2026
6b74b80
Merge test-qwennext refactor and migrate Qwen3-Next code and tests
fusheng-ji Oct 10, 2026
7317e5a
style(tools): mark the Qwen3-Next TP gate imports after sys.path setu…
fusheng-ji Oct 10, 2026
5cacf98
style(tools): mark the Qwen3-Next TP gate imports after sys.path setu…
fusheng-ji Oct 10, 2026
52b97e3
fix(qwen3-next): type annotations for mypy in the MoE prior-art runner
fusheng-ji Oct 11, 2026
d44704b
Merge the MoE prior-art type fix into the TP4 mixers
fusheng-ji Oct 11, 2026
4d7d3d8
fix(qwen3-next): type annotations for mypy in the attention prior-art…
fusheng-ji Oct 11, 2026
c331746
Merge test-qwennext refactor and migrate the Qwen3-Next full-model chain
fusheng-ji Oct 11, 2026
7bf6ef7
Merge test-qwennext refactor and migrate the Qwen3-Next batch-invaria…
fusheng-ji Oct 11, 2026
e485f0f
Merge the full-model chain evidence (c67ec14) into the migrated chain
fusheng-ji Oct 11, 2026
b66a8d0
build(pre-commit): leave the byte-exact Qwen3-Next chain evidence unt…
fusheng-ji Oct 11, 2026
13ba74a
Merge the full-model chain evidence into the batch-invariance matrix
fusheng-ji Oct 11, 2026
848ea44
build(pre-commit): leave the byte-exact batch-invariance evidence unt…
fusheng-ji Oct 11, 2026
84a7bfe
test(qwen3-next): pin the case-manifest subprocess rootdir to its tem…
fusheng-ji Oct 11, 2026
6684427
Merge the case-manifest test fix into the batch-invariance matrix
fusheng-ji Oct 11, 2026
cbb7b3d
docs(qwen3-next): figure for the full-model chain evidence
fusheng-ji Oct 11, 2026
e5a6987
Merge the chain evidence figure into the batch-invariance matrix
fusheng-ji Oct 11, 2026
9cc2fbc
docs(qwen3-next): figure for the batch-invariance evidence
fusheng-ji Oct 11, 2026
bddabba
test(qwen3-next): digest every TP4 cross-rank point and compare acros…
fusheng-ji Oct 11, 2026
99c244a
test(qwen3-next): require TP4 == TP1 for selected logprob and LSE only
fusheng-ji Oct 11, 2026
c5cea58
test(qwen3-next): prior art for the vocabulary-parallel selected-toke…
fusheng-ji Oct 11, 2026
da566bb
test(qwen3-next): no raw torch.distributed reduction in Qwen3-Next fo…
fusheng-ji Oct 11, 2026
f3a213d
docs(qwen3-next): TP distributed logprob contract and evidence figure
fusheng-ji Oct 11, 2026
682c233
test(qwen3-next): publisher for the distributed logprob parity evidence
fusheng-ji Oct 11, 2026
16171f5
docs(qwen3-next): record the prior-art NCCL-setting results and latency
fusheng-ji Oct 12, 2026
46ea845
feat(qwen3-next): EP4 routed experts over the TP4 group
fusheng-ji Oct 12, 2026
2314cc2
feat(qwen3-next): run the shared model at EP4 in vLLM rollout and VIM…
fusheng-ji Oct 12, 2026
1106e46
refactor(qwen3-next): focus TP review on validation and externalize e…
fusheng-ji Oct 12, 2026
a9d0ba8
fix(qwen3-next): hash validation artifacts on Python 3.10
fusheng-ji Oct 12, 2026
722ef1c
docs(qwen3-next): record post-review B200 validation results
fusheng-ji Oct 12, 2026
d8333f4
fix(validation): stream artifact SHA-256 on Python 3.10
fusheng-ji Oct 12, 2026
b5708e9
fix(validation): retain batch invariance with Python 3.10 hashing
fusheng-ji Oct 12, 2026
7756969
fix(validation): keep standalone evidence runtime free of torch imports
fusheng-ji Oct 12, 2026
6c7c49c
fix(validation): keep standalone evidence runtime free of torch imports
fusheng-ji Oct 12, 2026
3c3891f
fix(ci): resolve the repository root from ci/scripts/ in the Qwen3-Ne…
fusheng-ji Oct 13, 2026
a7d5c88
Merge the Qwen3-Next gate-script root fix
fusheng-ji Oct 13, 2026
a8d2f64
Merge the Qwen3-Next gate-script root fix
fusheng-ji Oct 13, 2026
e9bf995
Merge the Qwen3-Next gate-script root fix
fusheng-ji Oct 13, 2026
5a5aed4
fix(ci): resolve the repository root from ci/scripts/ in the Qwen3-Ne…
fusheng-ji Oct 13, 2026
25ded88
fix(ci): resolve the repository root from ci/scripts/ in the Qwen3-Ne…
fusheng-ji Oct 13, 2026
ffa0d1d
fix(ci): resolve the repository root from ci/scripts/ in the Qwen3-Ne…
fusheng-ji Oct 13, 2026
caf7212
Merge the reviewed TP logprob PR into the EP branch and adopt its layout
fusheng-ji Oct 13, 2026
343c8c9
Merge the batch-invariance fixes into the TP logprob PR
fusheng-ji Oct 13, 2026
4746836
test(qwen3-next): drop the duplicate file_sha256 import left by the m…
fusheng-ji Oct 13, 2026
c78a535
Merge the TP logprob launcher fix into the EP branch
fusheng-ji Oct 13, 2026
78b0db0
fix(qwen3-next): accept EP4 in the scheduler and L2 report checks
fusheng-ji Oct 13, 2026
3185114
fix(qwen3-next): name the measured EP size in the chain report
fusheng-ji Oct 14, 2026
6b1cb25
docs(qwen3-next): EP4 reduction contract and B200 evidence summary
fusheng-ji Oct 14, 2026
4daf3c4
Merge the latest #469 branch into the MoE route branch
fusheng-ji Oct 14, 2026
57216db
refactor(qwen3-next): MoE route branch to the reviewed layout
fusheng-ji Oct 14, 2026
f7338a7
Merge the MoE route layout change; TP4 mixers to the reviewed layout
fusheng-ji Oct 14, 2026
4ceaa22
Merge the TP4 mixers layout change into the full-model chain branch
fusheng-ji Oct 14, 2026
0d04347
refactor(qwen3-next): full-model chain to the reviewed layout
fusheng-ji Oct 14, 2026
0f5266d
Merge the full-model chain layout change into the batch-invariance br…
fusheng-ji Oct 14, 2026
299ea4c
refactor(qwen3-next): batch-invariance branch to the reviewed layout
fusheng-ji Oct 14, 2026
d18e0ff
Merge the batch-invariance layout change into the TP logprob branch
fusheng-ji Oct 14, 2026
72ce0a9
Merge the TP logprob layout follow-up into the EP branch
fusheng-ji Oct 14, 2026
f05aaef
docs(qwen3-next): EP4 evidence ZIP joins the shared Qwen3-Next release
fusheng-ji Oct 14, 2026
d75112c
test(qwen3-next): vLLM model contract covers EP4
fusheng-ji Oct 15, 2026
2cea4e9
test(qwen3-next): check the shared vLLM model with a real config and …
fusheng-ji Oct 15, 2026
78b3e1a
test(qwen3-next): initialise vLLM's parallel state inside the config …
fusheng-ji Oct 15, 2026
e5b15c7
test(qwen3-next): real-model check uses the certified cache dtypes an…
fusheng-ji Oct 15, 2026
dda0c2e
test(qwen3-next): real-model check uses the BF16 KV cache and the rig…
fusheng-ji Oct 15, 2026
1cb70f7
test(qwen3-next): build the real model under the loader's dtype and d…
fusheng-ji Oct 15, 2026
d8974d0
test(qwen3-next): real-model check builds the wrapper only
fusheng-ji Oct 15, 2026
3aa220d
test(qwen3-next): preflight runs the real vLLM config and four-rank m…
fusheng-ji Oct 15, 2026
d84fbf3
style(qwen3-next): black-format the relocated runtime path in the dig…
fusheng-ji Oct 15, 2026
e7d08a8
Merge the digest-test formatting fix
fusheng-ji Oct 15, 2026
c86ce74
Merge the digest-test formatting fix
fusheng-ji Oct 15, 2026
fa42043
Merge the digest-test formatting fix
fusheng-ji Oct 15, 2026
32040a7
docs(qwen3-next): record the full chain rerun on the stack top
fusheng-ji Oct 18, 2026
b59f628
Merge branch 'feat/ws1-qwen3-next-full-model-chain' into feat/ws1-qwe…
fusheng-ji Oct 18, 2026
7b71b93
docs(qwen3-next): record the full BI suite rerun on the stack top
fusheng-ji Oct 18, 2026
0165a26
Merge branch 'feat/ws1-qwen3-next-batch-invariance-ci' into feat/ws1-…
fusheng-ji Oct 18, 2026
91dceda
Merge branch 'feat/ws1-qwen3-next-distributed-logprob-tp' into feat/w…
fusheng-ji Oct 18, 2026
c4674df
docs(qwen3-next): record the EP4 full BI suite on the stack top
fusheng-ji Oct 18, 2026
4e00111
fix(qwen3-next): fail closed on a missing EP group and run the real-m…
fusheng-ji Oct 19, 2026
8943c14
docs: move the measurement outputs out of the pull request
fusheng-ji Oct 19, 2026
0fdbf80
docs: move the measurement outputs out of the pull request
fusheng-ji Oct 19, 2026
46d5108
docs: move the measurement outputs out of the pull request
fusheng-ji Oct 19, 2026
8fd657c
docs: move the measurement outputs out of the pull request
fusheng-ji Oct 19, 2026
3d4fb2e
docs: move the measurement outputs out of the pull request
fusheng-ji Oct 19, 2026
d3d9dda
docs: move the measurement outputs out of the pull request
fusheng-ji Oct 19, 2026
a705d28
fix(qwen3-next): allow Hopper H100/H200 in the four-GPU preflight and…
fusheng-ji Oct 19, 2026
bc43113
docs: point commit hashes at the rewritten history
fusheng-ji Oct 23, 2026
f9b85d1
docs: point commit hashes at the rewritten history
fusheng-ji Oct 23, 2026
4f61a39
docs: point commit hashes at the rewritten history
fusheng-ji Oct 23, 2026
804f0a4
docs: point commit hashes at the rewritten history
fusheng-ji Oct 23, 2026
78fc14a
docs: point commit hashes at the rewritten history
fusheng-ji Oct 23, 2026
16c3767
docs: point commit hashes at the rewritten history
fusheng-ji Oct 23, 2026
763e2f9
fix(qwen3-next): shorten qwen3_next_run docstring for flake8 E501
fusheng-ji Oct 25, 2026
68240ec
Merge #469 after #467 landed in test-qwennext
fusheng-ji Oct 26, 2026
b36b8f6
Merge #503 after #467 landed in test-qwennext
fusheng-ji Oct 26, 2026
d243839
Merge #507 after #467 landed in test-qwennext
fusheng-ji Oct 26, 2026
eb04b15
Merge #543 after #467 landed in test-qwennext
fusheng-ji Oct 26, 2026
fe6ed38
Merge #544 after #467 landed in test-qwennext
fusheng-ji Oct 26, 2026
8529700
Merge #549 after #467 landed in test-qwennext
fusheng-ji Oct 26, 2026
cfbd101
Merge #469 review fixes
fusheng-ji Oct 26, 2026
d53ac95
ci(qwen3-next): trigger the provider job on validation/common
fusheng-ji Oct 26, 2026
c9a0aed
Merge #503 review fixes
fusheng-ji Oct 26, 2026
b9390f3
ci(qwen3-next): trigger the provider job on the CUDA sources it builds
fusheng-ji Oct 26, 2026
47b0c78
Merge #507 review fixes
fusheng-ji Oct 26, 2026
73872d5
chore(qwen3-next): executable operator-gate script; say what is check…
fusheng-ji Oct 26, 2026
9220f1a
Merge #543 review fixes
fusheng-ji Oct 26, 2026
b3bd4ea
Merge #544 review fixes
fusheng-ji Oct 26, 2026
6d7d4bd
Merge #549 review fixes
fusheng-ji Oct 26, 2026
6dc9d41
ci(qwen3-next): trigger the provider job on the logprob kernels and t…
fusheng-ji Oct 26, 2026
9800e1a
Merge #469 second-round review fixes
fusheng-ji Oct 26, 2026
d849e18
Merge #503 second-round review fixes
fusheng-ji Oct 26, 2026
ecafcc2
Merge #507 second-round review fixes
fusheng-ji Oct 26, 2026
349d6ba
Merge #543 second-round review fixes
fusheng-ji Oct 26, 2026
e20c29b
Merge #544 second-round review fixes
fusheng-ji Oct 26, 2026
5ffd4c3
ci: run test_file_digest.py once in the CPU job
fusheng-ji Oct 26, 2026
65eaae4
Merge #549 second-round review fixes
fusheng-ji Oct 26, 2026
d78b22b
ci(qwen3-next): trigger the provider job on the code this PR's checks…
fusheng-ji Oct 27, 2026
dfb0832
Merge #543 provider path-filter fix
fusheng-ji Oct 27, 2026
db1eb37
ci(qwen3-next): trigger the provider job on the distributed collectives
fusheng-ji Oct 27, 2026
3f7cf6e
Merge #544 provider path-filter fixes
fusheng-ji Oct 27, 2026
88c4f02
Merge #549 provider path-filter fixes
fusheng-ji Oct 27, 2026
5294b76
Merge branch 'test-qwennext' into feat/ws1-qwen3-next-distributed-log…
Flink-ddd Nov 8, 2026
62cc122
fix(workload): sync operator case ids with pytest collection
fusheng-ji Nov 13, 2026
fd87c0e
feat(vime): load the Megatron model without DDP or an optimizer for r…
fusheng-ji Nov 13, 2026
f2e9aa9
Merge #543 operator-case ids and no-DDP recompute into #544
fusheng-ji Nov 13, 2026
7849c9c
Merge #544 operator-case ids and no-DDP recompute into #549
fusheng-ji Nov 13, 2026
34e475f
Merge #549 operator-case ids and no-DDP recompute into #557
fusheng-ji Nov 13, 2026
1c68102
fix(vime): pass skip_load_to_model_and_opt to the VIME checkpoint loader
fusheng-ji Nov 13, 2026
1472e8f
Merge #543 load_checkpoint fix into #544
fusheng-ji Nov 13, 2026
dee7a78
Merge #544 load_checkpoint fix into #549
fusheng-ji Nov 13, 2026
7261a37
Merge #549 load_checkpoint fix into #557
fusheng-ji Nov 13, 2026
2e85d22
ci(qwen3-next): assert the full pinned runtime, retrigger on pyprojec…
fusheng-ji Nov 13, 2026
74a1b1a
ci(qwen3-next): reject a malformed chain report with an explicit chec…
fusheng-ji Nov 13, 2026
7fee591
Merge #543 OCR round fixes into #544
fusheng-ji Nov 13, 2026
33e0ff6
Merge #544 OCR round fixes into #549
fusheng-ji Nov 13, 2026
0a054e5
Merge #549 OCR round fixes into #557
fusheng-ji Nov 13, 2026
4e2c33a
fix(ci): route unreadable chain reports to exit 2; fail closed on a m…
fusheng-ji Nov 14, 2026
81be081
Merge fix/543-ocr-r8 into fix/544-ocr-r8
fusheng-ji Nov 14, 2026
30c0e5a
Merge fix/544-ocr-r8 into fix/549-ocr-r8
fusheng-ji Nov 14, 2026
f935bf3
Merge fix/549-ocr-r8 into fix/557-ocr-r8
fusheng-ji Nov 14, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 30 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,9 +73,37 @@ jobs:

- name: Run Mocked Hardware Discovery Tests
run: |
python -m pytest tests/runtime/test_dispatch.py -v
python -m pytest tests/runtime/test_dispatch.py tests/validation/common/test_file_digest.py -v
PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/ops/attention/test_attention_correctness.py -q -rs
python -m pytest tests/validation/operators/test_forward_invariance.py tests/contracts/test_tolerance_contract.py tests/validation/operators/test_ws1_workload.py tests/validation/operators/test_gradient_invariance.py tests/validation/operators/test_elementwise_inventory.py tests/validation/operators/test_four_judgment_matrix.py tests/validation/operators/test_op_checks.py tests/validation/operators/test_operator_inputs.py tests/benchmarks/test_profiler.py tests/validation/operators/test_kv_consistency.py tests/models/qwen3/test_ws1_qwen3_dense.py tests/validation/models/test_ws1_chain_integration.py tests/models/qwen3_next/test_qwen3_next_norm.py tests/models/qwen3_next/test_gdn_state_contract.py -q
python -m pytest tests/validation/operators/test_forward_invariance.py tests/contracts/test_tolerance_contract.py tests/validation/operators/test_ws1_workload.py tests/validation/operators/test_gradient_invariance.py tests/validation/operators/test_elementwise_inventory.py tests/validation/operators/test_four_judgment_matrix.py tests/validation/operators/test_op_checks.py tests/validation/operators/test_operator_inputs.py tests/benchmarks/test_profiler.py tests/validation/operators/test_kv_consistency.py tests/models/qwen3/test_ws1_qwen3_dense.py tests/validation/models/test_ws1_chain_integration.py tests/models/qwen3_next/test_qwen3_next_norm.py tests/models/qwen3_next/test_gdn_state_contract.py tests/validation/common/test_tensor_identity.py tests/models/qwen3_next/test_qwen3_next_forward_contract.py tests/models/qwen3_next/test_qwen3_next_tp_blocks.py tests/models/qwen3_next/test_qwen3_next_tp_gdn.py -q
python -m pytest -q \
tests/models/qwen3_next/test_qwen3_next_binary_evidence.py \
tests/models/qwen3_next/test_qwen3_next_case_manifest.py \
tests/models/qwen3_next/test_qwen3_next_checkpoint.py \
tests/models/qwen3_next/test_qwen3_next_engine_weights.py \
tests/models/qwen3_next/test_qwen3_next_environment_identity.py \
tests/models/qwen3_next/test_qwen3_next_failure.py \
tests/models/qwen3_next/test_qwen3_next_imported_binary.py \
tests/models/qwen3_next/test_qwen3_next_installed_provider_identity.py \
tests/models/qwen3_next/test_qwen3_next_l2.py \
tests/models/qwen3_next/test_qwen3_next_layers.py \
tests/models/qwen3_next/test_qwen3_next_logprob_adapter.py \
tests/models/qwen3_next/test_qwen3_next_native_recompute.py \
tests/models/qwen3_next/test_qwen3_next_operator_gates.py \
tests/models/qwen3_next/test_qwen3_next_profile.py \
tests/models/qwen3_next/test_qwen3_next_rl_fixture.py \
tests/models/qwen3_next/test_qwen3_next_rotary_dtype.py \
tests/models/qwen3_next/test_qwen3_next_storage.py \
tests/models/qwen3_next/test_qwen3_next_training_audit.py \
tests/models/qwen3_next/test_qwen3_next_training_closeout.py \
tests/models/qwen3_next/test_qwen3_next_training_export.py \
tests/models/qwen3_next/test_qwen3_next_subnode.py \
tests/models/qwen3_next/test_qwen3_next_chain_report.py \
tests/models/qwen3_next/test_qwen3_next_run.py \
tests/models/qwen3_next/test_qwen3_next_evidence_redaction.py \
tests/models/qwen3_next/test_qwen3_next_bi_cases.py \
tests/models/qwen3_next/test_qwen3_next_tp_reductions.py \
tests/models/qwen3_next/test_qwen3_next_ep.py

- name: Run Cross-Configuration Contract Tests (CPU-safe)
run: |
Expand Down
43 changes: 40 additions & 3 deletions .github/workflows/qwen3-next-provider-gpu.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# SPDX-License-Identifier: Apache-2.0
# RFC #428 Qwen3-Next provider comparisons: the C1/C3/C4 norm providers and the C6
# GDN decode-step goldens, each checked against the real vLLM 0.30.0 provider.
# RFC #428 Qwen3-Next provider comparisons: the C1/C3/C4 norm providers, the C6
# GDN decode-step goldens, the shared GEMM/MoE route-combine provider and the TP4
# GDN/convolution/attention mixers and the vLLM model contract with the EP4 real model on four GPUs,
# each checked against the real vLLM 0.30.0 provider.
#
# These check_ files import vLLM, so the default pytest collection never runs them
# (tests/integrations/common/test_framework_operator_integrations.py asserts vLLM is not imported). This
Expand All @@ -23,10 +25,24 @@ on:
paths:
- "rl_engine/backends/**"
- "rl_engine/reference/**"
- "csrc/cuda/attention/**"
- "rl_engine/models/qwen3_next/**"
- "rl_engine/integrations/engines/train/vllm/qwen3_next*"
- "rl_engine/validation/models/qwen3_next*"
- "tools/validation/models/*qwen3_next*"
- "rl_engine/validation/models/qwen3_next/**"
- "tools/validation/models/qwen3_next/**"
- "tests/models/qwen3_next/**"
- "rl_engine/config/**"
- "rl_engine/validation/common/**"
- "csrc/cuda/norm/**"
- "csrc/bindings/**"
- "rl_engine/contracts/**"
- "csrc/cuda/logprob/**"
- "rl_engine/integrations/engines/rollout/vllm/qwen3_next*"
- "rl_engine/integrations/common/qwen3_next*"
- "rl_engine/integrations/orchestrators/vime/experiments/qwen3_next/**"
- "rl_engine/distributed/algorithms/collectives.py"
- ".github/workflows/qwen3-next-provider-gpu.yml"
workflow_dispatch:
inputs:
Expand Down Expand Up @@ -115,14 +131,35 @@ jobs:
print("rl_engine._C:", loaded)
PY
- name: Check pinned providers in their own process
# The FP32 router needs IEEE Triton dots from process start.
env:
TRITON_F32_DEFAULT: ieee
run: |
set -o pipefail
log="$RUNNER_TEMP/provider-pytest.log"
python3 -m pytest -q -rs tests/models/qwen3_next/check_qwen3_next_norm_providers.py tests/models/qwen3_next/check_gdn_recurrent_golden.py | tee "$log"
python3 -m pytest -q -rs tests/models/qwen3_next/check_qwen3_next_norm_providers.py tests/models/qwen3_next/check_gdn_recurrent_golden.py \
tests/models/qwen3_next/check_qwen3_next_forward.py tests/models/qwen3_next/check_qwen3_next_attention.py \
tests/models/qwen3_next/check_qwen3_next_gdn_bridge.py tests/models/qwen3_next/check_qwen3_next_gdn_sequence.py \
tests/models/qwen3_next/check_qwen3_next_conv_bridge.py tests/models/qwen3_next/check_qwen3_next_core_matrix.py \
tests/models/qwen3_next/check_qwen3_next_shared_core.py tests/models/qwen3_next/check_qwen3_next_paged_attention.py \
tests/models/qwen3_next/check_qwen3_next_batch_invariance.py tests/models/qwen3_next/check_qwen3_next_bi_requests.py | tee "$log"
# Fail closed: these modules skip as a whole when CUDA or vLLM is missing, and pytest
# exits 0 for an all-skipped run. A skipped provider check is not provider evidence.
if grep -Eq '^SKIPPED|[0-9]+ skipped' "$log"; then
echo "::error::a provider check was skipped"
exit 1
fi
grep -Eq '[0-9]+ passed' "$log"
Comment on lines +163 to +167

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test · high
grep -Eq '[0-9]+ passed' only proves that at least one test passed; it does not prove the intended provider modules actually ran. If a provider module errors during collection or fails to import (e.g. a missing symbol after a refactor), pytest reports that as an error, not a skip, and this check does not test for error/failed counts. So a run where most provider modules fail to collect but a single case in one module passes would exit 0 and be reported as green provider evidence.

Suggest asserting an explicit minimum passed count (or that no error/failed lines appear), so partial-collection regressions fail closed the way the skip check does.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changed. The step runs under bash -e with set -o pipefail, so pytest's exit status decides first: a module that fails to import or collect makes pytest exit 2 with no tests run, a wrong path exits 4 and an empty selection exits 5, and each fails the step before the greps. There is no -k, no addopts and no --continue-on-collection-errors, so a partial collection cannot pass. The greps add the two cases in which pytest exits 0: any skip (e8daba5, r4372859876) and nothing passed.

- name: Check the vLLM model contract and the EP4 real model (four GPUs)
# A real VllmConfig on four visible GPUs, the certified cache dtypes, the CUSTOM attention
# backend and the EP4 expert map. Fewer than four GPUs would skip these checks, so fail
# instead of reporting a pass that did not run them.
env:
TRITON_F32_DEFAULT: ieee
run: |
python3 - <<'PY'
import torch
assert torch.cuda.device_count() >= 4, "the real-model check needs four GPUs"
PY
python3 -m pytest -q -rs tests/models/qwen3_next/check_qwen3_next_vllm_model_contract.py \
tests/models/qwen3_next/check_qwen3_next_vllm_real_model.py
27 changes: 27 additions & 0 deletions ci/scripts/run_qwen3_next_chain_gate.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#!/usr/bin/env bash
# SPDX-License-Identifier: Apache-2.0
# Qwen3-Next chain-report adapter checks; with a report path, also validate that report.
# bash ci/scripts/run_qwen3_next_chain_gate.sh [chain-report.json]
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
cd "$ROOT"
PY="${PY:-python3}"
export PYTHONPATH="$ROOT${PYTHONPATH:+:$PYTHONPATH}"
"$PY" -m pytest -q tests/models/qwen3_next/test_qwen3_next_chain_report.py
if [ "$#" -ge 1 ]; then
"$PY" - "$1" <<'PY'
import dataclasses, json, sys

from rl_engine.validation.models.chain_gate import ChainGateReport
from rl_engine.validation.models.qwen3_next.qwen3_next_chain_report import COMPATIBLE_SCHEMA, SCHEMA_VERSION

report = json.load(open(sys.argv[1]))
missing = {f.name for f in dataclasses.fields(ChainGateReport)} - set(report)
assert not missing, f"chain report lacks ChainGateReport fields: {sorted(missing)}"
assert report["schema_version"] == SCHEMA_VERSION, report["schema_version"]
assert report["compatible_schema"] == COMPATIBLE_SCHEMA, report["compatible_schema"]
print(f"passed={report['passed']} first_drift={report['first_drift']} "
f"family={report['first_drift_family']} cells={len(report['cells'])}")
sys.exit(0 if report["passed"] else 1)
PY
fi
10 changes: 10 additions & 0 deletions ci/scripts/run_qwen3_next_operator_gates.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/bin/env bash
# Required operator acceptance and diagnostics use separate declared case sets.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
cd "$ROOT"
PY="${PY:-python3}"
OUT="${QWEN3_NEXT_EVIDENCE_DIR:?Set QWEN3_NEXT_EVIDENCE_DIR to an artifact directory}"
export PYTHONPATH="$ROOT${PYTHONPATH:+:$PYTHONPATH}"
exec "$PY" tools/validation/models/qwen3_next/qwen3_next_operator_gates.py --out "$OUT" \
--suite "${QWEN3_NEXT_OPERATOR_SUITE:-all}"
2 changes: 1 addition & 1 deletion ci/scripts/run_ws1_gtest.sh
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ print("[ws1-gtest] C8 gate passed")
PY

echo "[ws1-gtest] Qwen3-Next C3/C4 norms (operator scope)"
QWEN3_NEXT_NORM_MANIFEST=rl_engine/validation/models/qwen3_next_norm_manifest.json
QWEN3_NEXT_NORM_MANIFEST=rl_engine/validation/models/qwen3_next/qwen3_next_norm_manifest.json
# --hidden is the model hidden size the manifest pins: validate_norm_dimensions requires
# --hidden 2048 --head-dim 128 for both ops. rms_norm_gated does not run at 2048: its inputs and
# weight are built at --head-dim (128, the GDN value-head width), so the same flags give each
Expand Down
25 changes: 19 additions & 6 deletions csrc/cuda/attention/deterministic_attention.cu
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@

namespace {

constexpr int64_t kDeterministicAttentionHeadDim = 128;
constexpr int kSoftmaxThreads = 256;

// ---------------------------------------------------------------------------
Expand Down Expand Up @@ -220,15 +219,18 @@ void check_deterministic_attention_inputs(
const int64_t Hkv = k.size(1);
const int64_t Skv = k.size(2);

TORCH_CHECK(D == kDeterministicAttentionHeadDim,
"deterministic_attention: head dim D must be ",
kDeterministicAttentionHeadDim,
", got ",
D);
#if defined(USE_ROCM)
TORCH_CHECK(D == 128, "deterministic_attention: ROCm head dim D must be 128, got ", D);
#else
TORCH_CHECK(D == 128 || D == 256,
"deterministic_attention: CUDA head dim D must be 128 or 256, got ", D);
#endif
TORCH_CHECK(k.size(0) == B && v.size(0) == B,
"deterministic_attention: batch size mismatch between q/k/v");
TORCH_CHECK(v.size(1) == Hkv && v.size(2) == Skv && k.size(3) == D && v.size(3) == D,
"deterministic_attention: k/v shape mismatch");
TORCH_CHECK(B > 0 && Hq > 0 && Hkv > 0,
"deterministic_attention: batch size and head counts must be positive");
TORCH_CHECK(Hq % Hkv == 0,
"deterministic_attention: Hq (",
Hq,
Expand Down Expand Up @@ -573,6 +575,17 @@ std::vector<torch::Tensor> deterministic_attention_backward(
double scale,
torch::optional<torch::Tensor> key_padding_mask) {

check_deterministic_attention_inputs(q, k, v, key_padding_mask);
TORCH_CHECK(grad_output.device() == q.device() && grad_output.scalar_type() == q.scalar_type(),
"deterministic_attention_backward: grad_output must match q device and dtype");
TORCH_CHECK(grad_output.sizes() == q.sizes(),
"deterministic_attention_backward: grad_output must have q shape");
TORCH_CHECK(P.device() == q.device() && P.scalar_type() == at::kFloat,
"deterministic_attention_backward: P must be FP32 on the input device");
TORCH_CHECK(P.dim() == 4 && P.size(0) == q.size(0) && P.size(1) == q.size(1) &&
P.size(2) == q.size(2) && P.size(3) == k.size(2),
"deterministic_attention_backward: P must have shape [B, Hq, Sq, Skv]");

const at::cuda::OptionalCUDAGuard device_guard(at::device_of(q));

auto dO = grad_output.contiguous();
Expand Down
2 changes: 2 additions & 0 deletions docs/operators/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,6 @@ Every operator page should include:
- [Token Embedding](embedding.md)
- [Qwen3-Next RMSNorm (zero-centred)](qwen3-next-rms-norm.md)
- [Qwen3-Next Gated RMSNorm](qwen3-next-rms-norm-gated.md)
- [Qwen3-Next MoE route / combine](qwen3-next-moe-route-combine.md)
- [Qwen3-Next distributed logprob parity (TP and EP)](qwen3-next-distributed-logprob.md)
- [Operator Doc Template](../contributing/operator-doc-template.md)
2 changes: 1 addition & 1 deletion docs/operators/attention.md
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,7 @@ for measured peak memory at representative shapes.

## Known Limitations

- First version: `D=128` only (Qwen3-8B alignment).
- Head dim: `D=128` (Qwen3-8B) on CUDA and ROCm; `D=256` (Qwen3-Next full attention) on CUDA only.
- Supported dtypes: BF16, FP16.
- Full materialization of scores/P limits practical sequence length.
- `Hq` must be divisible by `Hkv` (raises `ValueError` otherwise).
Expand Down
Loading
Loading