Skip to content

[BI][DSv4] Pin sparse-MLA decode and prefill scheduling under batch invariance #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aoshen02 wants to merge 16 commits into
base: bi/indexer-topk
Choose a base branch
from
Open

[BI][DSv4] Pin sparse-MLA decode and prefill scheduling under batch invariance #21

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
149f29a
[BI][DSv4] Deterministic sparse-indexer top-k with fused Triton kernel
aoshen02 Aug 15, 2026
cd723f6
[BI][DSv4] Key out-of-window lanes below every in-window one
aoshen02 Aug 16, 2026
5ce5f47
[BI][DSv4] Keep the reference equal to the kernel on every bit pattern
aoshen02 Aug 16, 2026
cd2ea8e
[BI][DSv4] Rank the reference on validity, not on a reserved sentinel
aoshen02 Aug 16, 2026
7a0e32d
[BI][DSv4] Cover every fp32 class in the kernel/reference equivalence…
aoshen02 Aug 16, 2026
40ede92
[BI][DSv4] Cover index_topk=2048 in the bit-pattern equivalence tests
aoshen02 Aug 16, 2026
407d9e2
[BI][DSv4] Guard the constexpr instantiation for Triton-less imports
aoshen02 Aug 16, 2026
43aa95a
[BI][DSv4] Run test_indexer_topk_batch_invariant.py in CI
aoshen02 Aug 16, 2026
a772839
[BI][DSv4] Pin sparse-MLA decode and prefill scheduling under batch i…
aoshen02 Aug 15, 2026
5bdabbd
[BI][DSv4] Plan whole-request groups instead of falling back above th…
aoshen02 Aug 16, 2026
2588e12
[BI][DSv4] Never let the cold-start plan's output be the one observed
aoshen02 Aug 16, 2026
f39b0f3
[BI][DSv4] Run the sparse-MLA regressions when the model code changes
aoshen02 Aug 16, 2026
ddc1432
[BI][DSv4] Fail closed on an unpinnable plan, and cover the branch ab…
aoshen02 Aug 16, 2026
81cbc20
[BI][DSv4] Test the cold-start control flow, and stop claiming SM90 c…
aoshen02 Aug 16, 2026
7be588f
[BI][DSv4] Make the cold-start stub model its participants, and colle…
aoshen02 Aug 16, 2026
755c4ee
[BI][DSv4] Compare the grouped plan against the shipped planner, not …
aoshen02 Aug 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .buildkite/test_areas/misc.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -406,6 +406,10 @@ steps:
- pytest -v -s v1/determinism/test_rms_norm_batch_invariant.py
- VLLM_TEST_MODEL=deepseek-ai/DeepSeek-V2-Lite-Chat pytest -v -s v1/determinism/test_batch_invariance.py::test_v1_generation_is_deterministic_across_batch_sizes_with_needle -k TRITON_MLA
- VLLM_TEST_MODEL=Qwen/Qwen3-30B-A3B-Thinking-2507-FP8 pytest -v -s v1/determinism/test_batch_invariance.py::test_v1_generation_is_deterministic_across_batch_sizes_with_needle -k FLASH_ATTN
# The sparse-MLA backend accepts compute capability 9 as well as 10. The
# kernel tests skip here (only validated on SM100); the structural and
# control-flow ones do not, and this is the only lane that collects them.
- pytest -v -s v1/determinism/test_sparse_mla_batch_invariance.py

- label: Batch Invariance (B200)
key: batch-invariance-b200
Expand All @@ -414,6 +418,7 @@ steps:
source_file_dependencies:
- vllm/v1/attention
- vllm/model_executor/layers
- vllm/models/deepseek_v4/
- tests/v1/determinism/
commands:
- export VLLM_WORKER_MULTIPROC_METHOD=spawn
Expand All @@ -424,6 +429,8 @@ steps:
- VLLM_TEST_MODEL=Qwen/Qwen3-30B-A3B-Thinking-2507-FP8 pytest -v -s v1/determinism/test_batch_invariance.py::test_v1_generation_is_deterministic_across_batch_sizes_with_needle -k FLASH_ATTN
- pytest -v -s v1/determinism/test_nvfp4_batch_invariant.py
- pytest -v -s v1/determinism/test_matmul_batch_invariant.py
- pytest -v -s v1/determinism/test_indexer_topk_batch_invariant.py
- pytest -v -s v1/determinism/test_sparse_mla_batch_invariance.py
- pytest -v -s v1/determinism/test_cutlass_batch_invariance.py
- pytest -v -s v1/determinism/test_online_batch_invariance.py

Expand Down
Loading