Skip to content

feat(v0.33): Compact TopK selective KV dequant (SGLang #635) - #2

Open
dyldu wants to merge 1 commit into
feat/dsv41-v033-boundary-refactor-rebasefrom
pr/dsv41-compact-topk-dequant
Open

dyldu wants to merge 1 commit into
feat/dsv41-v033-boundary-refactor-rebasefrom
pr/dsv41-compact-topk-dequant

Conversation

@dyldu

@dyldu dyldu commented Nov 16, 2026

Copy link
Copy Markdown
Owner

Summary

Port Compact TopK-union selective KV dequant from SGLang #635 (KV slice only).

Includes:

  • Triton _pack_compact_sparse_token_ids / LightOp dsv4_pack_sparse_topk_workspace
  • LightOp dsv4_dequantize_sparse_topk_out (selective)
  • Companion opt-in full counted dequant (kv_full_dequant_lightop.py)

Does not port mHC/CP fusion or CP in-place reorder from the same SGLang PR.

Why

Provides the SparsePrefill flat-ID ABI and a measured full-counted dequant path. Selective TopK alone is not a microbench win at small shapes → both gates stay default OFF.

Accuracy

Suite Result
tests/runtime_patch/test_compact_topk_dequant_contract.py 4 passed
tests/accuracy/test_compact_topk_dequant_accuracy.py 5 passed

Performance (p50)

Selective vs full counted (same device class as fusion bench notes):

capacity topk Q selective full speedup
32 8 4 ~0.068 ms ~0.032 ms ~0.47×
256 32 8 ~0.069 ms ~0.032 ms ~0.47×
1024 64 16 ~0.069 ms ~0.032 ms ~0.46×

JSON: tests/accuracy/compact_topk_dequant_perf.json
Fair note from fusion rebase notes: vs vLLM-native, full counted is the ~3.5× path; selective alone is opt-in for ABI/parity, not speed.

How to use

export VLLM_HCU_USE_CUSTOM_OPS=1
# both default OFF:
export VLLM_HCU_USE_LIGHTOP_COMPACT_TOPK_DEQUANT=1   # selective TopK ABI
export VLLM_HCU_USE_LIGHTOP_KV_FULL_DEQUANT=1        # full counted companion

Contract:

  • full_ids.numel() <= Q * topk
  • V4 page stride ceil(P·584/576)·576
  • Install via bind_dsv41() → install_compact_topk_dequant() / install_kv_full_dequant()

Test plan

  • pytest tests/runtime_patch/test_compact_topk_dequant_contract.py tests/accuracy/test_compact_topk_dequant_accuracy.py -q
  • python tests/accuracy/bench_compact_topk_dequant_perf.py
  • Enable gates only when SparsePrefill flat-ID path is wired

Add opt-in pack/selective dequant plus companion full counted dequant,
contract/accuracy tests, and p50 microbench JSON (default OFF).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant