Repository navigation
Fix mega_mhc split barriers under CUDA graph replay - #478
Merged
zheanxu merged 1 commit intoOct 20, 2026
Merged
Conversation
A CUDA graph kernel node keeps the same %gridid on every replay. The mega_mhc split barriers tag each execution with %gridid, so after a replay a barrier line still holds encode_state(grid_idx, kNumSplits), which is exactly the state the next replay waits for. A Mix or Norm waiter whose m-block init CTA has not started yet proceeds immediately and reads the previous replay's split partials. Tag executions with an epoch instead. The epoch and an exit count live in the unused tail of the first barrier line, so the host-side buffer is unchanged. Every CTA reads the epoch after cudaGridDependencySynchronize, and the last CTA to retire resets the exit count and advances the epoch. Signed-off-by: lightseek-bot <243258330+lightseek-bot@users.noreply.github.com>
Contributor
Author
|
@LyricZhao PTAL Thanks! |
zyongye
added a commit
to vllm-project/DeepGEMM
that referenced
this pull request
Oct 29, 2026
…#48) A CUDA graph kernel node keeps the same %gridid on every replay. The mega_mhc split barriers tag each execution with %gridid, so after a replay a barrier line still holds encode_state(grid_idx, kNumSplits), which is exactly the state the next replay waits for. A Mix or Norm waiter whose m-block init CTA has not started yet proceeds immediately and reads the previous replay's split partials. Tag executions with an epoch instead. The epoch and an exit count live in the unused tail of the first barrier line, so the host-side buffer is unchanged. Every CTA reads the epoch after cudaGridDependencySynchronize, and the last CTA to retire resets the exit count and advances the epoch. (cherry picked from commit 5081f85) Co-authored-by: LightSeek Foundation <243258330+lightseek-bot@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
zcnrex
added a commit
to sgl-project/DeepGEMM
that referenced
this pull request
Nov 2, 2026
…#131) A CUDA graph kernel node keeps the same %gridid on every replay. The mega_mhc split barriers tag each execution with %gridid, so after a replay a barrier line still holds encode_state(grid_idx, kNumSplits), which is exactly the state the next replay waits for. A Mix or Norm waiter whose m-block init CTA has not started yet proceeds immediately and reads the previous replay's split partials. Tag executions with an epoch instead. The epoch and an exit count live in the unused tail of the first barrier line, so the host-side buffer is unchanged. Every CTA reads the epoch after cudaGridDependencySynchronize, and the last CTA to retire resets the exit count and advances the epoch. Signed-off-by: lightseek-bot <243258330+lightseek-bot@users.noreply.github.com> Co-authored-by: LightSeek Foundation <243258330+lightseek-bot@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
zcnrex
added a commit
to sgl-project/DeepGEMM
that referenced
this pull request
Nov 4, 2026
* Fix mega_mhc split barriers under CUDA graph replay (deepseek-ai#478) A CUDA graph kernel node keeps the same %gridid on every replay. The mega_mhc split barriers tag each execution with %gridid, so after a replay a barrier line still holds encode_state(grid_idx, kNumSplits), which is exactly the state the next replay waits for. A Mix or Norm waiter whose m-block init CTA has not started yet proceeds immediately and reads the previous replay's split partials. Tag executions with an epoch instead. The epoch and an exit count live in the unused tail of the first barrier line, so the host-side buffer is unchanged. Every CTA reads the epoch after cudaGridDependencySynchronize, and the last CTA to retire resets the exit count and advances the epoch. Signed-off-by: lightseek-bot <243258330+lightseek-bot@users.noreply.github.com> * Update instructions for deepgemm fork development * Fix SM100 kernels to use the allocated tensor memory base * Keep the pull request limited to the tensor memory base fix * Make the HC-prenorm regression test require a nonzero TMEM base * Skip the nonzero TMEM base HC-prenorm test on CC 10.7 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Signed-off-by: lightseek-bot <243258330+lightseek-bot@users.noreply.github.com> Co-authored-by: LightSeek Foundation <243258330+lightseek-bot@users.noreply.github.com> Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com> Co-authored-by: Chunan Zeng <zcnrex@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
kevmo314
added a commit
to kevmo314/DeepGEMM
that referenced
this pull request
Nov 11, 2026
SM90 grouped 1D2D heuristics, thread-local alignment, MoE precompile, and deepseek-ai#455/deepseek-ai#478
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A CUDA graph kernel node keeps the same
%grididon every replay. Themega_mhcsplit barriers tag executions with%gridid, so after a replay a barrier line still holdsencode_state(grid_idx, kNumSplits)— exactly what the next replay waits for. A Mix or Norm waiter whose m-block init CTA has not started yet proceeds at once and reads the previous replay's split partials (the graph-pool scratch is the same memory).wait_initonly protects producers.This tags each execution with an epoch instead of
%gridid. The epoch and an exit count live in the unused tail of the first barrier line, so the host-side barrier buffer and the compiled extension are unchanged (JIT headers only). Every CTA reads the epoch aftercudaGridDependencySynchronize(); the last CTA to retire resets the exit count and advances the epoch.The MegaMoE combine readiness has the same stale-tag problem and is addressed separately in #451.
Validation on B200: one graph node is replayed while alternating two input sets, and every replay is compared bit for bit with eager results (fresh JIT cache per run). Measured on 66081d4, whose versions of these headers differ from
mainonly by an added#include.Eager calls, and graph replays preceded by an eager launch that rewrites the tag, were 0/200 before the fix as well.