Skip to content

[Fix] diffusion: Cosmos3 Bug fixes - #43194

Merged
mickqian merged 16 commits into
sgl-project:mainfrom
kediwu0331:cosmos-bug-fix
Nov 2, 2026
Merged

mickqian merged 16 commits into
sgl-project:mainfrom
kediwu0331:cosmos-bug-fix

Conversation

@kediwu0331

@kediwu0331 kediwu0331 commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

This PR fixes four issues:

  1. In cosmos3, fp8 kernel will hit cuda illegal memory when sequence length is big.
  2. turn on ulysses sequence parallelism support for action generation.
  3. resolve history_mode from checkpoint config file for sim bimanual
  4. when using sp with ulysses, and when there is only one key/value on a rank, K and V will point to the same shared buffer in memory after all-to-all operation during attention calculation. Basically K is overwritten by V.

Modifications

  1. With 2 720p control videos, the full sequence length is 85560 tokens, super's down_proj input is 25600 wide. In that case 85600x25600 > 2^31. Currently, the tensor pointer y_ptr is int32, which is insufficient. The fix is cast g_id to int64 when calculating the element position.
  2. remove NotImplementedError for action generation when sp > 1. Added some code refactor for sequence sharding.
  3. for sim-bimanual, history_mode defaults to auto. A resolve_history_mode method turns it into full or sliding from the checkpoint config.
  4. in _usp_input_all_to_all, force tensor copy instead of implicit contiguous() call, which will point to shared buffer if tensor is already contiguous. This happens when there is just one key/value on a rank.

Accuracy Tests

Did parity test for action generation with sp > 1.

Speed Tests and Profiling

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ❌ Run #41222912562
Latest PR Test (Extra): ❌ Run #41222912354
Latest PR Test (AMD ROCm 10): ❌ Run #41222912548

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

@kediwu0331

Copy link
Copy Markdown
Contributor Author

@mickqian these are some fixes for cosmos3. Could you review? Thanks

@mickqian

Copy link
Copy Markdown
Collaborator

/tag-and-rerun-ci

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Oct 19, 2026
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

nv-kediw and others added 2 commits October 19, 2026 23:52
@github-actions github-actions Bot added the quant LLM Quantization label Oct 20, 2026
@kediwu0331 kediwu0331 changed the title Cosmos3 fp8 bug fix and action generation improvement [Feat] diffusion: Cosmos3 fp8 bug fix and action generation improvement Oct 20, 2026
@mintlify

mintlify Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit ec2a14e.

Approve preview

@kediwu0331 kediwu0331 changed the title [Feat] diffusion: Cosmos3 fp8 bug fix and action generation improvement [Feat] diffusion: Cosmos3 fp8 bug fix, action generation improvement, sim model improvement Oct 30, 2026
@kediwu0331 kediwu0331 changed the title [Feat] diffusion: Cosmos3 fp8 bug fix, action generation improvement, sim model improvement [Feat] diffusion: Cosmos3 Bug fixes Oct 31, 2026
@kediwu0331

kediwu0331 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor Author

@mickqian I added two more fixes. 3 is just a config fix for cosmos dreams. 4 is more general fix. I encountered this bug when I tested ulysses degree 8 for cosmos3 transfer auto. This issue will be triggered when there is only 1 kv head on a sp rank. Let me know if you are ok with the change. (ps. I think current test failure is not related to this PR. Could you help rerun the failed cases.)

@mickqian mickqian changed the title [Feat] diffusion: Cosmos3 Bug fixes [Fix] diffusion: Cosmos3 Bug fixes Nov 2, 2026
@mickqian
mickqian merged commit 33c8d72 into sgl-project:main Nov 2, 2026
413 of 448 checks passed
Yuxingwang-intel pushed a commit to Yuxingwang-intel/sglang that referenced this pull request Nov 6, 2026
Co-authored-by: Kedi Wu <kediw@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

diffusion SGLang Diffusion documentation Improvements or additions to documentation jit-kernel quant LLM Quantization run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants