Skip to content

[Fix] [AMD] attention: Default deterministic inference to Triton on HIP - #48083

Open
XinyuJiangCMU wants to merge 2 commits into
sgl-project:mainfrom
XinyuJiangCMU:pr/amd-deterministic-triton-default-20261103
Open

XinyuJiangCMU wants to merge 2 commits into
sgl-project:mainfrom
XinyuJiangCMU:pr/amd-deterministic-triton-default-20261103

Conversation

@XinyuJiangCMU

@XinyuJiangCMU XinyuJiangCMU commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

On ROCm, --enable-deterministic-inference without --attention-backend picks FA3, which is not available on ROCm, so the server fails to start (for example in test_return_routed_experts.py).

Modifications

On HIP, pick Triton instead.

Accuracy Tests

On MI350X, a server started with only --enable-deterministic-inference now starts with Triton and generates normally.

Co-author-with: @JessicaJiang-123


CI States

Latest PR Test (Base): 🚫 Run #43989559859
Latest PR Test (Extra): ❌ Run #43989559441
Latest PR Test (AMD ROCm 10): ✅ Run #43989559411

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

ROCm builds have no FA3, so deterministic inference without an
attention backend failed at startup. Pick Triton on HIP instead.

Co-authored-by: Zhiyao Jiang <jessicajiang324@gmail.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@kevin-mii

Copy link
Copy Markdown
Collaborator

/tag-and-rerun-ci

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Nov 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants