Repository navigation
[ROCm] Bump AITER to v0.1.27 - #63774
Conversation
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
Can we pause this until all test groups are back active? Currently there are several test groups deactivated because of security vulnerabilities |
Yes that makes sense pausing this. |
|
/ci run all |
|
✅ Triggered Buildkite CI #98001 for commit |
|
/ci run all |
|
✅ Triggered Buildkite CI #98082 for commit |
|
/ci retry |
|
❌ @josh-schell-amd, PR authors cannot run CI while the PR is a draft. |
|
/ci retry |
|
✅ Triggered Buildkite CI #98370 for commit |
Signed-off-by: josh schell <josh.schell@amd.com>
6ec84b3 to
9d396c3
Compare
josh-schell-amd
changed the title
|
/ci run all |
|
✅ Triggered Buildkite CI #98414 for commit |
|
These failed on 1:00 PM CST daily full CI run on upstream. Let's wait for them to be resolved. But we could probably ask for a force merge too. |
|
/ci retry |
|
✅ Triggered Buildkite CI #98590 for commit |
micah-wil
left a comment
micah-wil
left a comment
There was a problem hiding this comment.
Waiting to see perf/accuracy sweep before merging
…ecode and hybrid models ROCM_ATTN's HIP paged-attention kernel only serves single-token decode with a power-of-two block size. Speculative decoding verifies several tokens per request, so every step goes through the Triton prefix-prefill kernel (context_attention_fwd). Hybrid models enlarge the attention block to the mamba page size (e.g. 832), which also disables the HIP kernel. In both cases ROCM_AITER_UNIFIED_ATTN / TRITON_ATTN handle the batch in one kernel and are much faster. Keep ROCM_ATTN valid (an explicit --attention-backend ROCM_ATTN still works) but rank it right after TRITON_ATTN for auto-selection in these cases. With aiter v0.1.27 (pinned since vllm-project#63774), unified attention no longer has the 2-4 GiB KV cache bug on gfx950 (vllm-project#62606). On ROCm, the async scheduling accuracy test allows a 10% acceptance drop for spec decode, as for preemption: unified attention is not batch invariant, and the n-gram acceptance count is small. Signed-off-by: Chen, Todd <zhenchen@amd.com>
Bump the ROCm base image to AITER v0.1.27