Skip to content
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
c7414f8
feat(rdna3): native HIP W4A16 GPTQ kernel for gfx1100 (fp16 + bf16)
JartX Apr 30, 2026
2776808
docs(rdna3): update docstrings + README to match current code state
JartX Apr 30, 2026
78072c0
update readme
JartX Apr 30, 2026
fd73fc0
perf(rdna3): WMMA v2 — 2 waves per block + double-buffered LDS
JartX Apr 30, 2026
9211611
pre-commit
JartX Apr 30, 2026
db41f95
pre-commit
JartX Apr 30, 2026
6bbfd86
Merge branch 'main' into perf/rdna3_w4a16_squashed
JartX Apr 30, 2026
9d549cf
perf(rdna3): tune triton_attn prefill for gfx11 — up to 3.7× faster
JartX May 2, 2026
b648066
perf(rdna3): emit v_dot2_f32_bf16 in W4A16 M=1 bf16 decode path
JartX May 5, 2026
9db576a
perf(rdna3): WMMA v3 — 4 waves/block, 64M × 16N tile for prefill
JartX May 5, 2026
5aaa6b6
perf(rdna3): WMMA v4 (64M×32N) + v_dot2 M=2/4/8 — close gap to Hybrid
JartX May 5, 2026
6579196
perf(rdna3): WMMA v5 — 64M × 64N tile, 4 wmma/wave/K-iter, beat Hybrid
JartX May 5, 2026
2aaa71d
perf(rdna3): K_SPLIT M/N-aware heuristic for v3/v4/v5 launchers
JartX May 5, 2026
1439a70
pre-commit
JartX May 5, 2026
e061486
perf(rdna3): M=1 W4A16 reads A direct from global, no LDS staging
JartX May 5, 2026
9ee907d
perf(rdna3): fp32-internal bf16 dequant for WMMA path, eliminate NaN …
JartX May 5, 2026
3c18e7d
perf(rdna3): drop NaN canon on bf16 narrow in WMMA dequant epilogue
JartX May 5, 2026
bffeb20
perf(rdna3): WMMA V7 — 128M×64N tile, 8 waves/block for prefill
JartX May 5, 2026
9b4fc3a
wip(rdna3): V9 kernel — lanes→K coalesced dequant for [N,K/8] skinny …
JartX May 6, 2026
85da019
perf(rdna3): V7 K-contiguous LDS + remove b_q_perm from hot path
JartX May 6, 2026
cfcc41d
perf(rdna3): cache scale/zero in registers across K-iterations
JartX May 6, 2026
485b840
perf(rdna3): enable WMMA V7 dispatch for fp16 prefill (M≥64)
JartX May 6, 2026
a0939d8
style(rdna3): clang-format fixes in q_gemm_rdna3_wmma.cu
JartX May 6, 2026
9334b9c
perf(rdna3): V8 WMMA kernel — K=32/iter, 8-wave dequant
JartX May 6, 2026
13ca04e
fix(rdna3): use M/N-aware k_split for output zero-init decision
JartX May 7, 2026
33d7461
docs: update README_RDNA3 with V7/V8, fp16 gate, roofline analysis
JartX May 7, 2026
4603f4e
fix(rdna3): always zero-init WMMA output to prevent intermittent garbage
JartX May 7, 2026
0e39944
fix(rdna3): force LDS staging for fp16 M=1 to fix uninitialized reads
JartX May 7, 2026
c03e0dc
style(rdna3): pre-commit format on q_gemm_rdna3.cu
JartX May 11, 2026
c9e6789
Merge branch 'main' into perf/rdna3_w4a16_squashed
JartX May 25, 2026
5ab3df5
cleanup(rdna3): drop diagnostic WMMA ops, rename kernels by tile
JartX May 26, 2026
ecaa8b6
cleanup(rdna3): drop triton attention tuning, keep only W4A16 kernel
JartX May 26, 2026
ec7de95
cleanup(rdna3): remove README_RDNA3.md, drop dead doc refs
JartX May 26, 2026
35d575a
fix(rdna3): gate W4A16 kernel to gfx1100 only, not all gfx11
JartX May 26, 2026
b14e189
feat(rocm): add on_gfx1151() arch helper
JartX May 26, 2026
8e8e619
cleanup(rdna3): drop stale gfx11 dispatch comments
JartX May 26, 2026
649688c
fix(rdna3): correct act-order on M>=128 prefill, validate N%8
JartX May 26, 2026
63b9187
fix(rdna3): move kernels to csrc/quantization/gptq/, guard for NVCC
JartX May 27, 2026
7e66513
refactor(rdna3): move W4A16 kernels to _rocm_C extension
JartX May 27, 2026
e6c1da3
test(rdna3): add W4A16 correctness and kernel-selection tests
JartX May 27, 2026
3929c72
precommit
JartX May 27, 2026
e182874
fix(rdna3): gate W4A16 _rocm_C kernels behind gfx1100 target
JartX May 27, 2026
05f48af
fix(rdna3): apply act-order permutation in bf16 M=1 W4A16 path
JartX May 27, 2026
b7759d9
style(rdna3): clang-format the act-order a_words ternary
JartX May 27, 2026
972ff8d
Merge branch 'main' into perf/rdna3_w4a16_squashed
JartX May 27, 2026
b1074bd
Merge branch 'main' into perf/rdna3_w4a16_squashed
JartX May 28, 2026
bb44707
[Kernel][ROCm] Guard RDNA3 W4A16 kernels for multi-arch wheel builds
JartX May 28, 2026
bc4bc7d
Merge branch 'main' into perf/rdna3_w4a16_squashed
tjtanaa May 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,8 @@ set(VLLM_EXT_SRC
"csrc/topk.cu"
"csrc/cuda_view.cu"
"csrc/quantization/gptq/q_gemm.cu"
"csrc/quantization/gptq/q_gemm_rdna3.cu"
"csrc/quantization/gptq/q_gemm_rdna3_wmma.cu"
"csrc/quantization/w8a8/int8/scaled_quant.cu"
"csrc/quantization/w8a8/fp8/common.cu"
"csrc/quantization/fused_kernels/fused_layernorm_dynamic_per_token_quant.cu"
Expand Down
26 changes: 26 additions & 0 deletions csrc/ops.h
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,32 @@ torch::Tensor gptq_gemm(torch::Tensor a, torch::Tensor b_q_weight,

void gptq_shuffle(torch::Tensor q_weight, torch::Tensor q_perm, int64_t bit);

// W4A16 GPTQ kernels for AMD RDNA3 (gfx1100). See csrc/quantization/gptq/
// README_RDNA3.md for kernel architecture and diagnostic op usage.
torch::Tensor gptq_gemm_rdna3(torch::Tensor a, torch::Tensor b_q_weight,
torch::Tensor b_qzeros, torch::Tensor b_scales,
torch::Tensor b_g_idx, bool use_v2_format);

torch::Tensor gptq_gemm_rdna3_wmma(torch::Tensor a, torch::Tensor b_q_weight,
torch::Tensor b_qzeros,
torch::Tensor b_scales,
torch::Tensor b_g_idx,
bool use_v2_format);

torch::Tensor gptq_gemm_rdna3_wmma_probe(torch::Tensor a, torch::Tensor b,
int64_t mode);

torch::Tensor gptq_gemm_rdna3_wmma_dump(torch::Tensor a,
torch::Tensor b_q_weight,
torch::Tensor b_qzeros,
torch::Tensor b_scales,
bool use_v2_format);

torch::Tensor gptq_gemm_rdna3_wmma_lds_check(torch::Tensor b_q_weight,
torch::Tensor b_qzeros,
torch::Tensor b_scales,
bool use_v2_format);

void static_scaled_fp8_quant(
torch::Tensor& out, torch::Tensor const& input, torch::Tensor const& scale,
std::optional<std::tuple<int64_t, int64_t>> group_shape = std::nullopt);
Expand Down
903 changes: 903 additions & 0 deletions csrc/quantization/gptq/README_RDNA3.md

Large diffs are not rendered by default.

Loading
Loading