Skip to content

[FlyDSL] Add gfx950 FHMoE tuner in fhmoe_tune.py #5751

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
a-canadasruiz wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

[FlyDSL] Add gfx950 FHMoE tuner in fhmoe_tune.py #5751

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
cd6cfa6
[FlyDSL] Add the FHMoE tuner in fhmoe_tune.py
a-canadasruiz Oct 29, 2026
bf2a57a
[FlyDSL] Add CPU tests for FhmoeTuner
a-canadasruiz Oct 29, 2026
cb4aa5c
[FlyDSL] Cover FHMoE in the Tuning Tests
a-canadasruiz Oct 29, 2026
3d3d54d
[FlyDSL] Run the FHMoE tuner from op_tune.sh
a-canadasruiz Oct 29, 2026
8b80abe
[FlyDSL] Check FHMoE candidates against references that quantize like…
a-canadasruiz Oct 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/scripts/op_tune.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ declare -a tune_jobs=(
"ck_gemm_a8w8_blockscale_bpreshuffle:csrc/ck_gemm_a8w8_blockscale_bpreshuffle:op_tests/test_gemm_a8w8_blockscale.py:python3 csrc/ck_gemm_a8w8_blockscale/gemm_a8w8_blockscale_tune.py --preshuffle -i aiter/configs/a8w8_blockscale_bpreshuffle_untuned_gemm.csv -o aiter/configs/a8w8_blockscale_bpreshuffle_tuned_gemm.csv"
"ck_gemm_a8w8_bpreshuffle:csrc/ck_gemm_a8w8_bpreshuffle:op_tests/test_gemm_a8w8.py:python3 csrc/ck_gemm_a8w8_bpreshuffle/gemm_a8w8_bpreshuffle_tune.py -i aiter/configs/a8w8_bpreshuffle_untuned_gemm.csv -o aiter/configs/a8w8_bpreshuffle_tuned_gemm.csv"
"ck_gemm_moe_2stages_codegen:csrc/ck_gemm_moe_2stages_codegen:op_tests/test_moe.py:python3 csrc/ck_gemm_moe_2stages_codegen/gemm_moe_tune.py -i aiter/configs/untuned_fmoe.csv -o aiter/configs/tuned_fmoe.csv"
"fhmoe:csrc/ck_gemm_moe_2stages_codegen:op_tests/test_fhmoe.py:python3 csrc/ck_gemm_moe_2stages_codegen/fhmoe_tune.py -i aiter/configs/untuned_fhmoe.csv -o aiter/configs/tuned_fhmoe.csv"
#"ck_gemm_a4w4_blockscale:csrc/ck_gemm_a4w4_blockscale:op_tests/test_gemm_a4w4_blockscale.py:python3 csrc/ck_gemm_a4w4_blockscale/gemm_a4w4_blockscale_tune.py -i aiter/configs/a4w4_blockscale_untuned_gemm.csv -o aiter/configs/a4w4_blockscale_tuned_gemm.csv"
)

Expand All @@ -46,6 +47,16 @@ for job in "${tune_jobs[@]}"; do
continue
fi
fi
if [ "$shape" = "fhmoe" ]; then
gfx=$(python3 -c "from aiter.jit.utils.chip_info import get_gfx; print(get_gfx())") || gfx=""
if [ "$gfx" != "gfx950" ]; then
echo "============================================================"
echo "Skipping fhmoe: requires gfx950, got ${gfx:-unknown}"
echo "==============================================="
echo
continue
fi
fi
echo "============================================================"
echo "🧪 Processing shape: $shape under directory: $dir"
echo "------------------------------------------------------------"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tuning-tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ jobs:
-e PYTHONUNBUFFERED=1 \
-w /workspace \
tuning_tests_level01 \
bash -lc "set -euo pipefail; python3 -m unittest op_tests.tuning_tests.test_csv_validation op_tests.tuning_tests.test_tuner_infra op_tests.tuning_tests.test_mp_tuner_logic op_tests.tuning_tests.test_config_shape_collision op_tests.tuning_tests.test_fast_tuning op_tests.tuning_tests.test_a6w6_tuning op_tests.tuning_tests.test_mixed_mxfp_tuning op_tests.tuning_tests.test_a16w16_tuned_config_lookup -v 2>&1 | tee /workspace/${LOG_DIR}/level01.log && python3 -m pytest op_tests/tuning_tests/test_fhmoe_tuner.py -v 2>&1 | tee -a /workspace/${LOG_DIR}/level01.log"
bash -lc "set -euo pipefail; python3 -m unittest op_tests.tuning_tests.test_csv_validation op_tests.tuning_tests.test_tuner_infra op_tests.tuning_tests.test_mp_tuner_logic op_tests.tuning_tests.test_config_shape_collision op_tests.tuning_tests.test_fast_tuning op_tests.tuning_tests.test_a6w6_tuning op_tests.tuning_tests.test_mixed_mxfp_tuning op_tests.tuning_tests.test_a16w16_tuned_config_lookup op_tests.tuning_tests.test_fhmoe_dsv4_tuner -v 2>&1 | tee /workspace/${LOG_DIR}/level01.log && python3 -m pytest op_tests/tuning_tests/test_fhmoe_tuner.py -v 2>&1 | tee -a /workspace/${LOG_DIR}/level01.log"

- name: Upload level 0+1 artifacts
if: always()
Expand Down
14 changes: 13 additions & 1 deletion aiter/configs/untuned_fhmoe.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,13 @@
gfx,cu_num,token,model_dim,inter_dim,expert,topk,act_type,dtype,q_dtype_a,q_dtype_w,q_type,use_g1u1,doweight_stage1
gfx,cu_num,token,model_dim,inter_dim,expert,topk,shared_expert_id,act_type,dtype,q_dtype_a,q_dtype_w,q_type,use_g1u1,doweight_stage1,hidden_pad,intermediate_pad,gate_mode
gfx950,256,1,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
gfx950,256,2,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
gfx950,256,4,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
gfx950,256,8,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
gfx950,256,16,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
gfx950,256,32,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
gfx950,256,64,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
gfx950,256,128,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
gfx950,256,256,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
gfx950,256,512,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
gfx950,256,1024,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
gfx950,256,2048,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
Loading
Loading