Skip to content

[HIP] Tune DSV4.1 TP4 A4W4 fused MoE configs for gfx950 #5919

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
valarLip merged 1 commit into from
Sep 29, 2026
Merged

[HIP] Tune DSV4.1 TP4 A4W4 fused MoE configs for gfx950 #5919

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 32 additions & 14 deletions aiter/configs/model_configs/dsv41_fp4_tuned_fmoe.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,20 +31,38 @@ gfx950,256,4096,5120,2304,32,3,ActivationType.Silu,torch.bfloat16,torch.float4_e
gfx950,256,8192,5120,2304,32,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,64,0,386.7705,flydsl_moe1_afp4_wfp4_bf16_t64x128x256_w2_bnt0_xcd4_fp4,0.9%,282.22,flydsl_moe2_layout_afp4_wfp4_bf16_t64x256x256_atomic_persist_sbm64,1.2%,668.9905,0,0,0,2600.13,1880.88
gfx950,256,16384,5120,2304,32,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,64,0,715.8345,flydsl_moe1_afp4_wfp4_bf16_t64x128x256_w4_bnt0_xcd4_fp4,0.1%,574.9296,flydsl_moe2_afp4_wfp4_bf16_t64x128x256_atomic_bnt2,2.6%,1290.7641,0,0,0,2695.24,1072.33
gfx950,256,32768,5120,2304,32,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,64,0,1318.8618,flydsl_moe1_afp4_wfp4_bf16_t64x128x256_w3_bnt0_xcd4_fp4,0.1%,1119.4776,flydsl_moe2_afp4_wfp4_bf16_t64x128x256_atomic_bnt2,2.6%,2438.3394,0,0,0,2853.52,670.86
gfx950,256,4,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,16.3079,flydsl_moe1_afp4_wfp4_bf16_t32x32x256_w4_kw4_fp4,0.9%,13.1717,flydsl_moe2_layout_afp4_wfp4_bf16_t32x256x128_reduce_nt_sbm32,1.1%,29.4796,0,0,0,16.01,128052.45
gfx950,256,8,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,26.5519,flydsl_moe1_afp4_wfp4_bf16_t32x128x256_w3_fp4,0.2%,19.6944,flydsl_moe2_afp4_wfp4_bf16_t32x128x256_reduce,0.2%,46.2463,0,0,0,20.41,81628.08
gfx950,256,16,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,47.1763,flydsl_moe1_afp4_wfp4_bf16_t32x128x256_w3_fp4,0.2%,31.4068,flydsl_moe2_afp4_wfp4_bf16_t32x128x256_atomic,1.4%,78.5831,0,0,0,24.02,48039.84
gfx950,256,32,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,82.1226,flydsl_moe1_afp4_wfp4_bf16_t32x128x256_w3_fp4,0.2%,51.2073,flydsl_moe2_afp4_wfp4_bf16_t32x128x128_atomic,1.5%,133.3299,0,0,0,28.31,28315.97
gfx950,256,64,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,134.6773,flydsl_moe1_afp4_wfp4_bf16_t32x128x256_w4_fp4,0.2%,81.7526,flydsl_moe2_afp4_wfp4_bf16_t16x128x128_atomic_bnt2_xcd4_persist_sbm32,1.4%,216.4299,0,0,0,34.88,17446.1
gfx950,256,128,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,181.8681,flydsl_moe1_afp4_wfp4_bf16_t32x128x256_w4_fp4,0.2%,110.5162,flydsl_moe2_afp4_wfp4_bf16_t32x256x256_reduce_bnt2_persist,0.2%,292.3843,0,0,0,51.64,12917.38
gfx950,256,256,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,207.1576,flydsl_moe1_afp4_wfp4_bf16_t32x64x256_w3_kw2_fp4,0.9%,112.2781,flydsl_moe2_layout_afp4_wfp4_bf16_t32x256x128_atomic_nt_sbm32,1.1%,319.4357,0,0,0,94.54,11829.63
gfx950,256,512,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,206.8732,flydsl_moe1_afp4_wfp4_bf16_t32x128x256_w2_fp4,0.9%,116.5511,flydsl_moe2_layout_afp4_wfp4_bf16_t32x256x128_atomic_nt_sbm32,1.2%,323.4243,0,0,0,186.75,11695.9
gfx950,256,1024,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,210.2328,flydsl_moe1_afp4_wfp4_bf16_t32x128x256_w4_fp4,0.9%,125.1489,flydsl_moe2_layout_afp4_wfp4_bf16_t32x256x128_atomic_persist_nt_sbm32,1.2%,335.3817,0,0,0,360.17,11302.35
gfx950,256,2048,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,209.8211,flydsl_moe1_afp4_wfp4_bf16_t32x128x256_w3_fp4,0.9%,137.584,flydsl_moe2_layout_afp4_wfp4_bf16_t32x256x128_atomic_nt_sbm32,1.2%,347.4051,0,0,0,695.42,10956.46
gfx950,256,4096,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,64,0,218.2178,flydsl_moe1_afp4_wfp4_bf16_t64x128x256_w2_fp4,0.9%,204.8485,flydsl_moe2_layout_afp4_wfp4_bf16_t64x256x128_reduce_persist_nt_sbm64,1.2%,423.0663,0,0,0,1142.1,9071.36
gfx950,256,8192,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,128,0,287.2784,flydsl_moe1_afp4_wfp4_bf16_t128x128x256_fp4,0.9%,369.0678,flydsl_moe2_layout_afp4_wfp4_bf16_t128x128x128_reduce_nt_sbm128,1.2%,656.3462,0,0,0,1472.34,5943.06
gfx950,256,16384,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,128,0,500.1166,flydsl_moe1_afp4_wfp4_bf16_t128x128x256_w3_bnt0_fp4,0.9%,686.7928,flydsl_moe2_layout_afp4_wfp4_bf16_t64x256x128_reduce_persist_sbm128,1.2%,1186.9094,0,0,0,1628.38,3392.45
gfx950,256,32768,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,64,0,861.8038,flydsl_moe1_afp4_wfp4_bf16_t64x128x256_w3_bnt0_xcd4_fp4,0.9%,1348.7122,flydsl_moe2_layout_afp4_wfp4_bf16_t64x256x128_reduce_sbm64,1.2%,2210.516,0,0,0,1748.67,1935.38
gfx950,256,1,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,18.2894,flydsl_mxmoe_g1_a4w4_16x128x256_f16in_hpf_nt,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x128x128_atomic_sbm16,1.2%,18.2894,0,0,0,6.45,206397.64
gfx950,256,2,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,19.8613,flydsl_mxmoe_g1_a4w4_16x128x256_f16in_hpf_nt_xcd4,1.1%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x128x128_atomic_sbm16,1.1%,19.8613,0,0,0,11.88,190063.31
gfx950,256,4,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,26.5561,flydsl_mxmoe_g1_a4w4_16x128x256_f16in_hpf_nt,1.1%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x128x128_atomic_nt_sbm16,1.1%,26.5561,0,0,0,17.77,142149.45
gfx950,256,8,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,42.029,flydsl_mxmoe_g1_a4w4_16x128x256_f16in_nt,1.1%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x128x128_atomic_nt_sbm16,1.1%,42.029,0,0,0,22.45,89818.85
gfx950,256,16,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,71.6339,flydsl_mxmoe_g1_a4w4_16x256x256_f16in_hpf_nt,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x128x128_atomic_sbm16,1.2%,71.6339,0,0,0,26.35,52700.18
gfx950,256,32,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,125.319,flydsl_mxmoe_g1_a4w4_16x128x256_f16in_nt_xcd2,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x128x128_atomic_nt_sbm16,1.2%,125.319,0,0,0,30.12,30126.04
gfx950,256,64,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,203.8522,flydsl_mxmoe_g1_a4w4_16x128x256_f16in_nt,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x128x128_atomic_nt_sbm16,1.2%,203.8522,0,0,0,37.04,18522.52
gfx950,256,128,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,280.7103,flydsl_mxmoe_g1_a4w4_16x128x256_f16in_hpf_nt_xcd2,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x256x128_atomic_nt_sbm16,1.2%,280.7103,0,0,0,53.79,13454.58
gfx950,256,256,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,302.6034,flydsl_mxmoe_g1_a4w4_16x128x256_f16in_hpf_nt_xcd2,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x128x128_reduce_nt_sbm16,1.2%,302.6034,0,0,0,99.8,12487.65
gfx950,256,512,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,322.3888,flydsl_mxmoe_g1_a4w4_16x128x256_f16in_hpf_nt_xcd2,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x256x128_atomic_persist_nt_sbm16,1.2%,322.3888,0,0,0,187.35,11733.47
gfx950,256,1024,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,337.0362,flydsl_mxmoe_g1_a4w4_32x128x256_nt_kw2_xcd2,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t32x256x128_atomic_persist_nt_sbm32,1.2%,337.0362,0,0,0,358.41,11246.87
gfx950,256,2048,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,64,0,377.2891,flydsl_mxmoe_g1_a4w4_64x256x256_nt_xcd2,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t64x256x128_atomic_persist_nt_sbm64,1.2%,377.2891,0,0,0,640.34,10088.63
gfx950,256,4096,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,64,0,522.9601,flydsl_mxmoe_g1_a4w4_64x256x256_xcd4,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t64x128x128_atomic_sbm64,1.2%,522.9601,0,0,0,923.94,7338.59
gfx950,256,8192,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,64,0,777.1513,flydsl_mxmoe_g1_a4w4_64x256x256_xcd2,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t64x256x128_reduce_sbm64,1.2%,777.1513,0,0,0,1243.47,5019.23
gfx950,256,16384,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,64,0,1266.537,flydsl_mxmoe_g1_a4w4_64x256x256_xcd2,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t64x256x128_reduce_sbm64,1.2%,1266.537,0,0,0,1526.0,3179.17
gfx950,256,32768,5120,640,384,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,128,0,2336.8151,flydsl_mxmoe_g1_a4w4_128x256x256_xcd4,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t128x128x128_reduce_sbm128,1.2%,2336.8151,0,0,0,1654.16,1830.78
gfx950,256,1,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,9.4735,flydsl_moe1_afp4_wfp4_bf16_t32x32x256_w4_kw4_fp4,1.0%,7.0259,flydsl_moe2_layout_afp4_wfp4_bf16_t32x128x128_atomic_nt_sbm32,1.3%,16.4994,0,0,0,3.57,76263.78
gfx950,256,2,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,9.9936,flydsl_moe1_afp4_wfp4_bf16_t32x32x256_w3_bnt0_kw4_fp4,1.0%,7.9353,flydsl_moe2_layout_afp4_wfp4_bf16_t32x128x128_atomic_nt_sbm32,1.3%,17.9289,0,0,0,6.58,70184.0
gfx950,256,4,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,11.6665,flydsl_moe1_afp4_wfp4_bf16_t32x32x256_w4_bnt0_kw4_fp4,1.0%,8.7818,flydsl_moe2_layout_afp4_wfp4_bf16_t32x256x128_atomic_sbm32,1.2%,20.4483,0,0,0,11.54,61538.25
gfx950,256,8,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,25.4594,flydsl_mxmoe_g1_a4w4_16x128x256_f16in_hpf_nt_xcd4,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x128x128_atomic_nt_sbm16,1.2%,25.4594,0,0,0,18.53,49428.27
gfx950,256,16,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,41.705,flydsl_mxmoe_g1_a4w4_16x256x256_f16in_hpf_nt_xcd4,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x128x128_atomic_nt_sbm16,1.2%,41.705,0,0,0,22.63,30177.12
gfx950,256,32,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,60.5249,flydsl_mxmoe_g1_a4w4_16x128x256_f16in_nt,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x256x128_atomic_sbm16,1.2%,60.5249,0,0,0,31.18,20797.77
gfx950,256,64,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,86.5754,flydsl_mxmoe_g1_a4w4_16x128x256_f16in_hpf_nt_xcd2,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x128x128_atomic_sbm16,1.2%,86.5754,0,0,0,43.6,14545.4
gfx950,256,128,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,105.7823,flydsl_mxmoe_g1_a4w4_16x128x256_f16in_nt,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x256x128_atomic_sbm16,1.2%,105.7823,0,0,0,71.37,11913.69
gfx950,256,256,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,16,0,110.9747,flydsl_mxmoe_g1_a4w4_16x128x256_f16in_hpf_nt_xcd2,1.2%,0,flydsl_moe2_layout_afp4_wfp4_bf16_t16x256x128_atomic_sbm16,1.2%,110.9747,0,0,0,136.06,11373.97
gfx950,256,512,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,74.0799,flydsl_moe1_afp4_wfp4_bf16_t32x64x256_w4_kw2_fp4,0.9%,46.0471,flydsl_moe2_layout_afp4_wfp4_bf16_t32x256x128_atomic_persist_nt_sbm32,1.2%,120.127,0,0,0,251.39,10540.14
gfx950,256,1024,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,76.2482,flydsl_moe1_afp4_wfp4_bf16_t32x128x256_w3_fp4,0.9%,51.0778,flydsl_moe2_layout_afp4_wfp4_bf16_t32x128x128_atomic_persist_nt_sbm32,1.1%,127.326,0,0,0,474.36,10005.97
gfx950,256,2048,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,64,0,81.953,flydsl_moe1_afp4_wfp4_bf16_t64x128x256_w4_fp4,0.9%,65.7851,flydsl_moe2_layout_afp4_wfp4_bf16_t64x128x128_atomic_nt_sbm64,1.2%,147.7381,0,0,0,817.64,8729.97
gfx950,256,4096,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,128,0,103.1603,flydsl_moe1_afp4_wfp4_bf16_t128x128x256_w2_fp4,0.9%,107.4954,flydsl_moe2_layout_afp4_wfp4_bf16_t128x128x128_reduce_nt_sbm128,1.2%,210.6557,0,0,0,1146.86,6271.87
gfx950,256,8192,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,64,0,167.5292,flydsl_moe1_afp4_wfp4_bf16_t64x128x256_w2_bnt0_xcd4_fp4,0.9%,179.8919,flydsl_moe2_layout_afp4_wfp4_bf16_t64x256x128_reduce_sbm64,1.2%,347.4211,0,0,0,1390.77,3983.98
gfx950,256,16384,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,64,0,253.3591,flydsl_moe1_afp4_wfp4_bf16_t64x128x256_w4_bnt0_xcd4_fp4,0.9%,342.5746,flydsl_moe2_layout_afp4_wfp4_bf16_t64x256x128_reduce_persist_sbm64,1.2%,595.9337,0,0,0,1621.6,2533.75
gfx950,256,32768,5120,640,128,3,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,128,0,439.5701,flydsl_moe1_afp4_wfp4_bf16_t128x128x256_w2_bnt0_xcd4_fp4,0.9%,687.5897,flydsl_moe2_layout_afp4_wfp4_bf16_t64x256x128_reduce_sbm128,1.2%,1127.1598,0,0,0,1714.7,1562.87
gfx950,256,1,5120,2304,96,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,15.6341,flydsl_moe1_afp4_wfp4_bf16_t32x64x256_w4_kw4_fp4,0.9%,12.1345,flydsl_moe2_layout_afp4_wfp4_bf16_t32x128x256_reduce_sbm32,1.2%,27.7686,0,0,0,15.29,122346.88
gfx950,256,2,5120,2304,96,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,26.8472,flydsl_moe1_afp4_wfp4_bf16_t32x128x256_w2_fp4,0.9%,17.8067,flydsl_moe2_layout_afp4_wfp4_bf16_t32x256x256_reduce_nt_sbm32,1.1%,44.6539,0,0,0,19.02,76083.32
gfx950,256,4,5120,2304,96,6,ActivationType.Silu,torch.bfloat16,torch.float4_e2m1fn_x2,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,49.6491,flydsl_moe1_afp4_wfp4_bf16_t32x128x256_w3_fp4,0.9%,30.1218,flydsl_moe2_layout_afp4_wfp4_bf16_t32x128x256_atomic_nt_sbm32,1.2%,79.7709,0,0,0,21.29,42590.06
Expand Down
1 change: 1 addition & 0 deletions aiter/ops/moe_mxfp4_aux.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
(384, 7168, 768, 6),
(384, 7168, 512, 6),
(384, 5120, 768, 6),
(128, 5120, 768, 3),
(128, 5120, 1280, 3),
(384, 5120, 1280, 6),
(96, 5120, 2304, 6),
Expand Down
1 change: 1 addition & 0 deletions csrc/kernels/mxfp4_moe/moe_aux/codegen/gen_instances.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
(384, 7168, 768, 6), # dsv4_tp4
(384, 7168, 512, 6), # dsv4_tp6 (and dsv4_tp8: INTER 384->512 padded)
(384, 5120, 768, 6), # dsv4.1 TP=4 (INTER 576->640, support key pads to 768)
(128, 5120, 768, 3), # dsv4.1 DSpark draft TP=4 (INTER 576->640)
(128, 5120, 1280, 3), # dsv4.1 DSpark TP=2 (INTER 1152 pads to 1280)
(384, 5120, 1280, 6), # dsv4.1 target TP=2 (INTER 1152 pads to 1280)
(96, 5120, 2304, 6), # dsv4.1 EP=4, TP=1
Expand Down
Loading