Skip to content

feat(megatron): add --te-disable-flash-attn-4 so TE can fall back to FlashAttention 2 - #3662

Open
EvilSwordman wants to merge 3 commits into
radixark:mainfrom
EvilSwordman:pr/te-disable-flash-attn-4
Open

EvilSwordman wants to merge 3 commits into
radixark:mainfrom
EvilSwordman:pr/te-disable-flash-attn-4

Conversation

@EvilSwordman

Copy link
Copy Markdown

Problem

Transformer Engine prefers FlashAttention 4 whenever flash-attn-4 is installed. The miles image ships it next to flash_attn 2.7.4.post1, with TE 2.17.

This breaks one hardware case: sm_120 GPUs (the RTX PRO 6000 / RTX 6000D class) with head_dim 256, which is what the full-attention layers of Qwen3.5/3.6/3.8 use. There, FA4 fails to compile (MLIRError: Operation creation failed), and cuDNN has no fused kernel for the shape.

  • With --attention-backend flash, training crashes on the first forward.
  • Otherwise, TE runs the unfused kernel, whose memory grows with the square of the sequence length.

The installed flash_attn 2.7.4 is built for sm_120 and handles head_dim 256 in both forward and backward. The trouble is that TE has no switch for FA4 alone: NVTE_FLASH_ATTN=0 disables every FlashAttention version.

Change

  • A new flag, --te-disable-flash-attn-4, off by default.
  • maybe_hide_te_flash_attn_4(args) in megatron_utils/misc_utils.py. It sets TE's own FlashAttentionUtils.v4_is_installed = False, so TE's backend selection picks FlashAttention 2.
  • The helper is called once, in MegatronTrainRayActor.init, right after monkey_patch_torch_dist(). That means it runs in every training process before any model, bridge or native, is built.
  • A CPU unit test, tests/fast/backends/megatron_utils/test_te_flash_attn_4.py. It runs the helper against a stand-in TE module, so it needs neither TE nor a GPU.

Measurements (RTX 6000D, sm_120)

The first table covers one full-attention layer of Qwen3.8-27B at TP2 (12 query heads, 2 KV heads, head_dim 256, causal, thd), forward plus backward, with the TE backend forced:

tokens unfused: extra memory / time FA2 via this flag: extra memory / time
16K 48.6 GB / 0.68 s 0.6 GB / 0.045 s
32K OOM (a single 48 GiB allocation) 1.2 GB / 0.17 s
64K — 2.4 GB / 0.67 s

FA2's outputs and gradients match an fp32 reference at least as closely as a plain bf16 implementation does. Its worst max-abs error is 0.85x that of bf16.

End to end, we ran Qwen3.8-27B multi-LoRA through the Tinker gateway with TP2, full recompute, and one 2-GPU group per sample. The longest trainable single sample was:

Testing

  • test_te_flash_attn_4.py and the existing test_bridge_mtp_detachment.py both pass: 9 passed.
  • On an sm_120 GPU with this branch:
    • With the flag, TE selects FlashAttention 2.7.4.post1, and forward plus backward runs.
    • Without the flag, TE selects FA4 4.0.0b19 and fails with MLIRError.
  • The end-to-end numbers above come from a fork based on 8cdf379. On that fork, the helper was called from the two bridge model builders. This PR moves the call into the actor's init instead, so the native (non-bridge) path is covered too. The helper itself is unchanged.

…FlashAttention 2

Transformer Engine prefers FlashAttention 4 whenever it is installed. On sm_120
with head_dim 256 (the full-attention layers of Qwen3.5/3.6/3.8), FA4 fails to
compile and cuDNN has no fused kernel for the shape, so TE is left with the
unfused kernel, whose memory grows with the square of the sequence length. The
flash_attn 2 in the image handles this shape, but TE has no switch for FA4 alone
(NVTE_FLASH_ATTN=0 disables every version).

The new flag flips TE's own FA4 detection in every training process, from
MegatronTrainRayActor.init, before any model is built.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Keeps the change clear of the import block that main reworked, so the branch merges cleanly; actor.py already imports this way in several methods.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant