Activity

1. sirus20x6 commented on Oct 7, 2025

   

   sirus20x6

   on Oct 7, 2025

   More actions

   also have this issue on rtx pro 6000
2. daitisuki commented on Oct 14, 2025

   

   daitisuki

   on Oct 14, 2025

   More actions

   you can try this guy's idea I found,I solve this problem by it:
   It's because the build check in xformers incorrectly enables FA3 for Blackwell, while FA3 only supports GA100/sm80 and Hopper/sm90.

   The invalid argument is triggered by setting an unsupported shared memory size, I think due to limited shared memory size in customer cards (RTX Blackwell/sm120).

   I've found a PR #1262 resolving this issue, which hasn't been merged yet.

   For a temporary workaround, I changed line 19 in xformers/ops/fmha/dispatch.py, setting _USE_FLASH_ATTENTION_3 to False (changing the file in the site-packages directory is enough, no need to rebuild).

   👍React with 👍6Reacted by afu6609, Ray Phan, GaeDearg, paulgrassler, guidenglong1984 and Hans Brouwer
3. guidenglong1984 commented on Nov 28, 2025

   

   guidenglong1984

   on Nov 28, 2025

   More actions

   you can try this guy's idea I found,I solve this problem by it: It's because the build check in xformers incorrectly enables FA3 for Blackwell, while FA3 only supports GA100/sm80 and Hopper/sm90.

   The invalid argument is triggered by setting an unsupported shared memory size, I think due to limited shared memory size in customer cards (RTX Blackwell/sm120).

   I've found a PR #1262 resolving this issue, which hasn't been merged yet.

   For a temporary workaround, I changed line 19 in xformers/ops/fmha/dispatch.py, setting _USE_FLASH_ATTENTION_3 to False (changing the file in the site-packages directory is enough, no need to rebuild).

   thanks，niubi！working on xformers 0.0.32 post2(torch2.8+cuda12.9+python3.12)
4. riverrsea commented on Apr 25, 2026

   

   riverrsea

   on Apr 25, 2026

   Last edited by riverrsea

   More actions

   你可以试试我找到的这个人的方法，我就是用它来解决这个问题的： 这是因为 xformers 中的构建检查错误地为 Blackwell 启用了 FA3，而 FA3 只支持 GA100/sm80 和 Hopper/sm90。

   无效参数的触发原因是设置了不支持的共享内存大小，我认为这是因为客户显卡（RTX Blackwell/sm120）的共享内存大小有限。

   我找到了一个 PR #1262，它解决了这个问题，但尚未合并。

   作为临时解决方案，我修改了 xformers/ops/fmha/dispatch.py​​ 中的第 19 行，将 _USE_FLASH_ATTENTION_3 设置为 False（只需修改 site-packages 目录中的文件即可，无需重新构建）。

   我修改了该文件。但是仍然发生错误。错误如下：

   NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs:
        query       : shape=(10, 77, 8, 64) (torch.float32)
        key         : shape=(10, 77, 8, 64) (torch.float32)
        value       : shape=(10, 77, 8, 64) (torch.float32)
        attn_bias   : <class 'xformers.ops.fmha.attn_bias.LowerTriangularMask'>
        p           : 0.0
   `fa2F@2.5.7-pt` is not supported because:
       dtype=torch.float32 (supported: {torch.bfloat16, torch.float16})
   `cutlassF-pt` is not supported because:
       requires device with capability <= (9, 0) but your GPU has capability (12, 0) (too new)
   

   我正在使用 eva-clip。这是 eva-clip 的求注意力的实现
5. vermasachin6102

   added a commit that references this issue on Jul 25, 2026

   Install xformers in Colab with Blackwell FA3 workaround

   ...

   236077d