A problem with ver0.0.28post1 on ROCm for a1111 sdwebui picture generating

[PennyFranklin]

I git clone the a1111 sdwebui and managed to install the pytorch2.4.1-rocm6.1,and it work well .Then i installed the xformers0.0.28post1 ,the trouble comes.When i fill the prompt and cick che generate botton ,the terminal and GPU sound shows the process ,but after seconds turn out an error.

[lw]

The screenshot you posted doesn't actually show the error. Please post the full logs in text format if you want us to look into it.

[PennyFranklin]

When i run a benchmark in vlad's system info extension,it turns out that error:

penny@Neko:~/stable-diffusion-webui$ '/home/penny/stable-diffusion-webui/webui.sh' --reinstall-xformers --xformers

################################################################
Install script for stable-diffusion + Web UI
Tested on Debian 11 (Bullseye), Fedora 34+ and openSUSE Leap 15.4 or newer.
################################################################

################################################################
Running on penny user
################################################################

################################################################
Repo already cloned, using it as install directory
################################################################

################################################################
Create and activate python venv
################################################################

################################################################
Launching launch.py...
################################################################
glibc version is 2.35
Cannot locate TCMalloc. Do you have tcmalloc or google-perftool installed on your system? (improves CPU memory usage)
Python 3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0]
Version: v1.10.1
Commit hash: 82a973c04367123ae98bd9abdf80d9eda9b910e2
Installing xformers
Launching Web UI with arguments: --reinstall-xformers --xformers
WARNING:xformers:WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
PyTorch 2.4.1+cu121 with CUDA 1201 (you have 2.4.1+rocm6.1)
Python 3.10.15 (you have 3.10.12)
Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)
Memory-efficient attention, SwiGLU, sparse and more won't be available.
Set XFORMERS_MORE_DETAILS=1 for more details
*** Error running preload() for /home/penny/stable-diffusion-webui/extensions/stable-diffusion-webui-wd14-tagger/preload.py
Traceback (most recent call last):
File "/home/penny/stable-diffusion-webui/modules/script_loading.py", line 30, in preload_extensions
module = load_module(preload_script)
File "/home/penny/stable-diffusion-webui/modules/script_loading.py", line 13, in load_module
module_spec.loader.exec_module(module)
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/penny/stable-diffusion-webui/extensions/stable-diffusion-webui-wd14-tagger/preload.py", line 4, in
from modules.shared import models_path
ImportError: cannot import name 'models_path' from partially initialized module 'modules.shared' (most likely due to a circular import) (/home/penny/stable-diffusion-webui/modules/shared.py)

---

Tag Autocomplete: Could not locate model-keyword extension, Lora trigger word completion will be limited to those added through the extra networks menu.
sd-webui-prompt-all-in-one background API service started successfully.
*** Error loading script: tagger.py
Traceback (most recent call last):
File "/home/penny/stable-diffusion-webui/modules/scripts.py", line 515, in load_scripts
script_module = script_loading.load_module(scriptfile.path)
File "/home/penny/stable-diffusion-webui/modules/script_loading.py", line 13, in load_module
module_spec.loader.exec_module(module)
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/penny/stable-diffusion-webui/extensions/stable-diffusion-webui-wd14-tagger/scripts/tagger.py", line 5, in
from tagger.ui import on_ui_tabs
File "/home/penny/stable-diffusion-webui/extensions/stable-diffusion-webui-wd14-tagger/tagger/ui.py", line 10, in
from webui import wrap_gradio_gpu_call
ImportError: cannot import name 'wrap_gradio_gpu_call' from 'webui' (/home/penny/stable-diffusion-webui/webui.py)

---

Loading weights [7c819b6d13] from /home/penny/stable-diffusion-webui/models/Stable-diffusion/majicmixRealistic_v7.safetensors
Running on local URL: http://127.0.0.1:7860
Creating model from config: /home/penny/stable-diffusion-webui/configs/v1-inference.yaml
/home/penny/stable-diffusion-webui/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: resume_download is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use force_download=True.
warnings.warn(
Applying attention optimization: sdp-no-mem... done.
Model loaded in 1.8s (load weights from disk: 0.3s, create model: 0.2s, apply weights to model: 0.9s, calculate empty prompt: 0.1s).

To create a public link, set share=True in launch().
Startup time: 10.4s (prepare environment: 3.1s, import torch: 1.6s, import gradio: 0.3s, setup paths: 2.5s, other imports: 0.2s, load scripts: 0.3s, create ui: 0.2s, gradio launch: 2.2s).
正在现有浏览器会话中打开。
WARNING:root:Sampler Scheduler autocorrection: "Euler a" -> "Euler a", "None" -> "Automatic"
/home/penny/stable-diffusion-webui/modules/safe.py:156: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
return unsafe_torch_load(filename, *args, **kwargs)
0%| | 0/20 [00:00<?, ?it/s]/usr/lib/python3.10/contextlib.py:103: FutureWarning: torch.backends.cuda.sdp_kernel() is deprecated. In the future, this context manager will be removed. Please see torch.nn.attention.sdpa_kernel() for the new context manager, with updated signature.
self.gen = func(*args, **kwds)
100%|███████████████████████████████████████████| 20/20 [00:01<00:00, 18.65it/s]
ERROR:sd:SD-System-Info benchmark error: 1 No operator found for memory_efficient_attention_forward with inputs:
query : shape=(1, 4096, 1, 512) (torch.float16)
key : shape=(1, 4096, 1, 512) (torch.float16)
value : shape=(1, 4096, 1, 512) (torch.float16)
attn_bias : <class 'NoneType'>
p : 0.0
ckF is not supported because:
max(query.shape[-1], value.shape[-1]) > 256
operator wasn't built - see python -m xformers.info for more info
WARNING:root:Sampler Scheduler autocorrection: "Euler a" -> "Euler a", "None" -> "Automatic"
100%|███████████████████████████████████████████| 20/20 [00:00<00:00, 20.65it/s]
ERROR:sd:SD-System-Info benchmark error: 1 No operator found for memory_efficient_attention_forward with inputs:
query : shape=(1, 4096, 1, 512) (torch.float16)
key : shape=(1, 4096, 1, 512) (torch.float16)
value : shape=(1, 4096, 1, 512) (torch.float16)
attn_bias : <class 'NoneType'>
p : 0.0
ckF is not supported because:
max(query.shape[-1], value.shape[-1]) > 256
operator wasn't built - see python -m xformers.info for more info
WARNING:root:Sampler Scheduler autocorrection: "Euler a" -> "Euler a", "None" -> "Automatic"
100%|███████████████████████████████████████████| 20/20 [00:01<00:00, 11.10it/s]
ERROR:sd:SD-System-Info benchmark error: 2 No operator found for memory_efficient_attention_forward with inputs:
query : shape=(1, 4096, 1, 512) (torch.float16)
key : shape=(1, 4096, 1, 512) (torch.float16)
value : shape=(1, 4096, 1, 512) (torch.float16)
attn_bias : <class 'NoneType'>
p : 0.0
ckF is not supported because:
max(query.shape[-1], value.shape[-1]) > 256
operator wasn't built - see python -m xformers.info for more info
WARNING:root:Sampler Scheduler autocorrection: "Euler a" -> "Euler a", "None" -> "Automatic"
100%|███████████████████████████████████████████| 20/20 [00:03<00:00, 5.75it/s]
ERROR:sd:SD-System-Info benchmark error: 4 No operator found for memory_efficient_attention_forward with inputs:
query : shape=(1, 4096, 1, 512) (torch.float16)
key : shape=(1, 4096, 1, 512) (torch.float16)
value : shape=(1, 4096, 1, 512) (torch.float16)
attn_bias : <class 'NoneType'>
p : 0.0
ckF is not supported because:
max(query.shape[-1], value.shape[-1]) > 256
operator wasn't built - see python -m xformers.info for more info

when i generate the picture in txt2img ,here is the code:
*** Error completing request 4.43s/it]
*** Arguments: ('task(femn6y84yofgpcx)', <gradio.routes.Request object at 0x7b50d04eb640>, '1girl, ', '', [], 1, 1, 7, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', 'Use same scheduler', '', '', [], 0, 20, 'DPM++ 2M', 'Automatic', False, '', 0.8, -1, False, -1, 0, 0, 0, 'NONE:0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0\nALL:1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\nINS:1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0\nIND:1,0,0,0,1,1,1,0,0,0,0,0,0,0,0,0,0\nINALL:1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0\nMIDD:1,0,0,0,1,1,1,1,1,1,1,1,0,0,0,0,0\nOUTD:1,0,0,0,0,0,0,0,1,1,1,1,0,0,0,0,0\nOUTS:1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1\nOUTALL:1,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1\nALL0.5:0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5', True, 0, 'values', '0,0.25,0.5,0.75,1', 'Block ID', 'IN05-OUT05', 'none', '', '0.5,1', 'BASE,IN00,IN01,IN02,IN03,IN04,IN05,IN06,IN07,IN08,IN09,IN10,IN11,M00,OUT00,OUT01,OUT02,OUT03,OUT04,OUT05,OUT06,OUT07,OUT08,OUT09,OUT10,OUT11', 1.0, 'black', '20', False, 'ATTNDEEPON:IN05-OUT05:attn:1\n\nATTNDEEPOFF:IN05-OUT05:attn:0\n\nPROJDEEPOFF:IN05-OUT05:proj:0\n\nXYZ:::1', False, False, False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, False, False, False, 0, False) {}
Traceback (most recent call last):
File "/home/penny/stable-diffusion-webui/modules/call_queue.py", line 74, in f
res = list(func(*args, **kwargs))
File "/home/penny/stable-diffusion-webui/modules/call_queue.py", line 53, in f
res = func(*args, **kwargs)
File "/home/penny/stable-diffusion-webui/modules/call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "/home/penny/stable-diffusion-webui/modules/txt2img.py", line 109, in txt2img
processed = processing.process_images(p)
File "/home/penny/stable-diffusion-webui/modules/processing.py", line 847, in process_images
res = process_images_inner(p)
File "/home/penny/stable-diffusion-webui/modules/processing.py", line 1002, in process_images_inner
x_samples_ddim = decode_latent_batch(p.sd_model, samples_ddim, target_device=devices.cpu, check_for_nans=True)
File "/home/penny/stable-diffusion-webui/modules/processing.py", line 632, in decode_latent_batch
sample = decode_first_stage(model, batch[i:i + 1])[0]
File "/home/penny/stable-diffusion-webui/modules/sd_samplers_common.py", line 76, in decode_first_stage
return samples_to_images_tensor(x, approx_index, model)
File "/home/penny/stable-diffusion-webui/modules/sd_samplers_common.py", line 58, in samples_to_images_tensor
x_sample = model.decode_first_stage(sample.to(model.first_stage_model.dtype))
File "/home/penny/stable-diffusion-webui/modules/sd_hijack_utils.py", line 22, in
setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
File "/home/penny/stable-diffusion-webui/modules/sd_hijack_utils.py", line 36, in call
return self.__orig_func(*args, **kwargs)
File "/home/penny/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/home/penny/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 826, in decode_first_stage
return self.first_stage_model.decode(z)
File "/home/penny/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/autoencoder.py", line 90, in decode
dec = self.decoder(z)
File "/home/penny/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/penny/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/penny/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/model.py", line 631, in forward
h = self.mid.attn_1(h)
File "/home/penny/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/penny/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/home/penny/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/model.py", line 258, in forward
out = xformers.ops.memory_efficient_attention(q, k, v, attn_bias=None, op=self.attention_op)
File "/home/penny/stable-diffusion-webui/venv/lib/python3.10/site-packages/xformers/ops/fmha/init.py", line 301, in memory_efficient_attention
return _memory_efficient_attention(
File "/home/penny/stable-diffusion-webui/venv/lib/python3.10/site-packages/xformers/ops/fmha/init.py", line 462, in _memory_efficient_attention
return _memory_efficient_attention_forward(
File "/home/penny/stable-diffusion-webui/venv/lib/python3.10/site-packages/xformers/ops/fmha/init.py", line 481, in _memory_efficient_attention_forward
op = _dispatch_fw(inp, False)
File "/home/penny/stable-diffusion-webui/venv/lib/python3.10/site-packages/xformers/ops/fmha/dispatch.py", line 135, in _dispatch_fw
return _run_priority_list(
File "/home/penny/stable-diffusion-webui/venv/lib/python3.10/site-packages/xformers/ops/fmha/dispatch.py", line 76, in _run_priority_list
raise NotImplementedError(msg)
NotImplementedError: No operator found for memory_efficient_attention_forward with inputs:
query : shape=(1, 4096, 1, 512) (torch.float16)
key : shape=(1, 4096, 1, 512) (torch.float16)
value : shape=(1, 4096, 1, 512) (torch.float16)
attn_bias : <class 'NoneType'>
p : 0.0
ckF is not supported because:
max(query.shape[-1], value.shape[-1]) > 256
operator wasn't built - see python -m xformers.info for more info

---

Total progress: 100it [06:29, 4.43s/it]

[LeonardoSidney]

Same here, this error:

`ckF` is not supported because:
    max(query.shape[-1], value.shape[-1]) > 256
    dtype=torch.float32 (supported: {torch.bfloat16, torch.float16})

Always happen to me, I also tested with sd-scripts.

i liked this bench tool, then i will use it too:

System:
arch:x86_64
cpu: 5800x3D
gpu: RX 7900 XTX
system: Arch Linux
Linux :6.11.6-zen1-1-zen
python:3.10.14
torch: 2.5.1+rocm6.1
xformers: 0.0.28.post3

web-ui-output

❯ LD_PRELOAD=/usr/lib/libtcmalloc_minimal.so.4 ./webui.sh

################################################################

Install script for stable-diffusion + Web UI
Tested on Debian 11 (Bullseye), Fedora 34+ and openSUSE Leap 15.4 or newer.

################################################################

################################################################
Running on User user
################################################################

################################################################
Repo already cloned, using it as install directory
################################################################

################################################################
python venv already activate or run without venv: /run/media/User/repos/stable-diffusion-webui/venv
################################################################

################################################################
Accelerating launch.py...
################################################################
Python 3.10.14 (main, Oct 26 2024, 15:47:19) [GCC 14.2.1 20240910]
Version: v1.10.1
Commit hash: 82a973c04367123ae98bd9abdf80d9eda9b910e2
Launching Web UI with arguments: --upcast-sampling --no-half --xformers
/run/media/User/repos/stable-diffusion-webui/venv/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
  warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
Loading weights [6ce0161689] from /run/media/User/repos/stable-diffusion-webui/models/Stable-diffusion/v1-5-pruned-emaonly.safetensors
Running on local URL:  http://127.0.0.1:7860


To create a public link, set `share=True` in `launch()`.
Creating model from config: /run/media/User/repos/stable-diffusion-webui/configs/v1-inference.yaml
/run/media/User/repos/stable-diffusion-webui/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py:797: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
Startup time: 8.9s (prepare environment: 1.8s, import torch: 2.2s, import gradio: 0.4s, setup paths: 3.4s, initialize shared: 0.2s, other imports: 0.2s, load scripts: 0.2s, create ui: 0.3s, gradio launch: 0.3s).
Applying attention optimization: Doggettx... done.
Model loaded in 3.0s (load weights from disk: 0.5s, create model: 0.2s, apply weights to model: 1.9s, load textual inversion embeddings: 0.2s, calculate empty prompt: 0.2s).

WARNING:root:Sampler Scheduler autocorrection: "Euler a" -> "Euler a", "None" -> "Automatic"
/run/media/User/repos/stable-diffusion-webui/modules/safe.py:156: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  return unsafe_torch_load(filename, *args, **kwargs)
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 10.11it/s]
ERROR:sd:SD-System-Info benchmark error: 1 No operator found for `memory_efficient_attention_forward` with inputs:████████████| 20/20 [00:01<00:00,  9.81it/s]
     query       : shape=(1, 4096, 1, 512) (torch.float32)
     key         : shape=(1, 4096, 1, 512) (torch.float32)
     value       : shape=(1, 4096, 1, 512) (torch.float32)
     attn_bias   : <class 'NoneType'>
     p           : 0.0
`ckF` is not supported because:
    max(query.shape[-1], value.shape[-1]) > 256
    dtype=torch.float32 (supported: {torch.bfloat16, torch.float16})
WARNING:root:Sampler Scheduler autocorrection: "Euler a" -> "Euler a", "None" -> "Automatic"
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:05<00:00,  9.79it/s]
ERROR:sd:SD-System-Info benchmark error: 1 No operator found for `memory_efficient_attention_forward` with inputs:                                            
     query       : shape=(1, 4096, 1, 512) (torch.float32)
     key         : shape=(1, 4096, 1, 512) (torch.float32)
     value       : shape=(1, 4096, 1, 512) (torch.float32)
     attn_bias   : <class 'NoneType'>
     p           : 0.0
`ckF` is not supported because:
    max(query.shape[-1], value.shape[-1]) > 256
    dtype=torch.float32 (supported: {torch.bfloat16, torch.float16})
WARNING:root:Sampler Scheduler autocorrection: "Euler a" -> "Euler a", "None" -> "Automatic"
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:09<00:00,  5.15it/s]
ERROR:sd:SD-System-Info benchmark error: 2 No operator found for `memory_efficient_attention_forward` with inputs:
     query       : shape=(1, 4096, 1, 512) (torch.float32)
     key         : shape=(1, 4096, 1, 512) (torch.float32)
     value       : shape=(1, 4096, 1, 512) (torch.float32)
     attn_bias   : <class 'NoneType'>
     p           : 0.0
`ckF` is not supported because:
    max(query.shape[-1], value.shape[-1]) > 256
    dtype=torch.float32 (supported: {torch.bfloat16, torch.float16})
WARNING:root:Sampler Scheduler autocorrection: "Euler a" -> "Euler a", "None" -> "Automatic"
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:19<00:00,  2.59it/s]
ERROR:sd:SD-System-Info benchmark error: 4 No operator found for `memory_efficient_attention_forward` with inputs:
     query       : shape=(1, 4096, 1, 512) (torch.float32)
     key         : shape=(1, 4096, 1, 512) (torch.float32)
     value       : shape=(1, 4096, 1, 512) (torch.float32)
     attn_bias   : <class 'NoneType'>
     p           : 0.0
`ckF` is not supported because:
    max(query.shape[-1], value.shape[-1]) > 256
    dtype=torch.float32 (supported: {torch.bfloat16, torch.float16})

[rrunner77]

I have the same issue. Do I understand this right that the tensor:

query       : shape=(1, 4096, 1, 512) (torch.float32)
     key         : shape=(1, 4096, 1, 512) (torch.float32)
     value       : shape=(1, 4096, 1, 512) (torch.float32)
     attn_bias   : <class 'NoneType'>
     p           : 0.0

Has the 4th value 512, which is the cause of the error as the AMD composed_kernel tiled do not support higher value than 256 ?
So the only problem to fix this should be to split the tensor that the 4th parameter is only 256 ?

I would say that this is a issue on the side of either A1111 or stable diffusion.

Can anyone confirm my assumption ?