Decrease Inference time with torch.compile

[raj7AIfrac]

I am trying to use the torch.compile in the pipe.transformer in Hideam model pipeline.But its failing again and again.
And issue is like with caching (but not sure).
Please make Tal Hideam model pipeline compatible with torch.compile.

Here is some part of logs(its very large log)
from user code:
File "/data05/raj.yadav/HiDream-I1/venv/lib/python3.10/site-packages/diffusers/models/transformers/transformer_hidream_image.py", line 886, in forward
hidden_states, initial_encoder_hidden_states = block(
File "/data05/raj.yadav/HiDream-I1/venv/lib/python3.10/site-packages/diffusers/models/transformers/transformer_hidream_image.py", line 596, in forward
return self.block(
File "/data05/raj.yadav/HiDream-I1/venv/lib/python3.10/site-packages/diffusers/models/transformers/transformer_hidream_image.py", line 552, in forward
) = self.adaLN_modulation(temb)[:, None].chunk(12, dim=-1)
File "/data05/raj.yadav/HiDream-I1/venv/lib/python3.10/site-packages/torch/nn/modules/container.py", line 250, in forward
input = module(input)
File "/data05/raj.yadav/HiDream-I1/venv/lib/python3.10/site-packages/diffusers/quantizers/gguf/utils.py", line 456, in forward
weight = dequantize_gguf_tensor(self.weight)
File "/data05/raj.yadav/HiDream-I1/venv/lib/python3.10/site-packages/diffusers/quantizers/gguf/utils.py", line 386, in dequantize_gguf_tensor
tensor = tensor.view(torch.uint8)

I0530 15:10:47.395000 986910 venv/lib/python3.10/site-packages/torch/_dynamo/utils.py:399] TorchDynamo compilation metrics:
I0530 15:10:47.395000 986910 venv/lib/python3.10/site-packages/torch/_dynamo/utils.py:399] Function, Runtimes (s)
I0530 15:10:47.395000 986910 venv/lib/python3.10/site-packages/torch/_dynamo/utils.py:399] _compile.compile_inner, 0.0000
V0530 15:10:47.396000 986910 venv/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py:122] lru_cache_stats constrain_symbol_range: CacheInfo(hits=0, misses=0, maxsize=None, currsize=0)
V0530 15:10:47.396000 986910 venv/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py:122] lru_cache_stats evaluate_expr: CacheInfo(hits=5036, misses=357, maxsize=256, currsize=256)
V0530 15:10:47.396000 986910 venv/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py:122] lru_cache_stats _simplify_floor_div: CacheInfo(hits=0, misses=0, maxsize=None, currsize=0)
V0530 15:10:47.396000 986910 venv/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py:122] lru_cache_stats _maybe_guard_rel: CacheInfo(hits=10, misses=226, maxsize=256, currsize=226)
V0530 15:10:47.396000 986910 venv/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py:122] lru_cache_stats _find: CacheInfo(hits=2246, misses=143, maxsize=None, currsize=7)

[StrongerXi]

@raj7AIfrac this is some GGUF-related issue, you'll have to wait for PyTorch 2.8 or try nightly. Also see: https://x.com/_DhruvNair_/status/1909301171203846368.