Skip to content

[GGUF] Reduce peak RAM usage by casting dequantized tensors early during load - #45386

Merged
SunMarc merged 5 commits into
huggingface:mainfrom
UsamaKenway:gguf-early-dtype-casting
Apr 20, 2026
Merged

[GGUF] Reduce peak RAM usage by casting dequantized tensors early during load#45386
SunMarc merged 5 commits into
huggingface:mainfrom
UsamaKenway:gguf-early-dtype-casting

Merge branch 'main' into gguf-early-dtype-casting

7bee92f
Select commit
Loading
Failed to load commit list.
CircleCI Checks / setup_and_quality succeeded Apr 15, 2026 in 5m 21s

Workflow: setup_and_quality

View CircleCI Workflow

View more details on CircleCI Checks