RTX50 series GPU support?

[JELSTUDIO]

I have been able to get it running on CPU, but the torch v2.2 is incompatible with Nvidia RTX50 series GPU (And using a torch nightly for Cuda 12.8 made the infer.py not work)

Is there any chance of an update of some type that would make it GPU-compatible on an RTX50-series card?

Thank you.

[Cypress-Yang]

Thanks for your interest! Could you please provide more details about the error (PyTorch version for CUDA12.8 and the error log from infer.py)?

[JELSTUDIO]

Thanks for your interest! Could you please provide more details about the error (PyTorch version for CUDA12.8 and the error log from infer.py)?

I had a "conversation" with Microsoft's Copilot trying to fix the issue myself.

Here's the first traceback I got, which I mentioned to Copilot:

I'm getting this error because my GPU is too new for the python-script, but where do I enter the term "cpu" that it's asking for?

Traceback (most recent call last): File "D:\AI\SongBloom\infer.py", line 103, in main() File "D:\AI\SongBloom\infer.py", line 77, in main model = SongBloom_Sampler.build_from_trainer(cfg, strict=False, dtype=dtype, device=device) File "D:\AI\SongBloom\SongBloom\models\songbloom\songbloom_pl.py", line 85, in build_from_trainer model_light = SongBloom_PL(cfg) File "D:\AI\SongBloom\SongBloom\models\songbloom\songbloom_pl.py", line 34, in init self.vae = StableVAE(**cfg.vae).eval() File "D:\AI\SongBloom\SongBloom\models\vae_frontend_init_.py", line 53, in init self.vae.load_state_dict(torch.load(vae_ckpt)['state_dict']) File "D:\AI\SongBloom\venvPython310\lib\site-packages\torch\serialization.py", line 1026, in load return _load(opened_zipfile, File "D:\AI\SongBloom\venvPython310\lib\site-packages\torch\serialization.py", line 1438, in _load result = unpickler.load() File "D:\AI\SongBloom\venvPython310\lib\site-packages\torch\serialization.py", line 1408, in persistent_load typed_storage = load_tensor(dtype, nbytes, key, _maybe_decode_ascii(location)) File "D:\AI\SongBloom\venvPython310\lib\site-packages\torch\serialization.py", line 1382, in load_tensor wrap_storage=restore_location(storage, location), File "D:\AI\SongBloom\venvPython310\lib\site-packages\torch\serialization.py", line 391, in default_restore_location result = fn(storage, location) File "D:\AI\SongBloom\venvPython310\lib\site-packages\torch\serialization.py", line 266, in _cuda_deserialize device = validate_cuda_device(location) File "D:\AI\SongBloom\venvPython310\lib\site-packages\torch\serialization.py", line 250, in validate_cuda_device raise RuntimeError('Attempting to deserialize object on a CUDA ' RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.

Copilot then replied:

"You're absolutely right — this error is happening because the model checkpoint you're loading was saved with CUDA (GPU) tensors, but your environment doesn't have a compatible GPU or CUDA setup. To fix this, you need to explicitly tell PyTorch to load the model on the CPU."

I then replied:

"That worked, thanks. The thing is the repo requires torch v2.2 and I'm not sure if there is a torch-version of that that supports the RTX50 series GPU, which is the GPU I'm using. I did try to install torch for cuda 12.8 with a link to pytorch.org but that version of torch didn't seem to work. Am I left to CPU only mode do you think?"

Copilot then replied:

"You're not necessarily stuck with CPU-only mode — but official Torch 2.2.0 builds don't yet support the RTX 50 series (Blackwell architecture) or CUDA 12.8. You’ll need a workaround like a custom build or nightly version."

The torch I tried to install is this:

pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

Which comes out as torch 2.9.x

I also tried installing Cuda toolkit 12.6, but that doesn't support the RTX50-series GPU though it does support torch 2.2 (This only works up to the RTX40 series GPU)

Toolkit 12.8 is the lowest for the RTX50-series, and torch 2.2 is not available there.

I then wrote to Copilot:

"The repo does run on CPU, but not on the nightly 12.8 torch version (Which I am using with other repos where it does give Cuda support). So to make the repo work with the updated torch version, I would need to rewrite some of the python-scripts, correct?"

Copilot then said the reason might be this:

"Model serialization mismatch: If the repo loads .pt or .pth files saved with Torch 2.2.0, they may not deserialize cleanly in newer builds.

The file was probably saved using Torch 2.2.0 and CUDA 11.x or 12.1.

• Newer Torch builds (like the CUDA 12.8 nightly that supports RTX 50 series) may fail to load it due to serialization mismatches, missing ops, or tensor device incompatibilities.

• Even if the Python code is version-agnostic, the model file itself is not."

Copilot suggested I re-save the model-file, which would require access to training-data:

"What You Can Do
✅ Option 1: Re-save the Model with a Compatible Torch Version
If you have access to the original training code:

• Install the nightly Torch build with CUDA 12.8.
• Load the model using map_location='cpu'.
• Move it to GPU: model.to('cuda').
• Re-save it: torch.save(model.state_dict(), 'model_cuda128.pt')"

Copilot's final recommendation was this:

"- Watch for official updates: If Tencent releases a new .pt file trained with Torch 2.3+ or CUDA 12.8, you’ll be able to use your RTX 50 GPU without patching."

I hope all this info is somewhat useful :)

Thank you.
jacob.

[Helikaon23s]

Thanks for your interest! Could you please provide more details about the error (PyTorch version for CUDA12.8 and the error log from infer.py)?

I had a "conversation" with Microsoft's Copilot trying to fix the issue myself.

Here's the first traceback I got, which I mentioned to Copilot:

I'm getting this error because my GPU is too new for the python-script, but where do I enter the term "cpu" that it's asking for?

Traceback (most recent call last): File "D:\AI\SongBloom\infer.py", line 103, in main() File "D:\AI\SongBloom\infer.py", line 77, in main model = SongBloom_Sampler.build_from_trainer(cfg, strict=False, dtype=dtype, device=device) File "D:\AI\SongBloom\SongBloom\models\songbloom\songbloom_pl.py", line 85, in build_from_trainer model_light = SongBloom_PL(cfg) File "D:\AI\SongBloom\SongBloom\models\songbloom\songbloom_pl.py", line 34, in init self.vae = StableVAE(**cfg.vae).eval() File "D:\AI\SongBloom\SongBloom\models\vae_frontend_init_.py", line 53, in init self.vae.load_state_dict(torch.load(vae_ckpt)['state_dict']) File "D:\AI\SongBloom\venvPython310\lib\site-packages\torch\serialization.py", line 1026, in load return _load(opened_zipfile, File "D:\AI\SongBloom\venvPython310\lib\site-packages\torch\serialization.py", line 1438, in _load result = unpickler.load() File "D:\AI\SongBloom\venvPython310\lib\site-packages\torch\serialization.py", line 1408, in persistent_load typed_storage = load_tensor(dtype, nbytes, key, _maybe_decode_ascii(location)) File "D:\AI\SongBloom\venvPython310\lib\site-packages\torch\serialization.py", line 1382, in load_tensor wrap_storage=restore_location(storage, location), File "D:\AI\SongBloom\venvPython310\lib\site-packages\torch\serialization.py", line 391, in default_restore_location result = fn(storage, location) File "D:\AI\SongBloom\venvPython310\lib\site-packages\torch\serialization.py", line 266, in _cuda_deserialize device = validate_cuda_device(location) File "D:\AI\SongBloom\venvPython310\lib\site-packages\torch\serialization.py", line 250, in validate_cuda_device raise RuntimeError('Attempting to deserialize object on a CUDA ' RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.

Copilot then replied:

"You're absolutely right — this error is happening because the model checkpoint you're loading was saved with CUDA (GPU) tensors, but your environment doesn't have a compatible GPU or CUDA setup. To fix this, you need to explicitly tell PyTorch to load the model on the CPU."

I then replied:

"That worked, thanks. The thing is the repo requires torch v2.2 and I'm not sure if there is a torch-version of that that supports the RTX50 series GPU, which is the GPU I'm using. I did try to install torch for cuda 12.8 with a link to pytorch.org but that version of torch didn't seem to work. Am I left to CPU only mode do you think?"

Copilot then replied:

"You're not necessarily stuck with CPU-only mode — but official Torch 2.2.0 builds don't yet support the RTX 50 series (Blackwell architecture) or CUDA 12.8. You’ll need a workaround like a custom build or nightly version."

The torch I tried to install is this:

pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

Which comes out as torch 2.9.x

I also tried installing Cuda toolkit 12.6, but that doesn't support the RTX50-series GPU though it does support torch 2.2 (This only works up to the RTX40 series GPU)

Toolkit 12.8 is the lowest for the RTX50-series, and torch 2.2 is not available there.

I then wrote to Copilot:

"The repo does run on CPU, but not on the nightly 12.8 torch version (Which I am using with other repos where it does give Cuda support). So to make the repo work with the updated torch version, I would need to rewrite some of the python-scripts, correct?"

Copilot then said the reason might be this:

"Model serialization mismatch: If the repo loads .pt or .pth files saved with Torch 2.2.0, they may not deserialize cleanly in newer builds.

The file was probably saved using Torch 2.2.0 and CUDA 11.x or 12.1.

• Newer Torch builds (like the CUDA 12.8 nightly that supports RTX 50 series) may fail to load it due to serialization mismatches, missing ops, or tensor device incompatibilities.

• Even if the Python code is version-agnostic, the model file itself is not."

Copilot suggested I re-save the model-file, which would require access to training-data:

"What You Can Do ✅ Option 1: Re-save the Model with a Compatible Torch Version If you have access to the original training code:

• Install the nightly Torch build with CUDA 12.8.
• Load the model using map_location='cpu'.
• Move it to GPU: model.to('cuda').
• Re-save it: torch.save(model.state_dict(), 'model_cuda128.pt')"

Copilot's final recommendation was this:

"- Watch for official updates: If Tencent releases a new .pt file trained with Torch 2.3+ or CUDA 12.8, you’ll be able to use your RTX 50 GPU without patching."

I hope all this info is somewhat useful :)

Thank you. jacob.

Yo Jacob, don't know if this helps but I haven't got a 5090 to try it on (4090) but I installed the Comfy version of Songbloom into a Comfy git clone . I'm using a venv with Python 3.12 , Pytorch 2.8 and Cuda 12.8 (it works). https://github.com/fredconex/ComfyUI-SongBloom

[Helikaon23s]

git clone https://github.com/comfyanonymous/ComfyUI
cd comfyui
python.exe -m venv venv
venv\Scripts\activate.bat
pip install torch==2.8.0 torchvision torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt
python.exe -m pip install --upgrade pip

cd custom_nodes
git clone https://github.com/Comfy-Org/ComfyUI-Manager

Start Comfy and through manager install -
Songbloom

The nodes.py file has an error in it, so you have to place the models in the comfy installs model folder. If you go the repo and look in the 'pull requests' you can see that someone has input a new nodes.py file that allows you to place the models in a centralised models folder (if you want).

[JELSTUDIO]

git clone https://github.com/comfyanonymous/ComfyUI cd comfyui python.exe -m venv venv venv\Scripts\activate.bat pip install torch==2.8.0 torchvision torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128 pip install -r requirements.txt python.exe -m pip install --upgrade pip

cd custom_nodes git clone https://github.com/Comfy-Org/ComfyUI-Manager

Start Comfy and through manager install - Songbloom

The nodes.py file has an error in it, so you have to place the models in the comfy installs model folder. If you go the repo and look in the 'pull requests' you can see that someone has input a new nodes.py file that allows you to place the models in a centralised models folder (if you want).

I will check out if I can get it running on the GPU via ComfyUI and report back. Thank you very much :)

[Helikaon23s]

git clone https://github.com/comfyanonymous/ComfyUI cd comfyui python.exe -m venv venv venv\Scripts\activate.bat pip install torch==2.8.0 torchvision torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128 pip install -r requirements.txt python.exe -m pip install --upgrade pip
cd custom_nodes git clone https://github.com/Comfy-Org/ComfyUI-Manager
Start Comfy and through manager install - Songbloom
The nodes.py file has an error in it, so you have to place the models in the comfy installs model folder. If you go the repo and look in the 'pull requests' you can see that someone has input a new nodes.py file that allows you to place the models in a centralised models folder (if you want).

I will check out if I can get it running on the GPU via ComfyUI and report back. Thank you very much :)

You're welcome, I'm away tomorrow (drinking/concert) but if I can give any more help, please let me know. One other thing is that I tried to get it running on another older copy of Comfy with the same setup but it wouldn't work (must be all the nodes and their dependencies that I had installed) but the new one made by this did work. I couldn't get flash to work so it runs at about 12-13its/s.

[JELSTUDIO]

git clone https://github.com/comfyanonymous/ComfyUI cd comfyui python.exe -m venv venv venv\Scripts\activate.bat pip install torch==2.8.0 torchvision torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128 pip install -r requirements.txt python.exe -m pip install --upgrade pip
cd custom_nodes git clone https://github.com/Comfy-Org/ComfyUI-Manager
Start Comfy and through manager install - Songbloom
The nodes.py file has an error in it, so you have to place the models in the comfy installs model folder. If you go the repo and look in the 'pull requests' you can see that someone has input a new nodes.py file that allows you to place the models in a centralised models folder (if you want).

I will check out if I can get it running on the GPU via ComfyUI and report back. Thank you very much :)

You're welcome, I'm away tomorrow (drinking/concert) but if I can give any more help, please let me know. One other thing is that I tried to get it running on another older copy of Comfy with the same setup but it wouldn't work (must be all the nodes and their dependencies that I had installed) but the new one made by this did work. I couldn't get flash to work so it runs at about 12-13its/s.

I hope you had a great concert :)

You method works :)

I tried converting the new 240s model from .pt to .safetenser using fredconex' conversion-script (The one on the hugginface where the ComfyUI model is also located)

The conversion worked, but even though the ComfyUI node would load the model it apparently wasn't compatible (From what I could understand it lacks a VAE. It output a huge list of errors, and VAE was a term that was mentioned repeatedly)

Anyway, the 150s model worked and ComfyUI did create music :)

I then went ahead and tried installing the default SongBloom with your suggested python-version (3.12) and torch-version (2.8) and that worked too :)

Unfortunately I ran out of Vram during this test with the 240s model (It kept running, but started swapping between Vram and system-ram, so the speed dropped to near CPU-inference speed)

So it seems your solution is the magic piece for Songbloom at the moment (And apparently good for both the RTX40-series, which you mention you have, and the RTX50 series, which I have. I have a 5080, so 16 GB Vram, and that got maxed out during the test with the 240s model in the default SongBloom)

But the 150s model does now run on a RTX50-series GPU via ComfyUI :)

Perhaps fredconex is able to create a better conversion of the 240s.pt model, but I guess we will have to wait and see if he does that.

Thanks again Grey3016 :)

[rklugseder]

[Feature/Compatibility] RTX 5090 (Blackwell) Support - PyTorch 2.7.0+cu128

Problem Description

SongBloom does not run out-of-the-box on NVIDIA RTX 5090 (Blackwell architecture, compute capability sm_120) because the current requirements.txt specifies PyTorch 2.2.0, which only supports compute capabilities up to sm_90 (Ada Lovelace).

Error encountered:

RuntimeError: CUDA error: no kernel image is available for execution on the device
NVIDIA GeForce RTX 5090 with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_70 sm_75 sm_80 sm_86 sm_90.

Environment

• GPU: NVIDIA GeForce RTX 5090 (Blackwell, sm_120)
• OS: WSL2 Ubuntu (Linux 6.6.87.2-microsoft-standard-WSL2)
• NVIDIA Driver: 591.59 (CUDA 13.1)
• Model: songbloom_full_240s

Solution / Workaround

PyTorch 2.7.0 with CUDA 12.8 (cu128) adds native support for sm_120 compute capability. The following configuration works successfully:

# Create environment with Python 3.12
conda create -n SongBloom_240 python==3.12
conda activate SongBloom_240
Install PyTorch 2.7.0+cu128 (supports RTX 5090 sm_120)
pip install torch==2.7.0+cu128 torchvision==0.22.0+cu128 torchaudio==2.7.0+cu128 --index-url https://download.pytorch.org/whl/cu128
Install remaining dependencies (skip torch/torchaudio from requirements.txt)
pip install huggingface-hub==0.24.6 

jieba-fast==0.53 

pypinyin==0.51.0 

cn2an==0.5.22 

wordsegment==1.3.1 

g2p-en==2.1.0 

lightning==2.2.1 

nltk==3.8.1 

omegaconf==2.2.0 

transformers==4.44.1 

einops==0.8.0 

spacy==3.7.4 

num2words==0.5.13 

descript-audio-codec==1.0.0 

vector_quantize_pytorch==1.14.8
Fix numpy version for spacy/thinc compatibility (requires numpy<2.0)
pip install "numpy<2.0" --force-reinstall

Compatibility Matrix

PyTorch Version	CUDA Version	Max Compute Capability	RTX 5090 Support
2.2.0	11.8	sm_90	No
2.6.0	12.4	sm_100	No
2.7.0+cu128	12.8	sm_120	Yes

Verified Working Configuration

torch==2.7.0+cu128
torchaudio==2.7.0+cu128
torchvision==0.22.0+cu128
python==3.12
numpy==1.26.4

Test Results

Inference successful on RTX 5090:

• Model: songbloom_full_240s
• Generated audio: 216 seconds stereo @ 48kHz
• Performance: ~20 iterations/second
• Precision: bfloat16 (reduced VRAM usage)

Suggested Updates

1. Update README.md to note RTX 5090 compatibility with PyTorch 2.7.0+cu128
2. Create alternative requirements file (e.g., requirements_rt50series.txt) with updated PyTorch versions
3. Document in troubleshooting section for users with RTX 50-series GPUs