Repository navigation
Closed
Labels
Models: XTTSbugSomething isn't workingSomething isn't workinggood first issueGood for newcomersGood for newcomers
Description
Describe the bug
huggingface/transformers#30624 broke the XTTS streaming code (https://github.com/idiap/coqui-ai-TTS/blob/df088e99dfda8976c47235626d3afb7d7d70fea2/TTS/tts/layers/xtts/stream_generator.py) as reported in huggingface/transformers#31040. The streaming code needs to be updated accordingly.
To Reproduce
https://coqui-tts.readthedocs.io/en/latest/models/xtts.html#streaming-manually
Expected behavior
No response
Logs
Loading model...
DEBUG:fsspec.local:open file: ~/.local/share/tts/tts_models--multilingual--multi-dataset--xtts_v2/model.pth
Computing speaker latents...
Inference...
~/applications/miniconda3/envs/coqui-3.12/lib/python3.12/site-packages/TTS/tts/layers/xtts/stream_generator.py:138: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation)
warnings.warn(
Traceback (most recent call last):
File "~/projects/testing/xtts.py", line 33, in <module>
for i, chunk in enumerate(chunks):
File "~/applications/miniconda3/envs/coqui-3.12/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 35, in generator_context
response = gen.send(None)
^^^^^^^^^^^^^^
File "~/applications/miniconda3/envs/coqui-3.12/lib/python3.12/site-packages/TTS/tts/models/xtts.py", line 657, in inference_stream
gpt_generator = self.gpt.get_generator(
^^^^^^^^^^^^^^^^^^^^^^^
File "~/applications/miniconda3/envs/coqui-3.12/lib/python3.12/site-packages/TTS/tts/layers/xtts/gpt.py", line 602, in get_generator
return self.gpt_inference.generate_stream(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/applications/miniconda3/envs/coqui-3.12/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "~/applications/miniconda3/envs/coqui-3.12/lib/python3.12/site-packages/TTS/tts/layers/xtts/stream_generator.py", line 186, in generate
model_kwargs["attention_mask"] = self._prepare_attention_mask_for_generation(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/applications/miniconda3/envs/coqui-3.12/lib/python3.12/site-packages/transformers/generation/utils.py", line 473, in _prepare_attention_mask_for_generation
torch.isin(elements=inputs, test_elements=pad_token_id).any()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: isin() received an invalid combination of arguments - got (test_elements=int, elements=Tensor, ), but expected one of:
* (Tensor elements, Tensor test_elements, *, bool assume_unique, bool invert, Tensor out)
* (Number element, Tensor test_elements, *, bool assume_unique, bool invert, Tensor out)
* (Tensor elements, Number test_element, *, bool assume_unique, bool invert, Tensor out)Environment
{
"CUDA": {
"GPU": [],
"available": false,
"version": "12.1"
},
"Packages": {
"PyTorch_debug": false,
"PyTorch_version": "2.3.0+cu121",
"TTS": "0.24.0",
"numpy": "1.26.4"
},
"System": {
"OS": "Linux",
"architecture": [
"64bit",
"ELF"
],
"processor": "x86_64",
"python": "3.12.3",
"version": "#35~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue May 7 09:00:52 UTC 2"
}
}Additional context
No response
Reacted by sachin-seisei
Metadata
Metadata
Assignees
Labels
Models: XTTSbugSomething isn't workingSomething isn't workinggood first issueGood for newcomersGood for newcomers
Activity
build: set upper version limit for transformers
eginhard commentedon May 29, 2024
coqui-ttsversion 0.24.1 limits transformers to lower versions for a temporary fix until someone properly updates the streaming code.pseudotensor commentedon May 29, 2024
https://github.com/h2oai/h2ogpt/blob/main/docs/xtt.patch
eginhard commentedon May 30, 2024
@pseudotensor Cool, thanks! Do you want to send a PR?
refactor(stream_generator): update special tokens for transformers>=4…
refactor(stream_generator): update special tokens for transformers>=4…
refactor(stream_generator): update special tokens for transformers>=4…