VITS how to add new languages

[saibharani]

I have a VITS model trained in English, French and Hindi languages and I want to add extra languages. can anyone give instruction about how to add languages?

[erogol]

1. Run python TTS/tts/utils/text/phonemizers/__init__.py to see what languages are supported by the current set of phonemizer.
   You will see something like

{'lb': 'gruut', 'ru-ru': 'gruut', 'en-gb': 'espeak', 'sw': 'espeak', 'de-de': 'gruut', 'ar': 'espeak', 'sv-se': 'gruut', 'fa': 'espeak', 'nl': 'espeak', 'fr-fr': 'espeak', 'cs-cz': 'gruut', 'en-us': 'espeak', 'it-it': 'gruut', 'zh-cn': 'zh_cn_phonemizer', 'es-es': 'gruut', 'pt': 'espeak', 'af': 'espeak', 'am': 'espeak', 'an': 'espeak', 'as': 'espeak', 'az': 'espeak', 'ba': 'espeak', 'bg': 'espeak', 'bn': 'espeak', 'bpy': 'espeak', 'bs': 'espeak', 'ca': 'espeak', 'cmn': 'espeak', 'cs': 'espeak', 'cy': 'espeak', 'da': 'espeak', 'de': 'espeak', 'el': 'espeak', 'en-029': 'espeak', 'en-gb-scotland': 'espeak', 'en-gb-x-gbclan': 'espeak', 'en-gb-x-gbcwmd': 'espeak', 'en-gb-x-rp': 'espeak', 'eo': 'espeak', 'es': 'espeak', 'es-419': 'espeak', 'et': 'espeak', 'eu': 'espeak', 'fa-latn': 'espeak', 'fi': 'espeak', 'fr-be': 'espeak', 'fr-ch': 'espeak', 'ga': 'espeak', 'gd': 'espeak', 'gn': 'espeak', 'grc': 'espeak', 'gu': 'espeak', 'hak': 'espeak', 'hi': 'espeak', 'hr': 'espeak', 'ht': 'espeak', 'hu': 'espeak', 'hy': 'espeak', 'hyw': 'espeak', 'ia': 'espeak', 'id': 'espeak', 'is': 'espeak', 'it': 'espeak', 'ja': 'espeak', 'jbo': 'espeak', 'ka': 'espeak', 'kk': 'espeak', 'kl': 'espeak', 'kn': 'espeak', 'ko': 'espeak', 'kok': 'espeak', 'ku': 'espeak', 'ky': 'espeak', 'la': 'espeak', 'lfn': 'espeak', 'lt': 'espeak', 'lv': 'espeak', 'mi': 'espeak', 'mk': 'espeak', 'ml': 'espeak', 'mr': 'espeak', 'ms': 'espeak', 'mt': 'espeak', 'my': 'espeak', 'nb': 'espeak', 'nci': 'espeak', 'ne': 'espeak', 'om': 'espeak', 'or': 'espeak', 'pa': 'espeak', 'pap': 'espeak', 'pl': 'espeak', 'pt-br': 'espeak', 'py': 'espeak', 'quc': 'espeak', 'ro': 'espeak', 'ru': 'espeak', 'ru-lv': 'espeak', 'sd': 'espeak', 'shn': 'espeak', 'si': 'espeak', 'sk': 'espeak', 'sl': 'espeak', 'sq': 'espeak', 'sr': 'espeak', 'sv': 'espeak', 'ta': 'espeak', 'te': 'espeak', 'tn': 'espeak', 'tr': 'espeak', 'tt': 'espeak', 'ur': 'espeak', 'uz': 'espeak', 'vi': 'espeak', 'vi-vn-x-central': 'espeak', 'vi-vn-x-south': 'espeak', 'yue': 'espeak', 'en': 'espeak', 'ja-jp': 'ja_jp_phonemizer'}

2. If you target language is not in the list, you need to write your own phonemizer under TS/tts/utils/text/phonemizers. If it is the case let me know. Or you can simply train with the graphemes.

3. Assuming you have your language above, set these in the model config accordingly (I assume english)

use_phonemes=True,
phonemizer="espeak",
phoneme_language="en",

4. If you use graphemes,

use_phonemes=False,

5. If you have a different alphabet. Set the CharactersConfig

char_config = CharactersConfig(...)
config = ModelConfig(...
   characters=char_config
   ...
)

Briefly this is it. If you're stuck somewhere check the source code, example recipes. If all fails, post here.

[francois-vz]

Thanks, great answer! :)

[Divesh-M]

HI @erogol , am working on this VITS TTS model, i have to train telugu language which is not present in the phonemizer list, can you please guide here. How to create our own phonemizer and train the model for specific language.

Thanks in advance.

[Rakshitha-Ummadisetti]

Hi @Divesh-M . I've recently trained VITS with telugu dataset.
There are a couple of changes you need to make

1. In symbols.py include telugu symbols all matras and alphabet
2. Espeak-ng supports telugu. Make necessary changes to run the preprocess.py
3. If you are using single GPU please make changes in train.py to make it work for telugu dataset

If you have any particular issues do email 21211a7259@bvrit.ac.in

[thenameisAnurag]

I want you help as i want to train the model on multiple Indian language maily (hindi , tamil , telugu) and also i want to do code switching in multiple indian languagues means HINDI + ENGLISH =HINGLISH
TAMIL+ENGLISH =TANGLISH

how should be my approach please guide

[Divesh-M]

@saibharani did you try to train VITS model with hindi language, if yes can you please instruct how to train it.

Thanks in advance

[bharani-y]

I used the original VITS implementation here as it supports espeak better and the number of modifications required is less, change the dataloader to fit your data structure and use espeak-ng to convert hindi text to phoneme, you can select between LJspeech (Single speaker) config or VCTK (Multi speaker) config based on your dataset

[Divesh-M]

I addded hindi phonemizer in code, but when i trying to train the model. it throws error "[!] train_step() retuned None outputs. Skipping training step."

Data Loader Initialization config :

DataLoader initialization
| > Tokenizer:
| > add_blank: False
| > use_eos_bos: False
| > use_phonemes: True
| > phonemizer:
| > phoneme language: hi
| > phoneme backend: hi_phonemizer
| > Number of instances : 531
| > Preprocessing samples
| > Max text length: 288
| > Min text length: 19
| > Avg text length: 118.19585687382298
|
| > Max audio length: 514113.0
| > Min audio length: 47799.0
| > Avg audio length: 223526.23540489643
| > Num. instances discarded samples: 0
| > Batch group size: 0.