Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions examples/pytorch/speech-recognition/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,14 +59,23 @@ OMP_NUM_THREADS=1 python run_speech_recognition_ctc ...
If the environment variable is not set, the training script might freeze, *i.e.* see: https://github.com/pytorch/audio/issues/1021#issuecomment-726915239

---
### Loading Common Voice from the Hub (recommended)

Use the `datasets` library to load a pinned Common Voice version from the Hub:
Comment thread
tayo4christ marked this conversation as resolved.
Resolved

```python
from datasets import load_dataset
# pick the right version/lang that exists on the Hub (example: CV 17.0 English)
ds = load_dataset("mozilla-foundation/common_voice_17_0", "en", split="train+validation")


### Single GPU CTC

The following command shows how to fine-tune [XLSR-Wav2Vec2](https://huggingface.co/transformers/main/model_doc/xlsr_wav2vec2.html) on [Common Voice](https://huggingface.co/datasets/common_voice) using a single GPU in half-precision.

```bash
python run_speech_recognition_ctc.py \
--dataset_name="common_voice" \
--dataset_name="mozilla-foundation/common_voice_17_0" \
--model_name_or_path="facebook/wav2vec2-large-xlsr-53" \
--dataset_config_name="tr" \
--output_dir="./wav2vec2-common_voice-tr-demo" \
Expand Down Expand Up @@ -102,7 +111,7 @@ The following command shows how to fine-tune [XLSR-Wav2Vec2](https://huggingface
```bash
torchrun \
--nproc_per_node 8 run_speech_recognition_ctc.py \
--dataset_name="common_voice" \
--dataset_name="mozilla-foundation/common_voice_17_0" \
--model_name_or_path="facebook/wav2vec2-large-xlsr-53" \
--dataset_config_name="tr" \
--output_dir="./wav2vec2-common_voice-tr-demo-dist" \
Expand Down Expand Up @@ -149,7 +158,7 @@ However, the `--shuffle_buffer_size` argument controls how many examples we can
```bash
**torchrun \
--nproc_per_node 4 run_speech_recognition_ctc_streaming.py \
--dataset_name="common_voice" \
--dataset_name="mozilla-foundation/common_voice_17_0" \
--model_name_or_path="facebook/wav2vec2-xls-r-300m" \
--tokenizer_name_or_path="anton-l/wav2vec2-tokenizer-turkish" \
--dataset_config_name="tr" \
Expand Down Expand Up @@ -360,6 +369,10 @@ recognition on one of the [official speech recognition datasets](https://hugging
### Whisper Model
We can load all components of the Whisper model directly from the pretrained checkpoint, including the pretrained model weights, feature extractor and tokenizer. We simply have to specify our fine-tuning dataset and training hyperparameters.

> **Tip:** If you see errors about “dataset scripts are no longer supported”, make sure your
> `--dataset_name` points to a Hub dataset like `mozilla-foundation/common_voice_XX_X`
> (e.g., `mozilla-foundation/common_voice_17_0`) rather than a local script.
Comment thread
tayo4christ marked this conversation as resolved.
Resolved

#### Single GPU Whisper Training
The following example shows how to fine-tune the [Whisper small](https://huggingface.co/openai/whisper-small) checkpoint on the Hindi subset of [Common Voice 11](https://huggingface.co/datasets/mozilla-foundation/common_voice_11_0) using a single GPU device in half-precision:
```bash
Expand Down
Loading