[reedrya]

What does this PR do?

This PR updates the WavLM model card to comply with the format introduced in #36979.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

Who can review?

@stevhliu

Notes

• I did not include the AttentionMaskVisualizer section since I'm unfamiliar. Please advise if that should be added.

[stevhliu]

Thanks for your contribution!

[stevhliu]

Suggested change

[WavLM](https://huggingface.co/papers/2110.13900) is a self-supervised speech representation model from Microsoft designed to work across the “full stack” of speech tasks, from automatic speech recognition (ASR) to speaker diarization and audio event detection. It builds on HuBERT’s masked prediction approach but introduces denoising and data augmentation to make the learned representations more robust in noisy and multi-speaker conditions.

[WavLM](https://huggingface.co/papers/2110.13900) is a self-supervised speech representation model designed to work across the “full stack” of speech tasks, from automatic speech recognition (ASR) to speaker diarization and audio event detection. It builds on [HuBERTs](./hubert) masked prediction approach. It introduces gated relative position bias for better recognition accuracy and an unsupervised utterance-mixing strategy to improve speaker discrimination.

[stevhliu]

Suggested change

	You can find all the original WavLM checkpoints under the [WavLM](https://huggingface.co/models?other=wavlm) collection.
	You can find all the original WavLM checkpoints under the [Microsoft](https://huggingface.co/microsoft/models?search=wavlm) organization.

[stevhliu]

Suggested change

	The example below demonstrates how to extract audio features with [`Pipeline`] or the [`AutoModel`] class.
	The example below demonstrates how to automatically transcribe speech into text with [`Pipeline`] or the [`AutoModel`] class.

[stevhliu]

import torch
from transformers import pipeline

pipeline = pipeline(
    task="automatic-speech-recognition",
    model="patrickvonplaten/wavlm-libri-clean-100h-base-plus",
    torch_dtype=torch.float16,
    device=0
)

pipeline("https://huggingface.co/datasets/Narsil/asr_dummy/resolve/main/1.flac")

[stevhliu]

from transformers import AutoProcessor, AutoModelForCTC
from datasets import load_dataset
import torch

dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation")
dataset = dataset.sort("id")
sampling_rate = dataset.features["audio"].sampling_rate

processor = AutoProcessor.from_pretrained("patrickvonplaten/wavlm-libri-clean-100h-base-plus")
model = AutoModelForCTC.from_pretrained("patrickvonplaten/wavlm-libri-clean-100h-base-plus", torch_dtype=torch.float16)

# audio file is decoded on the fly
inputs = processor(dataset[0]["audio"]["array"], sampling_rate=sampling_rate, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
predicted_ids = torch.argmax(logits, dim=-1)

# transcribe speech
transcription = processor.batch_decode(predicted_ids)
transcription[0]

[stevhliu]

The model is not that large so we can remove the Quantization section

[stevhliu]

Suggested change

	- WavLM processes raw 16kHz audio waveforms provided as 1D float arrays. Use `Wav2Vec2Processor` for preprocessing.
	- WavLM processes raw 16kHz audio waveforms provided as 1D float arrays. Use [`Wav2Vec2Processor`] for preprocessing.

[stevhliu]

Suggested change

	- For CTC-based fine-tuning, model outputs should be decoded with `Wav2Vec2CTCTokenizer`.
	- For CTC-based fine-tuning, model outputs should be decoded with [`Wav2Vec2CTCTokenizer`].

[stevhliu]

Suggested change

	- The model works particularly well for tasks like speaker verification, identification, and diarization.
	## Resources
	- [Audio classification task guide](https://huggingface.co/docs/transformers/en/tasks/audio_classification)
	- [Automatic speech recognition task guide](https://huggingface.co/docs/transformers/en/tasks/asr)

[ArthurZucker]

Thanks for the PR, and sorry it didn't make it in! This branch is based on v4, and v5 is very different, so rather than rebasing we'd love to invite you to open a new PR against main using our latest tools and good practices:

• Transformers tenets
• mlinter and the modeling rules it enforces (make typing)
• Modular transformers
• Contributing guide

Closing this one in the meantime, thanks again!