Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions src/transformers/models/squeezebert/modeling_squeezebert.py
Original file line number Diff line number Diff line change
Expand Up @@ -534,6 +534,27 @@ def forward(
Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ...,
config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the
loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`

Example:

```python
>>> from transformers import AutoTokenizer, SqueezeBertForMaskedLM
>>> import torch

>>> tokenizer = AutoTokenizer.from_pretrained("squeezebert/squeezebert-uncased")
>>> model = SqueezeBertForMaskedLM.from_pretrained("squeezebert/squeezebert-uncased")

>>> inputs = tokenizer("The capital of France is [MASK].", return_tensors="pt")
>>> with torch.no_grad():
... logits = model(**inputs).logits

>>> # retrieve index of [MASK]
>>> mask_token_index = (inputs.input_ids == tokenizer.mask_token_id)[0].nonzero(as_tuple=True)[0]

>>> predicted_token_id = logits[0, mask_token_index].argmax(axis=-1)

Copilot AI Dec 2, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the doctest example, logits is a PyTorch tensor, but argmax(axis=-1) uses the NumPy keyword. In PyTorch, the correct keyword is dim. Using axis will raise an error. Suggest updating to argmax(dim=-1) to get the vocabulary index of the predicted token(s).

Suggested change
>>> predicted_token_id = logits[0, mask_token_index].argmax(axis=-1)
>>> predicted_token_id = logits[0, mask_token_index].argmax(dim=-1)

Copilot uses AI. Check for mistakes.
>>> tokenizer.decode(predicted_token_id) # doctest: +SKIP
'paris'
```
"""
return_dict = return_dict if return_dict is not None else self.config.use_return_dict

Expand Down
1 change: 1 addition & 0 deletions utils/slow_documentation_tests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@ src/transformers/models/idefics2/modeling_idefics2.py
src/transformers/models/kosmos2/modeling_kosmos2.py
src/transformers/models/musicgen_melody/modeling_musicgen_melody.py
src/transformers/models/musicgen_melody/processing_musicgen_melody.py
src/transformers/models/squeezebert/modeling_squeezebert.py
Loading