Repository navigation

[`Mistral Tokenizers`] Fix tokenizer detection - #42389

Merged

ArthurZucker merged 21 commits into

huggingface:mainfrom

vasqu:fix-mistral-detection

Nov 25, 2025

Merged

[`Mistral Tokenizers`] Fix tokenizer detection#42389
ArthurZucker merged 21 commits into
huggingface:mainfrom
vasqu:fix-mistral-detection

Commits on Nov 25, 2025
- fix
  vasqu
  committed
  4c147c9
- sanity check
  vasqu
  committed
  5606702
- style
  vasqu
  committed
  bb9a461
- comments
  vasqu
  committed
  4bdb0df
- make it v5 explicit
  vasqu
  committed
  7ffbe21
- make explicit fixes possible in local tokenizers
  vasqu
  committed
  0e509d3
- remove hub usage on local
  vasqu
  committed
  bc0c2f0
- fix
  vasqu
  committed
  9aaf2c0
- extend test for no config case
  vasqu
  committed
  3857739
- move mistral patch outside to separate fn
  vasqu
  committed
  f56961f
- fix local path only
  ArthurZucker
  committed
  1bebb90
- add a tes
  ArthurZucker
  committed
  15bf24f
- make sure test does not pass before this PR
  ArthurZucker
  committed
  d3970fd
- styling
  ArthurZucker
  committed
  2a1c8e3
- make sure it exists
  ArthurZucker
  committed
  ee332ad
- fix
  ArthurZucker
  committed
  82cd35d
- fix
  vasqu
  committed
  5d8546f
- rename
  vasqu
  committed
  ede0e3f
- up
  ArthurZucker
  committed
  1f41bc5
- Merge branch 'fix-mistral-detection' of github.com:vasqu/transformers into fix-mistral-detection
  ArthurZucker
  committed
  a1cabf3
- last nit i hope lord
  ArthurZucker
  committed
  4dda987