Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,11 +39,23 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),

### Changed

- `generation_*.py` files in a model directory are now discovered, so rules see them for the first time. There are
ten in transformers (`generation_whisper.py`, `generation_parakeet.py`, `generation_csm.py`, ...) and they hold
model implementation code, but no pattern in `MODELING_PATTERNS` matched them. `TRF009` gained the matching prefix
and reports the two real cross-model imports the widening exposes: `nemotron3_5_asr` importing from
`nemotron_asr_streaming`, and that in turn from `parakeet`. A full run over transformers confirms the widening
adds nothing else -- 40 findings before, 42 after, both new ones `TRF009` -- so the dozen rules that gate on AST
content rather than on a file-name prefix are unaffected. Note the pattern also claims a `generation_utils.py`
helper, which matters mainly when linting a standalone model repository. Closes
[#49](https://github.com/huggingface/transformers-mlinter/issues/49).

- Widened `TRF009` on both axes it was missing. It now runs on every file in a model directory --
`configuration_*.py`, `processing_*.py`, `image_processing_*.py`, `video_processing_*.py`,
`feature_extraction_*.py` and `tokenization_*.py` as well as `modeling_*.py` -- since a config that imports
another model's config couples the two models just as tightly as a modeling file that does; `modular_*.py`
stays exempt, and is now the only exemption. It also recognises the public-API form,
stays exempt, while `convert_*.py` and `__init__.py` are out of scope -- a conversion script legitimately
builds a checkpoint out of whatever the original release shipped, and an `__init__.py` alias is the same
coupling already reported on the tokenizer file itself. It also recognises the public-API form,
`from transformers import CLIPTextModelWithProjection`, which reached another model's implementation without
naming its package and so went unreported. Because that form gives only a class name, the owning directory is
recovered from the name and then confirmed against the classes that directory really defines, so a shared class
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@

A standalone linter for [Hugging Face Transformers](https://github.com/huggingface/transformers) model
integration files — `modeling_*.py`, `modular_*.py`, `configuration_*.py`, `processing_*.py`,
`image_processing_*.py`, `video_processing_*.py`, `feature_extraction_*.py` and
`tokenization_*.py` under `src/transformers/models/`, plus `test_tokenization_*.py` under
`image_processing_*.py`, `video_processing_*.py`, `feature_extraction_*.py`,
`tokenization_*.py` and `generation_*.py` under `src/transformers/models/`, plus `test_tokenization_*.py` under
`tests/models/`. It enforces the
structural conventions that keep hundreds of model implementations consistent with each other.

Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ of the library is a few seconds of parsing.
Every file under `src/transformers/models/` matching one of these patterns:

`modeling_*.py`, `modular_*.py`, `configuration_*.py`, `processing_*.py`, `image_processing_*.py`,
`video_processing_*.py`, `feature_extraction_*.py`, `tokenization_*.py`
`video_processing_*.py`, `feature_extraction_*.py`, `tokenization_*.py`, `generation_*.py`

plus `test_tokenization_*.py` under `tests/models/` — the only test files walked, since
[TRF042](rules/trf042.md) is the only rule that reads one.
Expand Down
2 changes: 1 addition & 1 deletion docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ mlinter src/transformers/models/llama tests/models/llama

A directory is searched recursively for model integration files (`modeling_*.py`, `modular_*.py`,
`configuration_*.py`, `processing_*.py`, `image_processing_*.py`, `video_processing_*.py`,
`feature_extraction_*.py`, `tokenization_*.py`, `test_tokenization_*.py`); a file named explicitly is checked as given.
`feature_extraction_*.py`, `tokenization_*.py`, `generation_*.py`, `test_tokenization_*.py`); a file named explicitly is checked as given.
Since the search is recursive and takes several paths at once, the layout does not matter: a GitHub
project that keeps its model files in some directory of its own is checked by naming that directory,
or by naming each one when they are scattered. Rules gate on the file name, so a file named something
Expand Down
1 change: 1 addition & 0 deletions mlinter/mlinter.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@
"processing_*.py",
"feature_extraction_*.py",
"tokenization_*.py",
"generation_*.py",
)
# Test files a rule may target, discovered under TESTS_ROOT rather than MODELS_ROOT. Only tokenization
# tests are walked because TRF042 is the only rule that looks at a test file; `test_modeling_*.py` and
Expand Down
2 changes: 1 addition & 1 deletion mlinter/rules.toml
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ default_enabled = true
allowlist_models = ["dpr", "maskformer", "sam3_video", "vision_text_dual_encoder"]

[rules.TRF009.explanation]
what_it_does = "Checks every file in a model directory -- modeling_*.py, configuration_*.py, processing_*.py, image_processing_*.py, video_processing_*.py, feature_extraction_*.py, tokenization_*.py -- for imports that reach into another model's package. Those are the files that make up a model's shipped implementation. `modular_*.py` is exempt because building on another model's classes is what a modular file is for, and the converter flattens those imports away in the file it generates. The other two file kinds that live in a model directory are out of scope rather than exempt: a `convert_*.py` script is a one-off tool that legitimately builds a checkpoint out of whatever the original release used, and the cross-model aliases in an `__init__.py` are the same coupling already reported on the tokenizer file itself. Three import forms are recognised. The package path, `from transformers.models.other.modeling_other import X` or `import transformers.models.other...`. The relative path, `from ..other.configuration_other import X`, including the longer `from ...models.other.modeling_other import X` that names the models package on the way. And the public API, `from transformers import OtherModel`, which reaches the same implementation without naming its package at all: since only the class name is given, the owning directory is recovered from it (transformers names a model's classes after its directory, `clip` -> `CLIPModel`, with the casing of the prefix left to the author, so the match is made on lowercased, underscore-stripped text and the longest matching directory wins), then confirmed against the classes that directory really defines. That confirmation is what keeps `from transformers import BitsAndBytesConfig` -- a shared class that merely starts like the `bit` directory -- out of the results, and it also means a name the linter cannot resolve is left alone rather than guessed at: outside a transformers checkout there are no directories to match against, so the public-API form goes unchecked there. Two directories are always exempt as import targets, being the shared way in rather than another model's implementation: `auto`, which holds the mappings a composite model resolves its sub-models with, and `timm_wrapper`, the adapter that exposes any timm backbone as a transformers model, so `TimmWrapperConfig` is of a kind with `AutoConfig`. Files inside `auto` are skipped outright, since naming every model's classes is exactly what `configuration_auto.py` and `tokenization_auto.py` are for. So is the model's own directory: `from transformers import UdopConfig` inside models/udop is its own code."
what_it_does = "Checks every file in a model directory -- modeling_*.py, configuration_*.py, processing_*.py, image_processing_*.py, video_processing_*.py, feature_extraction_*.py, tokenization_*.py, generation_*.py -- for imports that reach into another model's package. Those are the files that make up a model's shipped implementation. `modular_*.py` is exempt because building on another model's classes is what a modular file is for, and the converter flattens those imports away in the file it generates. The other two file kinds that live in a model directory are out of scope rather than exempt: a `convert_*.py` script is a one-off tool that legitimately builds a checkpoint out of whatever the original release used, and the cross-model aliases in an `__init__.py` are the same coupling already reported on the tokenizer file itself. Three import forms are recognised. The package path, `from transformers.models.other.modeling_other import X` or `import transformers.models.other...`. The relative path, `from ..other.configuration_other import X`, including the longer `from ...models.other.modeling_other import X` that names the models package on the way. And the public API, `from transformers import OtherModel`, which reaches the same implementation without naming its package at all: since only the class name is given, the owning directory is recovered from it (transformers names a model's classes after its directory, `clip` -> `CLIPModel`, with the casing of the prefix left to the author, so the match is made on lowercased, underscore-stripped text and the longest matching directory wins), then confirmed against the classes that directory really defines. That confirmation is what keeps `from transformers import BitsAndBytesConfig` -- a shared class that merely starts like the `bit` directory -- out of the results, and it also means a name the linter cannot resolve is left alone rather than guessed at: outside a transformers checkout there are no directories to match against, so the public-API form goes unchecked there. Two directories are always exempt as import targets, being the shared way in rather than another model's implementation: `auto`, which holds the mappings a composite model resolves its sub-models with, and `timm_wrapper`, the adapter that exposes any timm backbone as a transformers model, so `TimmWrapperConfig` is of a kind with `AutoConfig`. Files inside `auto` are skipped outright, since naming every model's classes is exactly what `configuration_auto.py` and `tokenization_auto.py` are for. So is the model's own directory: `from transformers import UdopConfig` inside models/udop is its own code."
why_bad = "One model, one definition: a reader of modeling_acme.py should find the whole of Acme's behavior in Acme's own files, and a change to Llama should not be able to silently change Acme. Cross-model imports break both, and they do it in every file kind, not just the modeling one -- a configuration_*.py that imports another model's config, or a processing_*.py that subclasses another model's processor, couples the two models just as tightly. Composite models reach their sub-configs through AutoConfig and CONFIG_MAPPING instead, and a modular file is the supported way to build on another model's code."
diff = '''
-from transformers.models.llama.modeling_llama import LlamaAttention
Expand Down
1 change: 1 addition & 0 deletions mlinter/trf009.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@
"video_processing_",
"feature_extraction_",
"tokenization_",
"generation_",
)

# Model directories every model is meant to reach through rather than around. `auto` holds the mappings
Expand Down
13 changes: 10 additions & 3 deletions tests/test_mlinter.py
Original file line number Diff line number Diff line change
Expand Up @@ -683,7 +683,14 @@ def test_iter_modeling_files_returns_processing_files(self):
models_root = Path(tmpdir)
model_dir = models_root / "foo"
model_dir.mkdir()
filenames = ["modeling_foo.py", "processing_foo.py", "image_processing_foo.py", "video_processing_foo.py"]
filenames = [
"modeling_foo.py",
"processing_foo.py",
"image_processing_foo.py",
"video_processing_foo.py",
"tokenization_foo.py",
"generation_foo.py",
]
for name in filenames:
path = model_dir / name
path.write_text("import torch", encoding="utf-8")
Expand Down Expand Up @@ -711,7 +718,7 @@ def _write_standalone_model_repo(root: Path) -> Path:
" super().__init__(config)\n",
encoding="utf-8",
)
(repo / "generation_utils.py").write_text("def generate():\n pass\n", encoding="utf-8")
(repo / "sampling_helpers.py").write_text("def sample():\n pass\n", encoding="utf-8")
return repo

def test_search_paths_discover_model_files_outside_the_transformers_layout(self):
Expand Down Expand Up @@ -748,7 +755,7 @@ def test_changed_only_candidate_honours_search_paths(self):
with tempfile.TemporaryDirectory() as tmp_dir:
repo = self._write_standalone_model_repo(Path(tmp_dir))
self.assertTrue(mlinter._is_modeling_candidate(repo / "modeling_llada.py", [repo]))
self.assertFalse(mlinter._is_modeling_candidate(repo / "generation_utils.py", [repo]))
self.assertFalse(mlinter._is_modeling_candidate(repo / "sampling_helpers.py", [repo]))
self.assertFalse(
mlinter._is_modeling_candidate(Path("src/transformers/models/foo/modeling_foo.py"), [repo])
)
Expand Down
11 changes: 11 additions & 0 deletions tests/test_trf009.py
Original file line number Diff line number Diff line change
Expand Up @@ -183,3 +183,14 @@ def test_trf009_directory_scan_is_cached_per_path_not_per_name(self, _mock):
self.assertIn("`clip`", results[0][0].message)
# The second root's `clip` does not define ClipModel, so nothing is reported there.
self.assertEqual(results[1], [])

@patch("mlinter.trf009._known_model_dirs", return_value={"foo", "parakeet", "auto"})
def test_trf009_flags_cross_model_import_in_generation_file(self, _mock):
# A generation_*.py in a model directory is implementation code like any other: importing
# another model's generation mixin couples the two exactly as tightly.
source = """
from ...models.parakeet.generation_parakeet import ParakeetGenerationMixin
"""
trf009 = self._run(mlinter.TRF009, source, file_name="generation_foo.py")
self.assertEqual(len(trf009), 1)
self.assertIn("imports implementation code from `parakeet`", trf009[0].message)