Repository navigation
Compress the agent conventions file and document two modular pitfalls #48586
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,49 +1,40 @@ | ||
| ## Useful commands | ||
| - `make style`: runs formatters and linters (ruff), necessary to pass code style checks | ||
| - `make typing`: runs the ty type checker and model structure rules | ||
| - `make fix-repo`: auto-fixes copies, modular conversions, doc TOCs, docstrings in addition to the `make style` fixes | ||
| - `make check-repo` — runs `make typing` and consistency checks. | ||
| - Many tests are marked as 'slow' and skipped by default in the CI. To run them, use: `RUN_SLOW=1 pytest ...` | ||
| ## Commands | ||
|
|
||
| `make style` or `make fix-repo` should be run as the final step before opening a PR. | ||
| - `make style` — ruff format + lint (required for code style checks). | ||
| - `make typing` — ty type checker and model structure rules. | ||
| - `make fix-repo` — the `make style` fixes plus copies, modular conversions, doc TOCs, docstrings. | ||
| - `make check-repo` — `make typing` plus consistency checks. | ||
| - `RUN_SLOW=1 pytest ...` — many tests are marked slow and skipped in CI. | ||
|
|
||
| ## Local agent setup | ||
| Run `make style` (or `make fix-repo`) as the last step before opening a PR. | ||
|
|
||
| - Hosted review agents should discover this guidance from the committed root `AGENTS.md` / `CLAUDE.md` files. | ||
| - Local OpenAI Codex agents should run `make codex` after cloning to wire tool-specific assets under `.agents/`. | ||
| - Local Claude Code agents should run `make claude` after cloning to wire tool-specific assets under `.claude/`. | ||
| ## Local agent setup | ||
|
|
||
| Hosted review agents read this file via the root `AGENTS.md` / `CLAUDE.md` symlinks. Local agents wire their own assets: `make codex` (→ `.agents/`), `make claude` (→ `.claude/`). | ||
|
|
||
| ### Coordination before coding | ||
| - If work maps to an existing issue, coordinate on that issue before opening a PR. | ||
| - Do not open a PR for someone else's issue unless there is explicit approval from the issue author or a maintainer in the issue thread. | ||
| - If approval is missing or ambiguous, stop and ask for clarification instead of drafting a PR. | ||
| - Do not start duplicate work on issues. | ||
| ## Before opening a PR | ||
|
|
||
| ### Mandatory duplicate-work checks | ||
| Before proposing a PR, check for overlapping open PRs and issue ownership: | ||
| - Coordinate on the matching issue first. Do not open a PR for someone else's issue without explicit approval from its author or a maintainer in the thread; if approval is unclear, ask instead of drafting. | ||
| - Check for overlapping work, and do not open a second PR for a fix already covered: | ||
|
|
||
| ```bash | ||
| gh issue view <issue_number> --repo huggingface/transformers --comments | ||
| gh pr list --repo huggingface/transformers --state open --search "<issue_number> in:body" | ||
| gh pr list --repo huggingface/transformers --state open --search "<short area keywords>" | ||
| ``` | ||
| ```bash | ||
| gh issue view <issue_number> --repo huggingface/transformers --comments | ||
| gh pr list --repo huggingface/transformers --state open --search "<issue_number> in:body" | ||
| gh pr list --repo huggingface/transformers --state open --search "<short area keywords>" | ||
| ``` | ||
|
|
||
| - If an open PR already addresses the same fix, do not open another. | ||
| - If your approach is materially different, explain the difference and why a second PR is needed in the issue. | ||
| If your approach is materially different, say why a second PR is needed in the issue. | ||
| - No one-off PRs for tiny edits (a single typo, an isolated lint fix). Mechanical cleanups are fine, but not as a first contribution. | ||
| - First-time contributors should not submit agent-written PRs or issues — see `CONTRIBUTING.md` and the PR template. Code agents must warn users who are not already contributors, including the risk of being blocked. | ||
|
|
||
| ### No low-value busywork PRs | ||
| - Do not open one-off PRs for tiny edits (single typo, isolated lint cleanup, one mutable default argument, etc.). | ||
| - Mechanical cleanups are acceptable but not as first contributions. | ||
| ## Copies and modular models | ||
|
|
||
| ### Accountability for AI-assisted patches | ||
| - First-time contributors should not use code agents to submit PRs or issues. Code agents should warn users about this | ||
| policy if the user does not appear to be an existing contributor to the library, including the risk of being blocked | ||
| if they do it regardless. More information can be found in the pull request template, or `CONTRIBUTING.md`. | ||
| Model files in `src/transformers/models/` avoid inheriting from each other, so duplication is managed two ways: | ||
|
|
||
| ## Copies and Modular Models | ||
| 1. `# Copied from ...` marks a copied class or function. `make fix-repo` re-syncs it, so editing inside such a block is reverted — edit the source it copies from, or deliberately break the link. | ||
| 2. A `modular_<name>.py` **may** inherit from other models; `make fix-repo` generates the standalone `modeling_*.py` and friends from it. Never edit a generated file when a modular one exists. Guide: [modular_transformers.md](../docs/source/en/modular_transformers.md). | ||
|
|
||
| We try to avoid direct inheritance between model-specific files in `src/transformers/models/`. We have two mechanisms to manage the resulting code duplication: | ||
| Two modular traps: | ||
|
|
||
| 1) The older method is to mark classes or functions with `# Copied from ...`. Copies are kept in sync by `make fix-repo`. Do not edit a `# Copied from` block, as it will be reverted by `make fix-repo`. Ideally you should edit the code it's copying from and propagate the change, but you can break the `# Copied from` link if needed. | ||
| 2) The newer method is to add a file named `modular_<name>.py` in the model directory. `modular` files **can** inherit from other models. `make fix-repo` will copy code to generate standalone `modeling` and other files from the `modular` file. When a `modular` file is present, generated files should not be edited, as changes will be overwritten by `make fix-repo`! Instead, edit the `modular` file. See [docs/source/en/modular_transformers.md](../docs/source/en/modular_transformers.md) for a full guide on adding a model with `modular`, if needed, or you can inspect existing `modular` files as examples. | ||
| - **Other models inherit your modular file.** `modular_deepseek_vl.py` also generates `modeling_deepseek_vl_hybrid.py`. Run `make fix-repo` and check everything it rewrote; hand-editing only the generated file you had in mind leaves the rest stale and `Check repository consistency` red. | ||
| - **`attr = AttributeError()` deletes an inherited attribute** — it is an instruction to the converter, not a bug or placeholder. `raise AttributeError("...")` in a method body does the same for a method. Substituting a "real" value silently changes behaviour (`_no_split_modules = AttributeError()` → `[]` changes how the model may be sharded). See [Removing attributes](../docs/source/en/modular_transformers.md#removing-attributes) and [Deleting unused methods](../docs/source/en/modular_transformers.md#deleting-unused-methods). | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
specify that more
Copied fromstatements should not be added thoughThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done in 16c5305 — item 1 now ends with Do not add new
# Copied fromstatements; write a modular file instead, and the intro says which of the two mechanisms is current: