Repository navigation
Compress every prompt block, and guard the instruction against truncation - #115
Merged
Merged
Conversation
#114 pushed the `output_mismatch` instruction to 8,821 chars. serge's `build_task_user_prompt` head-truncates the instruction at MAX_INSTRUCTION_CHARS (8,000 at the time), so the 821 chars that fell off the end were the two bullets that PR existed to add — the guidance was written, merged, and never reached the model. serge raises the cap to 12,000; this is the half that belongs here, where the blocks are written and where the next one will grow. Every category's `build_instruction` output is checked against that number, with the failure message saying what truncation would cost. Sizes today: output_mismatch 8,821 · cluster 4,753 · crash 4,688 · load_error 3,217 · import_or_config 3,121 · trunk alone 2,641. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every one of these strings is sent to the model; the `#` comments around them are not, so the rationale and the dated precedents stay where they are and only the delivered text shrinks. Same rules, same thresholds, same PR numbers and examples — the 112 phrases the test module pins are all still there. block before after _INSTRUCTION 2641 2192 _MISMATCH_GUIDANCE 6178 5031 _CRASH_GUIDANCE 2058 1969 _CULPRIT_GUIDANCE 2110 2041 _OOM_LOAD_GUIDANCE 4624 4333 _OOM_RETENTION_GUIDANCE 1881 1734 the four small blocks 1649 1586 The largest single win is not prose: the trunk told the agent to go and read the root `AGENTS.md` / `CLAUDE.md` / `.ai/AGENTS.md`, which serge already injects verbatim into the system prompt as REPO CONVENTIONS. That paragraph paid for the conventions twice and bought three tool calls; it now says they are already in the prompt and points only at what the agent does need to open — the failing model's test file and its docs page. Delivered instruction, worst case: output_mismatch 8,821 -> 7,225 chars, which fits inside even the old 8,000-char cap with room to spare. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tarekziade
added a commit
to huggingface/serge
that referenced
this pull request
Sep 7, 2026
…118) `build_task_user_prompt` head-truncates the instruction at `MAX_INSTRUCTION_CHARS`, so an over-long one loses its **end** — and transformers-ci appends its newest guidance last. Its `output_mismatch` instruction (shared trunk + per-category block) is **8,821 chars** after [transformers-ci#114](huggingface/transformers-ci#114), against a cap of **8,000**. The 821 chars that fell off were exactly the bullets that change existed to add: ``` [... truncated, 821 chars omitted ...] ``` > cessor/tokenizer call) and check it against what the expectation assumes: a flat list of N tokens, one waveform, one `.squeeze()` that the test expects to collapse everything — all of those say batch size 1… So the guidance was written, reviewed, merged, and never reached the model. Nothing on either side noticed: the truncation is silent apart from a marker in the prompt. 12,000 leaves room above today's largest block (sizes: output_mismatch 8,821 · cluster 4,753 · crash 4,688 · load_error 3,217 · import_or_config 3,121 · trunk alone 2,641). The other half is [transformers-ci#115](huggingface/transformers-ci#115), a test that fails when a category outgrows this number — that repo is where the growth happens. `24 passed` in `tests/test_prompts.py`. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two things, both about how much of this reaches the model.
The instruction was being truncated
#114 pushed the
output_mismatchinstruction to 8,821 chars. serge'sbuild_task_user_prompthead-truncates the instruction atMAX_INSTRUCTION_CHARS(8,000 at the time), so the 821 chars that fell off the end were the two bullets that PR existed to add — "the ACTUAL side of the assertion is off limits" and "a SHAPE or LENGTH mismatch is never expectation drift". Written, reviewed, merged, never delivered.Every category's
build_instructionoutput is now checked against that cap, with a failure message saying what truncation would cost. serge#118 raises the cap to 12,000 for headroom; this is the half that belongs here, where the blocks are written and where the next one will grow.Compression
Only these strings are sent to the model — the
#comments around them are not — so the rationale and dated precedents stay put and only the delivered text shrinks. 21,141 → 18,886 chars. Same rules, same thresholds, same PR numbers and examples: the 112 phrases the test module pins are all still there, and328 passed._MISMATCH_GUIDANCE_OOM_LOAD_GUIDANCE_INSTRUCTION_CULPRIT_GUIDANCE_CRASH_GUIDANCE_OOM_RETENTION_GUIDANCEThe largest single win is not prose. The trunk told the agent to go and read the root
AGENTS.md/CLAUDE.md/.ai/AGENTS.md— which serge already injects verbatim into the system prompt as REPO CONVENTIONS. That paragraph paid for the conventions twice and bought three tool calls; it now says they are already in the prompt and points only at what the agent does need to open (the failing model's test file and its docs page).Delivered instruction, worst case:
output_mismatch8,821 → 7,225 chars, inside even the old 8,000 cap.Where the first load actually goes
For the worst-case task, with the compressed
.ai/AGENTS.md:So the prompts are now the small half. If first-load size is still a concern, the lever is
MAX_CONTEXT_CHARS(40,000) and how much traceback the triage packs into the report — not these blocks.