[sectoramen]

What

When coverage is cut short — the caller's context is done or fires, a --validator-budget fires, CLOUD_RESOURCES declines input over 5 MiB, a file does not parse — every redaction path now follows the rule --file already followed: the output never goes beyond --file's, the cut-short is always disclosed, and it is signalled (exit 3 under --fail-on-incomplete; for a Go caller, an error wrapping ErrIncomplete with the partial result alongside).

For someone running ferret-scan:

• --stdin --enable-redaction (the gateway) exits 3 under --fail-on-incomplete on a cut-short scan, as --file does; it exited 0 whatever the flag said. In pre-commit mode --stdin now prints a one-line notice (ferret-scan: <stdin> NOT fully examined (coverage cut short) — …); it printed 0 bytes.
• pkg/redact.Engine.Redact returns an error wrapping the new redact.ErrIncomplete (and the cause, so errors.Is(err, context.DeadlineExceeded) holds) together with the partial Result (Incomplete, IncompleteReason, both also on AuditRecord). It returned the input unredacted at err == nil for a pre-cancelled context, for 1/100/200 ms deadlines and for a >5 MiB CLOUD_RESOURCES request.
• pkg/scan.RedactFile and the new RedactFileContext return an error wrapping the new scan.ErrIncomplete with RedactFileResult.Coverage set, and never write the clean-file passthrough for a file that was not fully examined. An unparseable .pdf used to come back byte-identical at the output path, at err == nil.
• pkg/scan.ScanText / ScanFile honour their context: (nil, ctx.Err()) when it is done at entry; the partial Result and an error wrapping ctx.Err() when it fires mid-scan. They ignored it (_ context.Context). ScanText now fills NotExamined.
• A fired budget keeps what the validator found (it was discarded), scored exactly as a complete run scores it, and is not retried (each fired budget re-ran every check on the file twice more).
• examples/lambda-redact answers request_id=…: redaction incomplete instead of returning the request's own text as redacted; the README library snippets check err.

Complete runs are unchanged: golden corpus 0 files changed, make score identical, unbudgeted results byte-identical main vs branch.

Which of the issue's 12 items #797, #791 and #796 already covered

None completely; I checked each in c403e27 before writing code.

item	state at c403e27
1 Engine.Redact	not covered: runErr still dropped (engine.go)
2 RedactFile	not covered: core.RedactFile still checked only UnredactedFiles, then passed through. #791 added only the strict strategy check before any write
3 --stdin	not covered: runStdinRedaction still returned 0 / the pre-commit code without resolveIncompleteExitCode; the pre-commit warning was still dropped. #791/#796 touched only strategy/profile handling there
4 example + README	not covered
5 ctx threading	not covered: ScanContent and the worker pool still rooted on context.Background(), /scan passed none
6 ctx semantics	not covered: ScanText(_ context.Context…), ScanFile(_ …)
7 NotExamined, prefix once, RedactFileContext	not covered. Partly stale: the issue's "ScanConfig has no ValidatorBudgets" — #797 added core.ScanConfig.ValidatorBudgets (pkg/scan options still have none)
8 keepMatches	not covered: still errors.Is(r.err, execguard.ErrMatchBudgetExceeded) (now dual_path_bridge.go:1010-1012). #797 moved the routed-path, both-paths-failed, legacy and metadata branches to IsCoverageCutShort, but not this gate
9 score before send	not covered: the early send still preceded the contextual adjustment
10 grace drain	not covered: both drains still stopped at default:
11 no retry	not covered
12 predicate + AST guard	not covered: ScanContent's hand chain unchanged (omitting ErrValidatorPanicked). #797 added a go/ast derivation of the family for its own table test, which this guard reuses as a pattern

Verification

Measured: shared main binary built from c403e27 vs this branch. Inputs from the issue's recipe, regenerated by the script below: records.txt 2,177,780 B (matches the issue), infra.log 5,810,529 B with 5,518 ARN lines (the issue's padding is unspecified; it is 5,734,445 B there; both are over the 5,242,880 B cap), broken.pdf, one.txt. Empty config home, absolute paths, Linux, 16 cores.

gen_inputs.py

#!/usr/bin/python3
# Generates the synthetic inputs of issue #770 ("Inputs and method").
# All values are synthetic: Luhn-valid 453201... card bodies, 5xx-xx-xxxx
# SSN-shaped values, example.org addresses, one fixed fake S3 ARN.
import os
import sys

out = sys.argv[1] if len(sys.argv) > 1 else "."
os.makedirs(out, exist_ok=True)


def luhn_digit(body: str) -> str:
    total = 0
    for i, ch in enumerate(reversed(body)):
        d = int(ch)
        if i % 2 == 0:  # the digit that will sit left of the check digit is doubled
            d *= 2
            if d > 9:
                d -= 9
        total += d
    return str((10 - total % 10) % 10)


def card(i: int) -> str:
    body = "453201" + "%09d" % (100000000 + (i * 7919) % 899999999)
    full = body + luhn_digit(body)
    return " ".join(full[k:k + 4] for k in range(0, 16, 4))


def ssn(i: int) -> str:
    return "5%02d-%02d-%04d" % (10 + i % 80, 10 + i % 89, 1000 + i % 8999)


# one.txt -- the reproduction's single line.
with open(os.path.join(out, "one.txt"), "w") as f:
    f.write("Ticket 4471: caller verified SSN %s, reply to j7.example@example.org\n" % ("5%02d-%02d-%04d" % (17, 17, 1007)))

# records.txt -- 20,000 lines.
start = 0  # Record 0 .. Record 19999 gives the issue's 2,177,780 B
with open(os.path.join(out, "records.txt"), "w") as f:
    for i in range(start, start + 20000):
        f.write("Record %d: customer card %s, SSN %s, contact j%d.example@example.org on file.\n" % (i, card(i), ssn(i), i))

# records-N.txt -- the first N lines of records.txt, for library rows sized in records.
with open(os.path.join(out, "records.txt")) as f:
    lines = f.readlines()
for n in (2000, 4000, 10000):
    with open(os.path.join(out, "records-%d.txt" % n), "w") as g:
        g.writelines(lines[:n])

# infra.log -- one e-mail line, then the ARN on every 21st line and padding otherwise,
# sized just over the cloud-resources 5 MiB cap (5,734,445 B in the issue).
arn = "10-01 put arn:aws:s3:::synthetic-bucket-q7x2/logs ok\n"
pad = "10-01 heartbeat ok, queue depth nominal, all well\n"
with open(os.path.join(out, "infra.log"), "w") as f:
    f.write("2026-10-01T00:00:00Z INFO notify ops-oncall.example@example.org on failure\n")
    n = 0
    arns = 0
    while arns < 5518:
        n += 1
        if n % 21 == 0:
            f.write(arn)
            arns += 1
        else:
            f.write(pad)

# broken.pdf -- a %PDF-1.7 header and one SSN-shaped value, no PDF structure.
with open(os.path.join(out, "broken.pdf"), "w") as f:
    f.write("%%PDF-1.7\nCustomer SSN %s on file\n" % ("5%02d-%02d-%04d" % (21, 33, 4567)))

# clean.txt -- nothing sensitive.
with open(os.path.join(out, "clean.txt"), "w") as f:
    f.write("The quarterly report is attached. Nothing else to add.\n")

for name in sorted(os.listdir(out)):
    p = os.path.join(out, name)
    if os.path.isfile(p):
        print("%-18s %10d B" % (name, os.path.getsize(p)))

Matrix A — surfaces on cut-short inputs

CLI runs: --enable-redaction --format json --output findings.json (the two detect rows omit --enable-redaction). Exit: without / with --fail-on-incomplete.

input / trigger	route	exit main	exit branch	redacted output (branch; main where it differs)	stderr B (with flag) main→branch	findings.json B main→branch
one.txt, no budget	--file	0 / 0	0 / 0	copy, SSN and e-mail masked	0→0	2330→2330
one.txt, no budget	--stdin	0 / 0	0 / 0	stdout, both masked (byte-identical to the --file copy)	36→36	2152→2152
clean file	--file	0 / 0	0 / 0	no copy	0→0	237→237
clean file	--stdin	0 / 0	0 / 0	stdout = input	36→36	19→19
one.txt, all=1ns	--file	0 / 3	0 / 3	no copy	194→194	266→266
one.txt, all=1ns	--stdin	0 / 0	0 / 3	stdout = input (nothing found)	213→195	19→19
one.txt, all=1ns, --pre-commit-mode	--file	0 / 3	0 / 3	no copy	124→124	194→194
one.txt, all=1ns, --pre-commit-mode	--stdin	0 / 0	0 / 3	stdout = input	0→130	0→0
broken.pdf, SSN	--file	0 / 3	0 / 3	no copy	296→296	266→266
infra.log, CLOUD_RESOURCES,EMAIL	--file	0 / 3	0 / 3	copy: e-mail masked, 5,518/5,518 ARN lines in clear	288→288	1365→1365
infra.log, CLOUD_RESOURCES,EMAIL	--stdin	0 / 0	0 / 3	stdout byte-identical to the --file copy (main and branch)	306→289	1134→1134
records.txt, SSN, SSN=300ms	--stdin	0 / 0	0 / 3	main: stdout = input, 20,000/20,000 SSNs clear; branch: 2,048–2,816 masked (the kept partials; wall-clock budget)	215→197	19→203,371 (200 results, default --limit)
one.txt, all=1ns (detect)	--stdin	0 / 3	0 / 3	—	213→195	19→19
one.txt, all=1ns, --pre-commit-mode (detect)	--stdin	0 / 3	0 / 3	—	0→130	0→0

The stdin stderr shrinks by 17–18 B because the reason now reads coverage cut short: context deadline exceeded (was validator execution did not complete: …). The stdin JSON stats block (stats.files_not_examined) is not in this PR — see Deviations (U33).

Library calls (context.Background() unless noted)

call	main	branch
RedactFile(broken.pdf, SSN)	err=nil, count 0, output = input	scan: redaction incomplete: …/broken.pdf: cannot parse: Text Extractor: no text extracted from .pdf: …; no output file; Coverage{Incomplete:true, NotExamined:[{…/broken.pdf, cannot parse, …}]}
RedactFile(infra.log, CLOUD_RESOURCES,EMAIL)	err=nil, count 1, 5,518 ARN lines in clear	ErrIncomplete (…: coverage cut short: validator declined oversize content: cloud-resource scan skipped, content is 5810529 bytes against a 5242880-byte cap); count 1; the partial copy is kept (e-mail masked, ARNs in clear), as --file writes it; Coverage set
RedactFile(infra.log, CLOUD_RESOURCES)	err=nil, count 0, passthrough = input	ErrIncomplete; no output file
RedactFile(one.txt), RedactFile(clean.txt)	err=nil	err=nil, Coverage == nil (clean passthrough unchanged)
Engine.Redact(infra.log, CLOUD_RESOURCES)	err=nil, 0 findings, Redacted == input	redact: redaction incomplete: validator declined oversize content: … (errors.Is ErrIncomplete); Incomplete, AuditRecord().Incomplete true
Engine.Redact, 2,000 records, Background()	6,000 findings	6,000 findings, err=nil
… pre-cancelled / 1 ms deadline	err=nil, 0 findings, Redacted == input	ErrIncomplete + context.Canceled / DeadlineExceeded, 0 findings, Incomplete
… 100 ms / 200 ms deadline	err=nil, 0 findings, Redacted == input	ErrIncomplete + DeadlineExceeded, 4,768 / 5,536 findings kept (the grace drain)
… 2 s deadline (does not fire)	6,000	6,000, err=nil
ScanText(infra.log, CLOUD_RESOURCES)	err=nil, Incomplete, len(NotExamined)=0, reason validator declined oversize content: validator declined oversize content: …	err=nil (an engine guard), Incomplete, NotExamined=[{<text>, coverage cut short, …}], reason coverage cut short: validator declined oversize content: … (once)
ScanText / ScanFile, 10,000 records, pre-cancelled	30,000/30,000 findings, Incomplete=false	nil, context.Canceled
… 50 ms deadline	30,000/30,000, Incomplete=false	partial result (Incomplete, NotExamined cut short, 0 findings) + scan: cut short by the caller's context: context deadline exceeded

Cancellation leaves work running. Engine.Redact on 20,000 lines, 50 ms deadline, 5 samples: main returns at 50 ms with 0 findings and burns 0.213–0.221 s CPU in the next 4 s; the branch returns at 150 ms (deadline + CancelGrace) with 0 findings and an error, then 0.111–0.120 s. The issue reads that post-return CPU as validators' work being thrown away; on this host it is mostly the document context analysis that runs before any validator and cannot be interrupted (AnalyzeContext on records.txt: 262–264 ms, measured with a throwaway probe). So the grace period recovers nothing at that size; it recovers findings when validators are what is running (the 100/200 ms rows above).

Matrix C — fired budgets (--file records.txt --format json --confidence all --limit 0, 3 interleaved rounds, 1-min load 1.8–4.5)

--checks	budget	findings main	findings branch	files_not_examined	CPU s main	CPU s branch	wall s main	wall s branch
SSN	none	20,000	20,000	absent	3.91–3.95	3.88–3.90	2.97	2.95–2.96
SSN	SSN=300ms	0	2,816	1	1.99–2.05	0.80–0.83	5.15–5.41	0.68–0.69
SSN	SSN=1s	0	8,960–9,216	1	4.55–4.63	1.90–1.95	7.11–7.41	1.49–1.53
SSN	SSN=1.5s	0	13,312–13,568	1	6.48–6.49	2.71–2.75	8.44–8.77	2.09–2.13
SSN	SSN=3s / 20s (does not fire)	20,000	20,000	absent	3.90–4.02	3.86–3.99	2.96–3.03	2.96–3.00
SSN,EMAIL	none	20,000 SSN + 20,000 e-mail	same	absent	5.40–5.54	5.45–5.63	3.41–3.50	3.42–3.56
SSN,EMAIL	SSN=300ms	e-mail only	e-mail + 2,560–2,816 SSN	1	4.43–4.56	2.06–2.12	5.79–5.95	1.19–1.20
SSN,EMAIL	SSN=1s	e-mail only	e-mail + 8,704 SSN	1	7.36–7.44	3.22–3.29	7.66–7.81	1.91–1.93

Acceptance rows, measured with --show-match on the branch:

• Every kept SSN partial equals its unbudgeted twin on the whole result object — text, line, type, confidence and metadata: SSN SSN=1s 8,960/8,960; SSN,EMAIL SSN=1s 8,448/8,448; SSN,EMAIL SSN=300ms 2,560/2,560. (The issue's A/B saw each partial score 20 points under its twin: records.txt is classified CSV, worth +20, and the partials were sent before that adjustment.)
• --debug, SSN,EMAIL, SSN=300ms: email_validator validate_content records main 3 → branch 1 (no budget: 1 → 1); enhanced_validator_bridge process_content 3 → 1.
• --fail-on-incomplete on the fired budget: rc 3 on both.
• Unbudgeted runs, main vs branch: results identical (20,000 and 40,000 findings, metadata included).

Acceptance criteria

• Engine.Redact (pre-cancelled ctx, past deadline, >5 MiB CLOUD_RESOURCES): errors.Is(err, redact.ErrIncomplete), Result.Incomplete and AuditRecord().Incomplete true, no input bytes in reason, error or audit record (pkg/redact/incomplete_test.go); examples/lambda-redact returns an error and an empty Response (handler_test.go, build tag examples_lambda).
• RedactFile (unparseable .pdf; >5 MiB CLOUD_RESOURCES-only log): errors.Is(err, scan.ErrIncomplete), res.Coverage.Incomplete, no file at the output path; complete run res.Coverage == nil.
• [~] one.txt all=1ns (also --pre-commit-mode) and infra.log: --stdin and --file redaction exit 0 without the flag, 3 with it, and disclose on stderr — done. Stdin JSON stats.files_not_examined == 1 — not done, waits for U33 as the issue specifies.
• ScanText/ScanFile: pre-cancelled → nil, context.Canceled; mid-scan → Incomplete, NotExamined[0].Cause == "coverage cut short", error wrapping ctx.Err(); size-capped ScanText → non-empty NotExamined, the guard named once.
• scan.RedactFileContext exists, RedactFile keeps its signature, RedactFileResult stays comparable (TestRedactFileResultStaysComparable: Comparable() and a map key; there is no U38 snapshot yet), /scan passes the request's context.
• records.txt, SSN,EMAIL, a budget that fires: non-zero SSN findings, each identical to the unbudgeted run; --debug 1 email_validator record (was 3); exit 3 under the flag.
• A stub returning N matches with DeadlineExceeded yields N findings with confidence and metadata equal to the nil-error case (TestCutShortPartialsAreKeptAndScoredLikeTheirTwins, every member of the family that can carry matches).
• The AST guard passes, and fails when errors.Is(err, execguard.ErrMatchBudgetExceeded) is added outside execguard (below).
• Golden corpus byte-identical, docs and CHANGELOG present. make pr-checklist: see Gates.

Tests, and that each fails on main

Track C tests ran in this worktree with main's dual_path_bridge.go and validator_runner.go restored (git show c403e27:…) and two one-line constant shims so they compile. The rest ran in a pristine copy of c403e27, as is or, where the test names API this PR adds, as a probe with the same scenario and main's API (an ErrIncomplete check becomes err != nil).

test	pins	on main
internal/validators TestCutShortPartialsAreKeptAndScoredLikeTheirTwins	items 8, 9; members read from IsCoverageCutShort's source (#797's helper); ErrValidatorPanicked exempt (a panic carries no matches)	4/4 fail: kept 0 of the 7 matches the validator returned with context deadline exceeded (and context canceled, validator declined oversize content); ErrMatchBudgetExceeded: partial confidence 60, metadata keys [] vs twin confidence 80, metadata keys [confidence_adjustment context_confidence context_doctype context_domain original_confidence semantic_context validation_path]
TestLatePartialAfterCancellationIsCollected	item 10, bridge	kept 0 of the 4 matches sent 10 ms after the cancellation
TestValidatorThatNeverReturnsIsAbandonedWithinTheGrace	the bound	passes (main abandons at once)
internal/parallel TestGuardOutcomeIsNotRetried	item 11 through the real facade	the cut check ran 3 time(s) and its sibling 3 (in 3.149s); want 1 and 1; got 0 CUT and 1 SIB matches; want 5 and 1
TestCutShortPartialsSurviveRunValidators	item 8, one layer up	RunValidators returned 0 of the 9 matches the validator made before its budget fired
TestRunValidatorsCollectsALatePartial	item 10, RunValidators	kept 0 of the 3 matches sent 10 ms after the cancellation
TestTransientErrorIsStillRetried, TestRunValidatorsAbandonsAValidatorThatNeverReturns, TestDocumentGraceEndsInsideTheRunnerGrace	decoy, bound, the two constants	pass
internal/execguard TestNoHandCopiesOfTheCutShortFamily (+ TestTheGuardFindsWhatItIsFor, TestFamilyFromSourceMatchesThePredicate)	item 12	5 errors.Is test(s) name a member of the cut-short family outside internal/execguard: internal/core/scanner.go:444:30 context.DeadlineExceeded, :444:84 context.Canceled, :447:36 …ErrMatchBudgetExceeded, :452:36 …ErrContentTooLarge, internal/validators/dual_path_bridge.go:1011:47 …ErrMatchBudgetExceeded. On the branch with errors.Is(err, execguard.ErrMatchBudgetExceeded) injected into internal/core: fails naming it
internal/core TestScanContentReportsARecoveredPanic	item 12's omitted member	after a recovered panic: Incomplete=false NotExamined=[]
TestScanContentNamesASizeCapOnceAndInNotExamined	item 7	guard named 2 times, NotExamined=[]
TestRedactFileReportsAFileItDidNotFullyExamine	item 2 in core	pdf and log: err=<nil>, artifact written, identical to input: true
TestScanContentCtxKeepsWhatWasFoundWhenTheCallerCancelsMidScan, TestRedactFileCtxWithADoneContextWritesNothing	items 5–7 (a stub validator cancels the ctx itself: no clock)	no counterpart: main takes no ctx, which is the defect
pkg/redact TestRedactReportsADoneContext, …AnOversizeDecline, …AHardValidatorError, TestRedactAppliesThePartialFindings	item 1	pre-cancelled: err = <nil>, Redacted == input: true; past deadline: …; err = <nil>; findings 0; Redacted == input: true; err = <nil> for a validator that failed; err = <nil>; Redacted "first [EMAIL-REDACTED] then b@example.org"
pkg/scan TestScanTextAndScanFileReturnTheCtxErrorAtEntry	item 6	ScanText(pre-cancelled): res=true (findings 1) err=<nil>; same for ScanFile
TestScanTextAndScanFileReturnThePartialResultWhenTheCtxFiresMidScan	items 5, 6 (20 ms deadline over ~1 MB with PHONE, #797's pattern)	err=<nil> Incomplete=false findings=1
TestAnEngineGuardIsNotTheCallersCancellation	item 7, #391 contract	NotExamined=[] IncompleteReason="validator declined oversize content: validator declined oversize content: …"
TestRedactFileReportsAFileItDidNotFullyExamine	item 2	pdf, log: err=<nil>, count=0, output identical to input: true; log + address: err=<nil>, count=1
TestRedactFileContextWithADoneContextWritesNothing, TestWithCallerCancellation, TestCompleteRedactFileHasNoCoverage, TestRedactFileResultStaysComparable	new API / no-change guards	no main counterpart / pass on main by design
cmd TestCutShortRedactionFollowsTheFileRule (size cap and PHONE=1ns, both clock-independent)	items 3, matrix A	5/6 subtests fail: --stdin --enable-redaction rc 0, want 3 (the --file rule); budget, with flag: rc 0, want 3; pre-commit stderr has no cut-short notice (it printed 0 B) (×4); --stdin stderr does not disclose the cut-short (main's warning does not name the cause: validator declined oversize content: validator declined oversize content: …). budget, without flag passes on main
examples/lambda-redact TestIncompleteRedactionIsAnErrorNotARedactedResponse	item 4	no error; response {Redacted:reach j7.example@example.org about 517-17-1007 … RequestID:req-770}
internal/goldencorpus TestGoldenFileRedactionRefusesAFileItCouldNotParse	the issue's unparseable-PDF sink case	err = <nil>; artifact written: true

Golden corpus

UPDATE_GOLDEN=1 go test ./internal/goldencorpus/... on the branch: 0 snapshot files changed. No FileCase was added: the unparseable-PDF case lives in golden_file_redact_test.go as its own test, because TestGoldenFileRedactionSink treats any RedactFile error as a failure and every FileCase is fully examined (TestGoldenFileRedactionSink passes unchanged). TestGoldenRedact (the redact.Engine snapshots) unchanged, which is the "complete request byte-identical" check.

make score: PASS, identical on both: recall_all=1.0000 recall_hm=0.9906 prec_hm=0.9589 whole_leak=0 (158 cases, 212 gated labels, 16 checks); the reports differ only in timing lines.

Suppression round trip

Rules generated by main (--generate-suppressions on the complete scan, then all enabled), applied by this branch:

file · checks	run	rules	unsuppressed / suppressed
records.txt · SSN,EMAIL	complete	40,000	0 / 40,000
records.txt · SSN,EMAIL	SSN=300ms (fires)	40,000	0 / 22,048 (20,000 e-mail + 2,048 kept SSN partials)
records.txt · SSN,EMAIL	SSN=1s (fires)	40,000	0 / 26,656
one.txt · SSN,EMAIL	complete	2	0 / 2
infra.log · CLOUD_RESOURCES,EMAIL	complete (cap fires)	1	0 / 1

Every kept partial matches its twin's rule: Tracks A and B change no finding, and Track C only adds findings on cut-short runs, each equal to its twin (above), so legacy confidence-bearing rules match too.

Gates

Head b8998b3, rebased on nothing (base c403e27).

gate	result
gofmt -l .	empty
go vet ./... (whole output read)	clean, 0 lines; also go vet -tags examples_lambda ./examples/lambda-redact/ clean
go test -race -count=1 on every touched package	cmd, internal/{core,execguard,goldencorpus,parallel,preprocessors,validators,web}, pkg/{redact,scan}, examples/lambda-redact (tag): all ok, no race reported
race, whole module (./cmd/... ./internal/... ./pkg/...)	89 packages ok, 0 races
full suite -count=1	89 packages ok, 0 failures
tests/integration (FERRET_TEST_MODE=true, the rest of make test)	ok
make build	ok
UPDATE_GOLDEN=1	0 files changed
make score	identical to main

make pr-checklist BASE=origin/main on b8998b3:

-- changelog                       RAN — entry present under [Unreleased] (30 line(s)); required because it changes what a release
1 gofmt                            RAN — clean
1 go vet                           RAN — clean
1 go build                         RAN — ok
2 full suite (-count=1)            FAIL — --- FAIL: TestGrowthRatioMissesASparseQuadratic (1.41s) FAIL FAIL	github.com/awslabs/ferret-scan/v2/internal/goldencorpus	66.175s
3 golden regen                     RAN — 0 files changed
4 redaction                        NEEDS-MANUAL
5 suppression                      NEEDS-MANUAL
6 timing / O(n^2)                  NEEDS-MANUAL
7 all 7 formats                    NEEDS-MANUAL
8 web / CSP                        NEEDS-MANUAL — web assets changed
10 non-vacuity                     NEEDS-MANUAL
11 cross-platform vet              RAN — linux darwin windows
12 flake                           FAIL — a repeat failed
13 race (whole module)             RAN —  packages ok
-- -short mode                     RAN — ok
-- tree clean                      RAN — this run dirtied nothing

• 2 full suite FAIL / 12 flake FAIL: TestGrowthRatioMissesASparseQuadratic (internal/goldencorpus), the known host flake; test untouched. The script writes its suite and race output to the shared /tmp/_suite and /tmp/_race and discards the flake repeats' output, and other worktrees on this host were running it at the same time (13's empty count is grep: /tmp/_race: No such file or directory), so I re-ran 2, 12 and 13 with the output kept in my own directory: full suite 89/89 ok; race 89/89 ok; three uncached repeats of the ten touched packages: 10/10, 10/10, and 9/10 with the one failure TestGrowthRatioMissesASparseQuadratic: … 8.73x on the cpu clock … ABOVE the 8.0 threshold. Back to back, isolated, -count=3: branch 3/3 pass (5.70x, 7.09x, 7.61x), main 3/3 pass (5.62x, 7.08x, 5.84x), 1-min load 10. Of the nine whole-package goldencorpus runs on this branch whose output I kept, it failed in three, always alone; it reads the CPU clock of a synthetic validator this PR does not touch.
• 4 redaction: simple, format_preserving, synthetic × --file/--stdin × one.txt (complete) and infra.log (cut short, partial copy): one copy each, 0 reported values in clear in any copy or stream, rc 0 / 3 as expected; --stdin stdout byte-identical to the --file copy for simple and format_preserving (synthetic is random per run on main too).
• 5 suppression: the round trip above, rules from main's binary applied by this branch's.
• 6 timing: no per-line or per-match loop changed. The grace wait runs only once a context is done and ends when every validator has reported; the retry change removes work (Matrix C CPU).
• 7 formats: the golden corpus covers all seven (0 changed); the CLI rows above are JSON, stderr and exit codes.
• 8 web: internal/web/server.go passes request.Context() to the scan and changes nothing else: no template, asset, handler route or header.
• 10 non-vacuity: the table above.

Deviations and open questions

1. Item 11 is not in resilience.ClassifyError. internal/execguard imports internal/resilience (SafeRun builds a resilience.NewPermanentError), so ClassifyError cannot call execguard.IsCoverageCutShort without an import cycle. The check sits in the only retry wrapper that runs validators, parallel.retryValidatorStrategy.Run: by identity, before ClassifyError sees the error, so rewording a guard's message cannot change retry behaviour, which is the property the issue asked for. Ordinary transient errors still get MaxRetries=2 (TestTransientErrorIsStillRetried).
2. Two grace constants, not one. The document fan-out runs inside RunValidators' and both start waiting at the same cancellation, so the inner one must end first: parallel.CancelGrace 100 ms, validators.DocumentCancelGrace 50 ms, held together by a test (validators cannot import parallel). The bound is ≤ 100 ms as asked. Its cost: a deadline that fires while nothing can report returns up to 100 ms later (the 150 ms row above, 0 findings either way).
3. Context threading by …Ctx variants (core.ScanContentCtx, ScanFileCtx, RedactFileCtx, parallel.ProcessFilesWithProgressCtx), the repo's existing convention (ProcessContentCtx, ValidateContentCtx), with the old names as context.Background() wrappers, instead of changing the internal signatures: cmd/main.go and the 35-odd test and scorecorpus call sites keep compiling, and every path from pkg/scan, pkg/redact and /scan carries the caller's context. The CLI still roots on Background() (no signal context; out of scope). Inside the pool the context travels on a per-call copy of JobConfig (an unexported field), so the job-submitting goroutine stays in ProcessFilesWithProgress, which internal/goroutineguard audits under that name.
4. Only the jobs are rooted at the caller's context, not the pool. Submit and the workers select on the pool's context; rooting it at the caller's would let a cancellation drop a queued job or a finished result while the collector waits for exactly one result per file. A job that starts after the context is done does not open its file and is recorded as cut short.
5. ScanContent's reason is the taxonomy name once, then the error, for every member (coverage cut short: context deadline exceeded, coverage cut short: validator declined oversize content: …), rather than a label per sentinel — no wording switch, so the AST guard needs no exemption (it supports the issue's one shape, a wording switch under IsCoverageCutShort, and tests it on synthetic sources). Visible change: the --stdin warning's reason.
6. RedactFileContext with a context done at entry returns a non-nil result with Coverage and an error wrapping ErrIncomplete and ctx.Err() (the sink contract), not (nil, ctx.Err()) as ScanText does; it creates no output directory.
7. A nil context is treated as Background() by ScanText, ScanFile and RedactFileContext: they accepted (and ignored) nil before, so honouring the context would otherwise turn a nil into a panic. Engine.Redact already required a non-nil context and is unchanged there.
8. Engine guard vs caller cancellation in pkg/scan: an incomplete result is reported with an error only when the caller's context is done when the scan returns. A complete result is returned as complete even if the context expired after the last validator.
9. CHANGELOG: one Bug Fixes bullet (this wave's contract) rather than the issue's Security (Track A) + Bug Fixes (B, C) split; it names every surface, so it can be split at release.
10. Deferred to U33 (not filed): the stdin FormatterOptions constructor (Stats, NotExamined, OutputToFile). The exit code and stderr parts ship alone, as the issue allows; README-Stdin.md says the stdin JSON does not yet carry stats. Not done (optional): ValidatorBudgets on TextOptions/FileOptions (new public API; U38).
11. Not edited, outside this stream's files: internal/help/help.go (--validator-budget help: "stopped and the scan is marked incomplete" is still true; docs/configuration.md carries the fuller statement); THREAT_MODEL.md:39 (its exit-3 claim is now true for stdin as written); the file path's pre-commit notice in cmd/main.go, which says (contents unreadable) for every cause including a cut-short — the stdin notice says coverage cut short. Edited beyond the issue's file list because the change needed them: internal/parallel/parallel_processor.go (item 5's ProcessFiles*), pkg/redact/doc.go, examples/lambda-redact/README.md.
12. ScanContent hard errors (not cut-shorts) stay logged-not-flagged, as before; Engine.Redact flags every non-nil runErr, as the issue specifies.
13. What the issue got wrong or what moved: ScanConfig has had ValidatorBudgets since fix(validators): keep findings and disclose the cut-short when a metadata-capable file is scanned without METADATA #797; validator_runner.go is internal/parallel/, not internal/validators/; the post-return CPU after a 50 ms deadline on 20,000 lines is mostly context analysis, not validator work (above); line numbers moved (keepMatches is at dual_path_bridge.go:1010).
14. Open question: the stdin gateway still streams the partially redacted text on a cut-short (the issue's rule (a): --file writes its partial copy too). A caller that must never forward such a stream needs --fail-on-incomplete and to drop stdout on exit 3; README-Stdin.md now says so. Withholding output on both paths would be a separate decision.

Checklist

• make test and make build pass — ./internal/... in the full suite (the known goldencorpus flake attributed above) and tests/integration ok; make build ok
• CHANGELOG: an entry under ## [Unreleased] → ### 🐛 Bug Fixes
• Golden corpus regenerated and the diff explained: 0 files changed (complete runs are unchanged); the new sink case is a test, not a snapshot
• Closes #770

Closes #770

[github-actions]

🔍 Ferret Scan Results

Gate mode: report — findings are reported here and do not fail this check; a scan that fails or is incomplete still does.

⚠️ Sensitive data detected in 15 scanned file(s):

• High confidence: 1 findings
• Medium confidence: 3 findings

Findings Summary:

File	Type	Confidence	Line
stdin.go	PERSON_NAME	HIGH	112
stdin.go	INSURANCE_MEMBER_ID	MEDIUM	117
dual_path_bridge.go	PERSON_NAME	MEDIUM	1640
doc.go	BUSINESS	MEDIUM	125

Next Steps:

1. Review the findings above
2. Remove or redact any actual sensitive data
3. Add suppressions for false positives
4. Re-run the scan after making changes

---

🤖 This comment was automatically generated by Ferret Scan

[sectoramen]

Rebased onto main at 10ef847 (after #811 and #799 merged), b8998b3 -> 1a01027 + d5a395c.

• The fix commit applied cleanly; its diff against main is byte-identical to the pre-rebase diff (every file except CHANGELOG.md).
• CHANGELOG.md: the automatic merge put this PR's bullet under ### 🔒 Security, below the Go 1.27.3 entry. d5a395c moves it under ### 🐛 Bug Fixes, after the redaction: overlapping findings leave part of a value in cleartext; identical spans render by sort order, not a rule #773 entry, and restores the blank line before the Security header. changelogcheck passes in PR mode.
• Anchors are now distinct from fix(secrets): scope the line pre-check veto to benign manifest lines #812's (that one files first in Bug Fixes, this one last), so the two merge cleanly in either order; verified with a real git merge both ways.
• On the rebased head: go build ./... and go vet ./... clean, gofmt clean, go test -count=1 ok on all 10 touched packages (cmd, internal/core, execguard, goldencorpus, parallel, preprocessors, validators, web, pkg/redact, pkg/scan), with Go 1.27.3 per the new .go-version.

[github-actions]

🔍 Ferret Scan Results

Gate mode: report — findings are reported here and do not fail this check; a scan that fails or is incomplete still does.

⚠️ Sensitive data detected in 15 scanned file(s):

• High confidence: 1 findings
• Medium confidence: 3 findings

Findings Summary:

File	Type	Confidence	Line
stdin.go	PERSON_NAME	HIGH	112
stdin.go	INSURANCE_MEMBER_ID	MEDIUM	117
dual_path_bridge.go	PERSON_NAME	MEDIUM	1640
doc.go	BUSINESS	MEDIUM	125

Next Steps:

1. Review the findings above
2. Remove or redact any actual sensitive data
3. Add suppressions for false positives
4. Re-run the scan after making changes

---

🤖 This comment was automatically generated by Ferret Scan

[github-actions]

🔍 Ferret Scan Results

Gate mode: report — findings are reported here and do not fail this check; a scan that fails or is incomplete still does.

⚠️ Sensitive data detected in 15 scanned file(s):

• High confidence: 1 findings
• Medium confidence: 3 findings

Findings Summary:

File	Type	Confidence	Line
stdin.go	PERSON_NAME	HIGH	112
stdin.go	INSURANCE_MEMBER_ID	MEDIUM	117
dual_path_bridge.go	PERSON_NAME	MEDIUM	1640
doc.go	BUSINESS	MEDIUM	125

Next Steps:

1. Review the findings above
2. Remove or redact any actual sensitive data
3. Add suppressions for false positives
4. Re-run the scan after making changes

---

🤖 This comment was automatically generated by Ferret Scan