Repository navigation
fix(redaction): write overlapping findings once, over their union - #811
Merged
Merged
Conversation
Every writer applied findings one at a time, locating each by its text in
content an earlier write had already changed. When two findings on a line
shared bytes, the second could not be found and was skipped, leaving part
of it in cleartext at exit 0 with failed_redactions 0 in the audit:
Residence: 77 August Lane, Boise ID 83702
.txt/--stdin/pkg Residence: 77 August *********** ID 83702
.docx Residence: **************, Boise ID 83702
.doc/.xls/.ppt Residence: 77 ***********, Boise ID 83702
Legacy OLE had no overlap pass at all, so containment leaked there too
("Located at 100 ****************** Blvd"). For the same bytes reported
twice (PASSPORT and SSN on one 9-digit number) the plaintext writer's sort
tied, and sort.Slice breaks ties by algorithm, which changes at 13
elements: the last four digits survived or not depending on how many
unrelated findings the file held.
redactors.ResolveOverlaps now returns overlap groups: same-line matches
whose spans strictly intersect, transitively, with positions resolved per
type so one value reported by two checks groups once per occurrence.
replacement.RenderGroup renders a group by one rule, using the copied-byte
ranges replacement.Render now returns with each rendering: contained ->
the widest finding, copying no letter or digit inside a contained SECRETS
finding; identical -> the highest-confidence label, keeping a byte only if
every member would; partial -> the union masked (format_preserving) or the
representative's placeholder. The plaintext writer orders writes by
resolved offset, Office records one value per group with the stricter
rendering winning where a group touches a value, and legacy OLE groups,
writes longest first and masks mixed-type crossing groups. A group that
cannot be located refuses the file, and the audit records one entry per
group with member_types.
Measured over the repo's 1,717 tracked files (11,451 findings): 32 of 87
same-span pairs whose types disagree on the last four kept them, now 0;
21 of 22 crossing pairs stranded bytes, now 0. SARIF findings are byte-
identical, make score is unchanged, main-generated suppression rules
still suppress all 11,451 findings.
Closes #773
sectoramen
added
bug
Contributor
🔍 Ferret Scan ResultsGate mode:
Findings Summary:
... and 357 more findings
Next Steps:
🤖 This comment was automatically generated by Ferret Scan |
…s random digits TestRenderGroupRuleTable asserted that the synthetic rendering of every group contains neither "0009" (the card row's last four) nor "B7vg" (the secret row's prefix). A synthetic SSN or passport is nine freshly drawn digits, so "0009" appears in one of them about one run in seven: 14 of 100 runs on Linux, and the Windows leg of #811's second CI run, where the {{U18}} row drew "000910363". The assertion now reads what the rule actually governs: the rendering's Copies, the value bytes it wrote back by construction. No row of the table copies a value byte under synthetic (the card's issuer prefix, the one candidate, is hidden by its SECRETS co-member), so the check is that the copied set is empty. 200 consecutive runs pass.
Contributor
Author
|
Second commit, The first CI run's Windows red was the The assertion checked that the synthetic rendering's text contains neither The check now reads what the rule governs, the rendering's |
Contributor
🔍 Ferret Scan ResultsGate mode:
Findings Summary:
... and 357 more findings
Next Steps:
🤖 This comment was automatically generated by Ferret Scan |
Contributor
🔍 Ferret Scan ResultsGate mode:
Findings Summary:
... and 357 more findings
Next Steps:
🤖 This comment was automatically generated by Ferret Scan |
rustic
approved these changes
Oct 26, 2026
sectoramen
pushed a commit
that referenced
this pull request
Oct 26, 2026
The rebase onto main (now carrying #811 and #799) auto-merged the #765 bullet into the Security section below the Go 1.27.3 entry; it is a bug fix, so it moves under Bug Fixes, as its first entry: #813 appends its own bullet after the #773 entry, and two PRs inserting at one seam would conflict again at the second merge; distinct anchors merge in either order. corpus.go: both #811 (overlap_groups_render_once) and #765 (the two line-veto cases) appended to the case table; keep all three in that order. The metadata key census is regenerated on top of main: the only movement is +14 findings at 13 keys, which is the secrets_line_veto_triggers case; no other bucket and no existing snapshot changed, and the two new snapshots are byte-identical to the ones committed before the rebase.
This was referenced Oct 26, 2026
rustic
pushed a commit
that referenced
this pull request
Oct 27, 2026
…812) * fix(secrets): scope the line pre-check veto to benign manifest lines containsSecretIndicators, the pre-check in front of the SECRETS keyword pass, ran three whole-line substring checks before its provider-literal admits: a version or boolean after the line's first colon, an "integrity"/"checksum" key beside the bytes sha or md5, and a "url"/"resolved"/"homepage" key beside a scheme and a registry or forge host. A hit dropped the whole line, so one "url" member hid an AKIA key ID, a ghp_/glpat-/xoxb- token, a JWT or a keyword-assigned password anywhere else on it: no finding, no redacted copy, --pre-commit-mode exit 0, and the --stdin redaction gateway passed the token through. The provider-literal admits (JWT, every provider prefix, PEM markers) now run first, and the three checks are replaced by one veto, benignManifestLine, that rejects a line only when (a) it is exactly one JSON member whose value is a version under 20 bytes, a boolean, an SRI digest or a URL with no user info, query or '=', (b) no keyword stem from the lists the keyword patterns are compiled from appears anywhere on it, and (c) one of the old checks fires as well, so the veto is a subset of what was skipped before. Checking only the key, as the issue's prototype does, is unsound: in "https://api.session:8443/..." the unquoted pattern reads `session:` as an assignment. findKeywordSecrets is split into the gate and keywordCandidates so that the gate's contract (a rejected line has no candidate) is tested. Measured against v2.5.5 on identical inputs: the issue's nine triggers report what their controls report (30 of 30 draws); matrix 1 goes from 99 to 0 of 143 trigger cells reporting nothing (controls 78/78 on both); every matrix 2 cell, as low as 0/40 before, is 40/40; the redacted copies, pre-commit exits and stdin gateway are fixed. No existing finding changes type, text, line or confidence on the matrices, the reproduction, 39 public npm lockfiles or 3,691 public files that hit an old check, and main's suppression rules still suppress all of them. The veto still skips 99.1% of the lockfile lines the old checks skipped; SECRETS CPU on those lockfiles is 3-6% above v2.5.5 (with no veto at all, 2.7x). Closes #765 * test(goldencorpus): state the #765 trigger case's password length right The fifth line of secrets_line_veto_triggers carries a 41-byte password, not the 40-byte one its description named. The value stays 41 bytes: a 40-byte value beside an AKIA key ID is the shape secret scanners read as an AWS key pair. Description only; no snapshot changes. Refs #765 * docs(changelog): count all 31 seeded draws in the #765 entry The reproduction ran on seeds 1-31; the entry quoted the 30 draws of seeds 2-31. On all 31 every trigger reports what its control reports, and main lost the 40-character password on 10 of them. Refs #765 * chore(goldencorpus,changelog): rebase #765 onto main after #811 The rebase onto main (now carrying #811 and #799) auto-merged the #765 bullet into the Security section below the Go 1.27.3 entry; it is a bug fix, so it moves under Bug Fixes, as its first entry: #813 appends its own bullet after the #773 entry, and two PRs inserting at one seam would conflict again at the second merge; distinct anchors merge in either order. corpus.go: both #811 (overlap_groups_render_once) and #765 (the two line-veto cases) appended to the case table; keep all three in that order. The metadata key census is regenerated on top of main: the only movement is +14 findings at 13 keys, which is the secrets_line_veto_triggers case; no other bucket and no existing snapshot changed, and the two new snapshots are byte-identical to the ones committed before the rebase. --------- Co-authored-by: union <union@example.invalid>
rustic
added a commit
that referenced
this pull request
Oct 27, 2026
…ls (#813) * fix(redaction): fail a cut-short redaction on every sink, keep partials When coverage was cut short -- a caller's ctx done or firing, a validator budget, the CLOUD_RESOURCES size cap, an unparseable file -- every redaction path but --file returned partial or unredacted output as success. Measured on v2.5.5 (c403e27): --stdin --enable-redaction exited 0 under --fail-on-incomplete (1ns budget, a 5.8 MB log over the cap, SSN=300ms on 20,000 records) and printed 0 B in pre-commit mode; Engine.Redact returned err == nil and the input unredacted for a pre-cancelled ctx and for 1, 100 and 200 ms deadlines (0 of 6,000 findings); RedactFile copied an unparseable .pdf byte-identical to the output path, and the over-cap log with 5,518/5,518 ARN lines in clear; ScanText/ScanFile ignored their ctx (30,000/30,000 findings under a pre-cancelled one). Sinks (the --file rule: output never beyond --file's, disclosure always, exit 3 under the flag / an error wrapping ErrIncomplete): - pkg/redact: a done ctx at entry or any validator error returns the partial Result (Incomplete, IncompleteReason, also on AuditRecord) and an error wrapping the new redact.ErrIncomplete and the cause. - pkg/scan.RedactFile and the new RedactFileContext: a file in any of the three --fail-on-incomplete channels gets no passthrough copy, and an error wrapping the new scan.ErrIncomplete with RedactFileResult.Coverage (a pointer: the struct stays comparable). core.RedactFile does the same with core.ErrRedactionIncomplete; ScanFile and RedactFile share coverageOf. - --stdin: both exits go through resolveIncompleteExitCode; pre-commit mode prints a one-line notice. - examples/lambda-redact maps ErrIncomplete to "redaction incomplete"; the README snippets check err. Caller ctx: core.ScanContentCtx/ScanFileCtx/RedactFileCtx and ProcessFilesWithProgressCtx root the job contexts at the caller's (the pool's own context stays the pool's, or a cancellation would drop jobs the collector waits for); ScanText/ScanFile return (nil, ctx.Err()) at entry and the partial result with an error wrapping ctx.Err() mid-scan; /scan passes the request's. Partials: keepMatches uses execguard.IsCoverageCutShort (it tested ErrMatchBudgetExceeded alone); a kept partial is scored and annotated before it is sent; both fan-outs keep collecting for a bounded grace period after Done (100 ms RunValidators, 50 ms the nested document fan-out); the worker pool never retries a guard outcome; ScanContent uses the predicate (its chain omitted ErrValidatorPanicked and named the size cap twice) and fills NotExamined. A go/ast guard derives the family from IsCoverageCutShort and fails on any errors.Is hand copy outside internal/execguard; on main it finds 5 calls in 2 sites. records.txt, SSN,EMAIL, SSN=1s: SSN findings 0 -> 8,704 (each equal to its unbudgeted twin, metadata included), CPU 7.36-7.44 s -> 3.22-3.29 s, email_validator passes 3 -> 1. Complete runs are byte-identical: golden corpus 0 files changed, make score identical, rules generated by main still suppress every finding, kept partials included. Closes #770 * chore(changelog): place the #770 bullet under Bug Fixes after the rebase Rebasing onto main after #811 and #799 auto-merged the cut-short bullet into the Security section, below the Go 1.27.3 entry. It is a bug fix; move it under Bug Fixes after the #773 entry. Also restores the blank line before the Security header. --------- Co-authored-by: union <union@example.invalid> Co-authored-by: Lee Myers <ichilegend@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #773. Redaction-only: findings, recall, false positives and suppression hashes are unchanged; only the bytes written for findings that share bytes on a line change. No file in common with the open wave-2 streams except
CHANGELOG.md(the### 🐛 Bug Fixesheading hunk).pkg/redact,pkg/scan,internal/core/redact.goandcmd/stdin.goare untouched: all four reach the fixed writer throughplaintext.RedactString.What
When two findings on one line share bytes, ferret-scan wrote one of them and skipped the other, so part of a reported value stayed in cleartext, at exit 0, with
failed_redactions: 0in the audit log:Legacy
.doc/.xls/.ppthad no overlap pass at all, so containment leaked there too (Located at 100 ****************** Blvd). And for the same bytes reported twice (PASSPORTandSSNon512010001), which type was written depended onsort.Slice's tie order, which changes at 13 elements: the passport's last four digits survived (*****0001) or not depending on how many unrelated findings the file held.Now every writer makes one write per overlap group, over the union of its members' bytes:
format_preservingwould copy (last four digits, an email's first character) is kept only if every finding on it would keep it*mask (format_preserving) or that type's rendering when all members share one type; the highest-confidence placeholder (simple)syntheticuses the representative's generator and falls back to its placeholder when the fake would carry a byte a member hides. The label is chosen by confidence, then type name, then input index: never by input order or file size. A group that cannot be located refuses the file instead of writing part of it, and the audit log records one entry per group withmember_types. Touching values are not a group.Verification
All values synthetic (
512010001area 512; the repo's fixture names and addresses; computed check digits). Main = the sharedc403e27binary, branch = this head, isolated empty config home, no suppressions.Cross-format table (the issue's script, plus
.xls/.pptbuilt withinternal/olefixture.MustBuild)simple | format_preserving | synthetic, last line of the redacted copy:.txt77 August [PERSON-NAME-REDACTED]·77 August ***********·77 August Gussie Shih[US_STREET_ADDRESS-REDACTED]·*********************·[US_STREET_ADDRESS-REDACTED].docx[US_STREET_ADDRESS-REDACTED], Boise·**************, Boise·FZauqAR1XZ1MMT, Boise[US_STREET_ADDRESS-REDACTED]·*********************·[US_STREET_ADDRESS-REDACTED].doc/.xls/.ppt77 ***********, Boise(all three strategies)*********************(all three).txt/.docx[US_STREET_ADDRESS-REDACTED]· 27*· random.doc/.xls/.ppt100 ****************** Blvd(all)*(all).txt[SSN-REDACTED]·*****0001·999577937(SSN generator)[PASSPORT-REDACTED]·*********· passport generator.docx[PASSPORT-REDACTED]·*********.doc*********rc 0 and one output file on every run, both sides.
Identical-span cases (
--stdin)*****0001/[SSN-REDACTED]at 11-before, 30-before, 30-both, 60-*;*********/[PASSPORT-REDACTED]elsewhere*********/[PASSPORT-REDACTED]in all 18Passport number: 5120100NN(40 findings)[SSN-REDACTED]12 /[PASSPORT-REDACTED]8;--file12;.docx0 of 20--file0;.docx0 of 20driver license 512010001,Provider NPI: 1000000004 at the clinic,MRN: 2125550100,Account number: 2125550100, alone → after 11 email lines (FP)*****0001,******0004,******0100,******0100; simple flips to[SSN-REDACTED]/[PHONE-REDACTED]password = "<24 random alnum>@example.org"(generated at run time), k = 0..20 lines each sideX***…@example.org/[EMAIL-REDACTED](and at k = 40, 60: full mask /[API_KEY_OR_SECRET-REDACTED])[API_KEY_OR_SECRET-REDACTED]at every ksecret = "4532-0000-0000-0009"alone (VISA + API_KEY_OR_SECRET)****-****-****-0009****-****-****-****.txt(simple / FP)Renée [PERSON-NAME-REDACTED], François Lefèvre/Renée *****…[PERSON-NAME-REDACTED], François Lefèvre/ 29*.docx[PERSON-NAME-REDACTED], José García, …/**************, José García, ….txt--redaction-audit-log, x1 / namesdata_types [PERSON_NAME], failed 0data_type US_STREET_ADDRESS,member_types [PERSON_NAME PERSON_NAME US_STREET_ADDRESS],data_typeslists both typesThreshold, through
plaintext.RedactString(FP, k unrelated matches then the pair;P=*********,S=*****0001):Population (the issue's method, re-run at
c403e27)--format sarif --confidence all --limit 0over the 1,717 tracked files (git ls-tree, minusconfig.yaml,*.png/gz/pyc), grouped by (uri, line, line text), pairs classified by SARIF byte columns. 11,451 findings (13 without columns). 109 identical, 22 partial, 200 contained pairs (type breakdown in the issue's format is reproduced exactly for partial; identical/contained grew with #761/#762/#766, see below).--fileFP keeps the last 4--show-match)duration_secondsand the wall-clockreconstruction_timestampdiffer, as between two main runs)files_not_redacted5 simple / 3 FP,values_not_redacted24)Every line the branch writes differently from main, accounted for by a script that diffs the two redacted trees line by line (
simple: 78 lines, FP: 84):*(all 22 are EMAIL/BUSINESS ⊃ API_KEY_OR_SECRET from #766: the email mask's first character)The last row is a leak mechanism the issue did not measure. On main the skipped member of an identical pair fell through
findMatchPosition's whole-document fallback and wrote its value over the first remaining copy anywhere in the document:internal/scorecorpus/cases_multicheck.go:391came outUPC 0[US_BANK_ACCOUNT-REDACTED]5(a UPC with nothing reported on it), and ininternal/validators/otp/validator_test.goline 398's API_KEY_OR_SECRET/OTP_SECRET loser wrote[OTP_SECRET-REDACTED]inside line 32'sotpauth://URI, so line 32's own OTPAUTH_URI finding could not be located and was never written. Each such write also landed at an earlier offset than the descending-order invariant allows, so every later finding in the file was re-located by search (--debug:text_mismatch_warning+position_corrected, mapping confidence 0.95 → 0.665 for the rest of the file). On the branch the pair is one write at its own line, so those lines get their own findings' renderings and lines with no finding are left alone — includinginternal/tabular/tabular_test.go:30, whose unreported\t987654321copies main had masked only as that side effect (the completeness sweep declines embedded occurrences by design).Byte identity of non-group shapes
TestNonGroupShapesAreByteIdenticalpins disjoint same-line, touching, other-line, unresolvable (noFullLine) and contained (card ⊃ phone, IP ⊃ date) shapes to the bytes main writes, under simple and FP: passes on main and on the branch (a safeguard, by design). The population table above is the same check at scale.Tests, and how each fails on
mainRun against main's production files (
git show c403e27:<path>for the 8 production files, this branch's tests; a two-constant shim for the metadata keys so tests asserting on them compile):plaintextTestOverlapGroupsLeaveNoMemberBytegot "Residence: 77 August [PERSON-NAME-REDACTED] ID 83702","Renée" survived,"4532" survived: "secret = \"4532-3687-6365-1651\""plaintextTestOverlapGroupRenderingIsOrderIndependentn=2 simple: passport line not rendered by the ruleplaintextTestUnwritableOverlapGroupRefusesan unwritable group must refuse, wrote "Residence: [US_STREET_ADDRESS-REDACTED],\nBoise ID 83702"plaintextTestOverlapGroupRecordsOneMappingWithMemberTypesDataType:PERSON_NAME, no membersplaintext_testTestScanRedactTextRendersIdenticalSpansOrderIndependentlypkg/scan.ScanText→ 200 shuffledRedactTextk=0 (2 findings): got "Passport number: *****0001"plaintext_testTestRedactEngineRendersIdenticalSpansAtAnySizepkg/redact.Engine, k = 0..30format_preserving k=30: got "*****0001",simple k=30: got "[SSN-REDACTED]"officeTestOfficeOverlapGroupIsOneValue/…IdenticalSpanRendersByTheRule/…StricterRenderingWinsForAGroupedValue/…UnwritableGroupRefuses*****0001to the group; refusal"Boise" survived;order [SSN PASSPORT]: got "*****0001";got "SSN *****0001\nPassport number: *****0001";must refuse the documentlegacyoleTestLegacyContainedGroupIsWrittenWhole/…PartialGroupIsMaskedWhole/…LongestValueIsWrittenFirst/…IdenticalGroupKeepsNoDigitAMemberHides/…UnwritableGroupRefuses.doc/.xls/.pptgot "Located at 100 ****************** Blvd\r";"Residence: 77 ***********, Boise ID 83702\r";*****0001; no refusalgoldencorpusTestOverlapGroupSinkEveryFormatcore.RedactFile,.txt/.docx/.doc/.xls/.ppt× 3 strategies, 51 subtests, asserting no member fragment survives"77 August" survived in the redacted .txt,", Boise" … .docx,"at 100" … .doc/.xls/.ppt,"0001" … .txt (format_preserving))goldencorpusTestGoldenRedactgoldencorpusTestCorpusHoldsEveryOverlapShape(derived guard: groups computed from the real scan of everyCasesentry)no corpus case holds a same-line identical overlap group any more, same for partial (onlysecrets_uri_userinfo_email_overlapholds a contained one)redactorsTestResolveOverlaps_*(grouping: contained both orders, three nested, identical both orders, representative tie-breaks, partial over the union — replaces…PartialOverlapKeepsBoth—, chain, touching, disjoint, other line, same line number different source, unresolvable, same value two checks twice on a line, input order),TestIsSecretsMember,TestAuditRecordsOneEntryPerGroupWithMemberTypes;replacementTestRender*,TestRenderGroup*(rule table × 2 orders × 3 strategies incl. {{U18}}, #766, unattributed member, synthetic fallback, touching values not merged)Golden corpus
UPDATE_GOLDEN=1changes:secrets_uri_userinfo_email_overlap.redact_format_preserving.txtredis://default:B***************@cache.orders-svc.io→****************@…on both lines (contained group, SECRETS member inside: the case was marked "PENDING #773, expected to change"). Findings section unchanged; simple snapshot unchangedoverlap_groups_render_once.{txt,csv,json.json,yaml,junit.xml,sarif.json,gitlab-sast.json,redact_simple.txt,redact_format_preserving.txt}203.12.*.*, unchanged rule), PHONE ⊃ SSN (+* ***-**-4100), disjoint card + phonemetadata-key-census.txtIP_ADDRESSenters the census through it (17 keys, band 2). No metadata key is added by codemake score: PASS, identical to main modulo timings (recall_all=1.0000 recall_hm=0.9906 prec_hm=0.9589 whole_leak=0, 158 cases, 212 labels).Suppression round-trip: main
--generate-suppressionsover the 1,717 files → 11,180 rules (enabled for the test). Main with them: 0 findings, 11,451 suppressed; branch: 0 findings, 11,451 suppressed.Gates
gofmt -l .empty;go vet ./...clean (empty output).go test -race -count=1on every touched package (internal/redactors/...,internal/validators,internal/goldencorpus): ok. Whole module-race(./cmd/... ./internal/... ./pkg/...): 89 ok, 0 FAIL, no data race.make pr-checklist: changelog, gofmt, vet, build, golden regen (0 files), cross-platform vet RAN; full suite and the 3× flake repeat FAIL only onTestGrowthRatioMissesASparseQuadratic; the run was then terminated by the host during step 13 (the whole-module race, re-run by hand above: green).go test -short -count=1 ./cmd/... ./internal/... ./pkg/...re-run with output kept: 88 ok, 1 FAIL =TestGrowthRatioMissesASparseQuadratic.go test -count=10 -run TestGrowthRatioMissesASparseQuadratic ./internal/goldencorpus/, interleaved: branch 2/10 then 1/10 failed, main (c403e27) 1/10 then 2/10 — 3/20 each.Deviations and open questions
RenderGrouptakes one strategy and "an unattributed member resolves tosimple" is not implemented. SECRETS membership readsMatch.Validator == "secrets", the attribution aMatchcarries today (Match.Checkdoes not exist); an emptyValidatorcounts as SECRETS.simpleandformat_preservingboth renderformat_preserving, length-preserving; {{U45}} owns that remap): crossing groups of different types are masked whole, identical groups follow the byte rule, and a contained group keeps the widest member's same-length rendering as in.txt/.docxrather than a tagmeta-style full mask, so contained shapes that did not leak in.dockeep their bytes. Value writes are applied longest first (they were in input order): legacy replaces a value in every stream, so a short value reported on another line used to mask the same name inside a longer value before the longer one was found.detector.ResolveLineSpans' cursor is per text, so when two checks report the same value twice on a line the second check's matches both resolve to the first occurrence and the second occurrence of the first check stands alone. Grouping resolves each type separately; the reported columns (AssignLineColumns) are untouched.recordCoverageForRemainingParts) now receives the write units, not the old survivor list: an unreported copy of a crossing member's value in another part is no longer rewritten by that pass, and the residue guard refuses the document instead. Recording members there would let the cross-run pass rewrite a member before its group's union.internal/redactors/index.go(ContentRedaction.MemberTypes,member_types,omitempty;data_typesgains member types) andinternal/redactors/manager.go(copies the metadata into the audit entry), additive, for Resolution 6's "audit records … member types".partcoverage.go,internal/redactverify/*andtagmeta/*are unchanged.creditcard_phone_overlapcannot be made to overlap without changing the PHONE validator (it no longer fires inside a card); its description now says so and points to the new case and the guard. The "identical PHONE/SSN" decoy could not be produced by today's validators in a probe; the case uses PHONE ⊃ SSN containment (both keep the same last four) instead, and identical same-type unanimity is pinned inTestRenderGroupRuleTable. The cross-format sink cases are a test (TestOverlapGroupSinkEveryFormat) rather thanFileCasessnapshots, because theFileCasessink checks only whole values. The order test throughpkg/redactvaries the file size (the engine scans its own input);plaintextandpkg/scan.RedactTextpermute.3df48acd, as it anticipated: 1,717 files / 11,451 findings; 109 identical (87 disagreeing on the last 4, of which main kept 32: the extra one isCHANGELOG.md:978, MRN/PHONE, past SARIF's snippet cap); 200 contained. "0 of 147 contained pairs change" and "golden redact snapshots hold no group" no longer hold: secrets: a password or token in URI user info (scheme://user:pass@host) is never detected, so it survives redaction #766 added 22 EMAIL/BUSINESS ⊃ API_KEY_OR_SECRET pairs, which the SECRETS byte rule now changes as Resolution 2 prescribes, and the golden case it left PENDING. Two of the issue's details do not reproduce onc403e27: the SECRETS/EMAIL line is not identical at every padding on main (at 40 and 60 lines each side main already writes the full mask), and the card-in-secret pair only forms when the line is alone (with ≥ 5 unrelated lines around it SECRETS does not report it, so****-****-****-0009is the correct single-finding rendering on both sides).Checklist
make testandmake buildpass — build, vet and the whole-module race run pass; the non-race suite and-shortfail only on the known host flakeTestGrowthRatioMissesASparseQuadratic, attributed above (3/20 on this branch, 3/20 onmain).## [Unreleased]→### 🐛 Bug FixesCloses #773Closes #773