Repository navigation
fix(cli): refuse an unrecognised --redaction-strategy or FERRET_PRECOMMIT_EXIT_ON (#772) - #791
Conversation
🔍 Ferret Scan Results✅ No sensitive data detected in the changed files. 🤖 This comment was automatically generated by Ferret Scan |
…MMIT_EXIT_ON instead of defaulting
Any value other than exactly simple, format_preserving or synthetic, given
to --redaction-strategy (--file or --stdin), in a profile, in an explicit
config or in an auto-discovered .ferret-scan.yaml, redacted with
format_preserving at exit 0 with an empty stderr: `SIMPLE` wrote masked
values (***-**-8901) where `simple` writes placeholders ([SSN-REDACTED]).
A typo on the flag threw away a valid `strategy: simple` from the config,
and the explicit config's schema trimmed and comma-split, so " simple ",
"simple," and "simple,synthetic" passed it and reached a parser that
matched none of them. FERRET_PRECOMMIT_EXIT_ON ignored any value outside
high/medium/low/none, so with one MEDIUM finding `medium` blocked the
commit and `MEDIUM` let it through: a gate failing open on a typo.
Changes:
- redactors.ParseRedactionStrategy returns (RedactionStrategy, error),
exact match only, with a did-you-mean hint for a case or whitespace
variant; RedactionStrategyNames is exported; RedactionStrategy.Valid.
The lenient parser is gone, so the compiler found every fallback site.
- The strategy travels typed: finalConfiguration, parallel.JobConfig and
core.ScanConfig hold a redactors.RedactionStrategy; the worker refuses a
value outside the definition; core.RedactFile parses its name strictly
before any output path exists ("" keeps its documented default).
- One CLI helper, applyRedactionStrategy, runs on both entry points right
after resolveConfiguration, before any scan, output directory or report,
unconditionally, and names the source: --redaction-strategy, profile "x"
redaction.strategy in <path>, or redaction.strategy in <path>
(auto-discovered). resolveConfiguration records the winning name and its
layer; only the literal empty string means unset.
- The config schema applies the parser's exact rule to redaction.strategy
(validateExactEnumField); --format keeps the lenient rule since it exits
1 downstream. An agreement test pins schema and parser together.
- precommit: an unknown FERRET_PRECOMMIT_EXIT_ON is recorded as
ConfigError; in pre-commit mode both paths exit 1 before scanning,
naming the variable, without "commit blocked" text. The in-code default
branch of ShouldExitOnFindings stays, with its comment corrected.
- pkg/scan: ParseStrategyStrict added; ParseStrategy deprecated,
unchanged. RedactText/RedactFile reject an out-of-range RedactStrategy
before writing. resolveConfig returns an error for a pinned ConfigPath
that exists and fails the strict load (schema or parse), which is what
made the schema safe to tighten; a missing path still falls back (#317).
Measured on v2.5.4 vs this build: every refused spelling now exits 1 with
no output directory and 0 bytes on --stdin; the canonical names write
byte-identical output; the four pre-commit names keep rc 0/1/1/0 (MEDIUM
fixture) and 1/1/1/0 (HIGH fixture); the eight misspellings exit 1 on
both paths; outside pre-commit mode the variable has no effect. Golden
corpus: 0 files changed. make score unchanged. No finding text, line or
type changes, so suppression hashes are unchanged.
Tests: CLI contract tests across the five channels and the pre-commit
matrix (all fail on main); a derived class guard that selects every
flag.String whose usage lists values (via go/ast) and every FERRET_* row
of the pre-commit docs table that lists values, and requires each to
refuse an out-of-domain value -- on main it selects eight inputs and
fails on exactly --redaction-strategy and FERRET_PRECOMMIT_EXIT_ON, with
three reasoned exemptions that fail if stale; the schema/parser agreement
test; parser, worker, core and pkg/scan unit tests; a sequencing guard
for the schema-rejected pinned ConfigPath.
Docs: the redaction guide, configuration reference, stdin guide,
pre-commit integration table, team setup guide and --help state the
exact-spelling rule.
Closes #772
21ae95d to
4bf34d4
Compare
|
Rebased onto |
🔍 Ferret Scan Results✅ No sensitive data detected in the changed files. 🤖 This comment was automatically generated by Ferret Scan |
Catches up with #792, #789 and #790. CHANGELOG.md merged without conflict through the merge=union attribute (#781), but union put the result in the wrong place: this branch adds a new ### Features subsection after Bug Fixes, and union emits this side's lines before main's, so #790's and #789's bug-fix bullets landed under Features. [Unreleased] is rebuilt by hand: Bug Fixes holds main's three bullets in main's order and then this branch's, and Features holds this branch's one. Every bullet is moved byte-for-byte; the section's multiset of non-blank lines is unchanged.
🔍 Ferret Scan ResultsGate mode:
Findings Summary:
Next Steps:
🤖 This comment was automatically generated by Ferret Scan |
Closes #772. Independent of #786/#789/#790: no file in common except
CHANGELOG.md(the usual### 🐛 Bug Fixesheading hunk).What was wrong
Two enum-valued inputs silently became their default on any unrecognised value, both measured on
v2.5.4:--redaction-strategy, on--fileand--stdin, in a profile, in an explicit--config, or in an auto-discovered.ferret-scan.yaml: anything other than exactlysimple,format_preservingorsyntheticredacted withformat_preservingat exit 0, empty stderr.SIMPLEwroteSSN: ***-**-8901wheresimplewritesSSN: [SSN-REDACTED](last four digits, email's first character and domain left in the clear). A typo on the flag threw away a validstrategy: simplein the config. The explicit config's schema trimmed and comma-split, so" simple ","simple,","simple,synthetic"and" "passed it and reached a parser that matched none of them. No report named the strategy applied.FERRET_PRECOMMIT_EXIT_ONignored any value outsidehigh, medium, low, none: with one MEDIUM finding,mediumblocked the commit (rc 1) andMEDIUM,Medium,LOWlet it through (rc 0). A gate failing open on a typo.Root causes (
main@e8c6fe2):redactors.ParseRedactionStrategyhad no error return and adefault:offormat_preserving, called from four sites; the flag was an unvalidatedflag.String;validateEnumFieldwas lenient where the parser was exact;detector.go'sswitchhad nodefault;pkg/scan.ParseStrategy/RedactFilemapped out-of-range values toformat_preserving.What changed (the issue's Resolution 1–9)
None/NONEblock a HIGH finding today (the gate stays athigh) and folding them tononewould let that commit through after an upgrade;SIMPLEruns would switch from masked to placeholder output. Refusing keeps every rc-0 run byte-identical. Adid you mean "simple"?hint is printed for a case/whitespace variant and the value is still refused.redactors.ParseRedactionStrategy(s) (RedactionStrategy, error);RedactionStrategyNamesexported;RedactionStrategy.Valid(). The lenient parser is removed, so the compiler found every fallback site.finalConfiguration.redactionStrategy,parallel.JobConfig.RedactionStrategyandcore.ScanConfig.RedactionStrategyareredactors.RedactionStrategy; the worker refuses a value outside the definition and reports it as a redaction failure for the file (nothing written). Zero value stayssimple(audit-log integers unchanged).applyRedactionStrategy, called on both entry points right afterresolveConfiguration, before any scan, output directory or report.resolveConfigurationrecords the winning name and its layer; the message names the source:--redaction-strategy,profile "share" redaction.strategy in <path>, orredaction.strategy in <path> (auto-discovered)(Trust-boundary decision for a config discovered inside the scan target: opt-in gating, a THREAT_MODEL entry, and the pre-commit gate that silences the provenance disclosure #293).redaction.strategy(validateExactEnumField, defaults and profiles);--formatkeeps the lenient rule since it already exits 1 downstream.TestRedactionStrategySpellingsAgreeWithTheRedactorpins the two together.FERRET_PRECOMMIT_EXIT_ON: an unknown value is recorded asPrecommitDetector.ConfigError(); in pre-commit mode (--pre-commit-modeor a detected hook) both paths exit 1 before scanning, naming the variable, without "commit blocked" text (gate integrity: three ways the tool reports success on work it did not do (consolidates exit resolver, coverage ledger, --exclude) #667). Outside pre-commit mode the variable is not read.ShouldExitOnFindingskeeps its in-codedefault:with the comment corrected.pkg/scan:ParseStrategyStrictadded;ParseStrategyDeprecated:and unchanged.RedactText/RedactFilerejectRedactStrategy(7)before writing, aspkg/redact.NewEnginedoes.resolveConfigreturns an error for a pinnedConfigPaththat exists and fails the strict load (parse or schema), soScanText/ScanFile/RedactFilesurface it; a missing path keeps the fix(config): name a config discovered in the working directory, and let pkg/scan pin or disable discovery #317 fallback. Without this, tightening the schema turned a pinned config with a custom pattern plusstrategy: " simple "from 1 finding into 0 witherr=nil.One deviation from the issue text, stated:
core.RedactFile'sStrategy: ""keeps its documented meaning (format_preserving). It is an in-process Go API whose empty value means "not set", mirrored by its own tests; any set value that does not match is refused before the output path exists.Measured,
main@e8c6fe2vs this PR--file --enable-redaction --redaction-strategy <v>, SSN line of the redacted copy:simpleSSN: [SSN-REDACTED]format_preservingSSN: ***-**-8901SIMPLE***-**-8901(format_preserving applied)invalid redaction strategy "SIMPLE": valid values are simple, format_preserving, synthetic (did you mean "simple"?) (from --redaction-strategy)simpel," simple",bogus***-**-8901--stdinwithSIMPLE/bogus" simple ","simple,","simple,synthetic"," "strategy: simple+--redaction-strategy bogus.ferret-scan.yamlstrategy: simpel(from redaction.strategy in .ferret-scan.yaml (auto-discovered))--pre-commit-mode, exit code on a MEDIUM-only fixture / a HIGH-only fixture:FERRET_PRECOMMIT_EXIT_ON"",highmedium,lownoneMEDIUM,Medium,meduim," medium",LOW,None,NONE,offhigh)FERRET_PRECOMMIT_EXIT_ON="MEDIUM" is not one of high, medium, low, none (did you mean "medium"?)--stdin --pre-commit-modewithMEDIUMMEDIUMGolden corpus:
UPDATE_GOLDEN=1changes 0 files.make score: PASS, identical to main. No finding text, line or type changes, so suppression hashes are unchanged.Tests, and that each fails on
mainmaincmdTestAnUnusableRedactionStrategyIsRefusedOnEveryChannel--file(rc 1, value + 3 names + source quoted, no output dir, empty stdout),--stdin(0 stdout bytes), explicit config defaults and profile, config-vs-flag precedence, auto-discovered file (message says auto-discovered), unconditional validation; canonical names rc 0 with the v2.5.4 bytes (non-vacuity: simple and format_preserving must differ)cmdTestPreCommitExitOnIsRefusedWhenMisspelled--stdin, variable named, no "commit blocked", empty stdout; no effect outside pre-commit modecmdTestEveryEnumValuedInputRejectsAnOutOfDomainValue--redaction-strategyandFERRET_PRECOMMIT_EXIT_ONconfig_testTestRedactionStrategySpellingsAgreeWithTheRedactor""unset for the schema, refused by the parser" simple ","simple,","simple,synthetic"," "redactorsTestParseRedactionStrategyIsExact,TestRedactionStrategyValidString(),Valid, zero valueparallelTestRedaction_OutOfRangeStrategyIsRefusedNotDefaultedRedactionStrategy(7)is reported inUnredactedFiles, nothing written; zero value redactscoreTestRedactFile_MisspelledStrategyIsRefusedBeforeAnyOutputsimpel/SIMPLE/" simple"/boguserror before the output dir exists;""default unchangedpkg/scanTestParseStrategyStrict…,TestRedactTextRejects…,TestRedactFileRejects…BeforeWriting,TestASchemaRejectedPinnedConfigPathIsAnErrorRedactStrategy(7)refused without writing, the sequencing guard with an exact-spelling controlThe class guard derives its set rather than keeping a list: every
flag.Stringincmd/main.gowhose usage text enumerates values (parsed withgo/ast; astrings.Joinover a name list counts, which is how--checksis registered) and everyFERRET_*row ofdocs/PRE_COMMIT_INTEGRATION.md's environment table whose options cell lists backticked values. Today that is--checks,--confidence,--disable-ip-types,--format,--redaction-strategy,FERRET_PRECOMMIT,FERRET_PRECOMMIT_EXIT_ON,FERRET_PRECOMMIT_EXIT_ON_FIRST, exactly the eight the issue predicted. Three exemptions carry a reason and fail if their input stops being selected:--disable-ip-types(warns and keeps detecting; the--stdinhalf is a sibling item),FERRET_PRECOMMIT_EXIT_ON_FIRST(nothing reads it),FERRET_PRECOMMIT(an unrecognised opt-out keeps the gate on).Compatibility
pkg/scan: only callers passing an undefinedRedactStrategyconstant, or a pinnedConfigPathnaming a file that exists and is invalid, see a new error.ParseStrategyis unchanged.Docs
docs/user-guides/README-Redaction.md,docs/user-guides/README-Stdin.md,docs/configuration.md,docs/PRE_COMMIT_INTEGRATION.md(table row),docs/TEAM_SETUP_GUIDE.md, the--redaction-strategyusage text andinternal/help/help.gonow state the exact-spelling rule and that other values exit 1 naming the source.Test plan
TestGrowthRatioMissesASparseQuadratic(goldencorpus), the perfguard host flake (8.15x against 8.0 on this Cloud Desktop; CI reads 4.6–5.0x), untouched by this PR. Re-run with output kept: that is the only failure. The full suite, the race run and the 3x flake repeats on every touched package were green.simple,format_preserving(byte-identical to main) andsynthetic(accepted, random output) on--fileand--stdin; the refused spellings create no output directory and write 0 bytes on--stdin.redaction.strategyin a config, is now refused rather than re-hashed.mainas listed, and the class guard onmainselects the same eight inputs and fails on exactly the two this issue is about.