You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-`crates/kingfisher-scanner/`: embeddable high-level scanning API
34
+
-`crates/kingfisher-scanner/src/validation/`: shared typed and raw credential validators
35
35
-`tests/`: integration/e2e tests
36
36
-`testdata/`: test fixtures
37
37
-`docs/`: user and developer docs
@@ -81,18 +81,21 @@ Key capabilities:
81
81
-`use-mimalloc` (default)
82
82
-`use-jemalloc`
83
83
-`system-alloc`
84
-
- Validation modules live in `crates/kingfisher-scanner/src/validation/`; optional validation feature sets are defined in `crates/kingfisher-scanner/Cargo.toml` (e.g., `validation-aws`, `validation-gcp`, `validation-database`, `validation-all`).
84
+
- Validation modules live in `crates/kingfisher-scanner/src/validation/`; optional validation feature sets are defined in `crates/kingfisher-scanner/Cargo.toml` (e.g., `validation-raw`, `validation-aws`, `validation-gcp`, `validation-database`, `validation-all`).
85
85
86
86
## Validation and Revocation Policy
87
-
- Default rule: define validation logic in rule YAML (`validation:` block), not Rust code.
88
-
- Code-based validation in `crates/kingfisher-scanner/src/validation/` is an exception path for cases that cannot be expressed reliably in YAML alone (for example AWS, GCP, Coinbase, MongoDB, and similar complex/provider-specific flows).
87
+
- Default rule: define validation logic in rule YAML (`validation:` block), especially `Http` or `Grpc`, not Rust code.
- Raw validators use `validation: { type: Raw, content: <name> }` and are the ad-hoc exception path for provider-specific or protocol-specific validation that cannot be expressed reliably in YAML alone. Implement them in `crates/kingfisher-scanner/src/validation/raw.rs`.
- If a Rust exception path is required, prefer adding a raw validator before introducing a new typed validator. Add a new typed validator only when it represents a reusable schema-level validation family.
92
+
- Do not convert existing typed validators to `Raw` just for consistency.
90
93
- For rules that include validation, add a `revocation:` section whenever the third-party API safely supports revocation.
91
94
92
95
## Common Development Tasks
93
96
- Add a detection rule: follow the workflow below and validate with relevant tests.
94
97
- Add a CLI command: implement under `src/cli/commands/` and register in the CLI command wiring.
95
-
- Add a validator (rare exception path): implement in `crates/kingfisher-scanner/src/validation/`and wire feature flags/dependencies in `crates/kingfisher-scanner/Cargo.toml` only when YAML validation cannot express the required logic.
98
+
- Add a validator (rare exception path): implement it in `crates/kingfisher-scanner/src/validation/`, prefer `raw.rs` for one-off provider flows, and wire the narrowest feature/dependencies in `crates/kingfisher-scanner/Cargo.toml` only when YAML validation cannot express the required logic.
96
99
97
100
## Rule Authoring Workflow
98
101
Use this when creating or updating rules in `crates/kingfisher-rules/data/rules/`.
@@ -105,7 +108,7 @@ Use this when creating or updating rules in `crates/kingfisher-rules/data/rules/
105
108
-`pattern_requirements` (e.g., `min_digits`, `min_uppercase`, `min_lowercase`, `min_special_chars`, `ignore_if_contains`) when format constraints are known.
106
109
-`pattern_requirements.checksum` when provider formats include check digits/signatures.
107
110
5. Add `validation` only when a reliable provider/API check exists.
108
-
6. Put validation in YAML by default; only use Rust validator logic for rare, justified exceptions.
111
+
6. Put validation in YAML by default. If YAML cannot express the check, use an existing typed validator or `type: Raw` exception path; add new Rust validator logic only for rare, justified cases.
109
112
7. Add `revocation` when the provider API supports safe revocation and the flow is well understood.
110
113
8. If a rule needs context from another match (for example ID + secret pair), use `depends_on_rule` and consider `visible: false` on the helper rule.
Copy file name to clipboardExpand all lines: CHANGELOG.md
+6Lines changed: 6 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,6 +2,12 @@
2
2
3
3
All notable changes to this project will be documented in this file.
4
4
5
+
## [v1.95.0]
6
+
- Added 80+ built-in rules, bringing the bundled ruleset to 820 total. New coverage includes Amazon OAuth, Asaas, multiple Azure credential families, Bitrise, Canva, CockroachDB, eBay, Elastic, hCaptcha, Highnote, Lichess, MailerSend, Onfido, Paddle, Pangea, Persona, Pinterest, Proof, Rootly, Runpod, Telnyx, Thunderstore, Valtown, Volcengine, and more.
7
+
- Replaced tree-sitter with a lighter parser-based context verifier built from handwritten lexers plus `tl`/`cssparser`, preserving context-dependent matching while cutting about 19 MB from the release binary.
8
+
- Added a `validation: type: Raw` exception path for provider-specific checks, with new raw validators for Azure Batch, FTP, Kraken, LDAP, RabbitMQ, and Redis. Also added stable request-scoped template values plus new Liquid filters for HMAC-SHA384 hex output and timestamp generation.
9
+
- Expanded live validation coverage for several built-in rules, including Agora, Bitfinex, DocuSign, Dwolla, GitLab, KuCoin, RingCentral, Snowflake, Tableau, Trello, and Webex. Also tightened newly added helper regex to avoid high-match scan regressions, and made preflight-blocked raw validations report as skipped/not attempted instead of failed.
10
+
5
11
## [v1.94.0]
6
12
- Updated vendored `vectorscan-rs` from v0.0.5 (Vectorscan 5.4.11) to v0.0.6 (Vectorscan 5.4.12). The upstream crate now ships pre-extracted sources instead of a tarball+patch, and fixes the `cpu_native` feature flag. Local Windows and musl build patches have been re-applied.
0 commit comments