Skip to content

Commit 2bb3dea

Browse files
sectoramenunion
sectoramen
and
union
authored
fix(secrets): do not cap a hex password in URI user info as an unlabelled identifier (#831)
unlabelledHexIdentifierCap holds a bare 32/40/64-hex generic finding at 55 unless one of secretLabelWords is on the line. It was applied to every candidate in processScopedCandidates, including the structural ones findURIUserinfo produces. A URI user-info password is labelled by its position in the URI grammar (#766), not by a word, and `REDIS_URL` names none of the label words, so a hex password (the shape `openssl rand -hex 16` prints) was held at 55 LOW with confidence_ceiling 55 and --pre-commit-mode exited 0, while the same URI with a mixed-case password reported 100 HIGH. The ceiling is now skipped for a structural candidate, as the M24 cap already is. The user-info dispositions' own LOW ceiling (cand.ceiling) still applies, so default vocabulary and a repeated user name stay LOW. Main (0501dcd) vs this branch, --checks all --confidence all: REDIS_URL=redis://default:<32 hex>@cache.internal:6379/0 55 LOW, exit 0 -> 100 HIGH, exit 1 same URI, 32-char mixed-case password 100 HIGH -> 100 HIGH 40-hex and 64-hex URI passwords 55 LOW -> 100 HIGH 40-hex token as a clone-URL user name 55 LOW -> 100 HIGH ImageUniqueID: "<32 hex>" 55 LOW -> 55 LOW redis://default:password@... 55 LOW -> 55 LOW the #766 matrix (TestURIUserinfoMatrix) 185/185 -> 185/185 Co-authored-by: union <union@example.invalid>
1 parent 944a6bc commit 2bb3dea

14 files changed

Lines changed: 251 additions & 5 deletions

‎changelog.d/0808-uri-hex-password-ceiling.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
### 🐛 Bug Fixes
2+
3+
- **secrets (hex password in URI user info):** a hexadecimal password in URI user info, the shape
4+
`openssl rand -hex 16` prints (`REDIS_URL=redis://default:<32 hex>@cache:6379/0`), is no longer
5+
held at 55 LOW, so `--pre-commit-mode` blocks on it. `unlabelledHexIdentifierCap` demotes a bare
6+
32/40/64-hex value when no secret word is on the line. It read the line for a label, and a URI
7+
password is labelled by its position in the URI grammar (#766), so a keyword-free URL with a hex
8+
password was capped while the same URL with a mixed-case password reported 100 HIGH. A URI
9+
user-info candidate is now exempt from that ceiling, as it already was from the M24 cap; the
10+
user-info dispositions' own LOW ceiling is unchanged. Measured on main vs this branch: the
11+
issue's line, 55 LOW (exit 0) to 100 HIGH (exit 1); 40- and 64-hex passwords and a 40-hex token
12+
in a clone URL, 55 to 100; `ImageUniqueID: "<32 hex>"` stays at 55
13+
([#808](https://github.com/awslabs/ferret-scan/issues/808)).

‎internal/goldencorpus/corpus.go‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -548,6 +548,33 @@ var Cases = []Case{
548548
"postgres://user:REDACTED@db1.orders.invalid:5432/appdb\n" +
549549
"postgres://user:xxxxxxxx@db1.orders.invalid:5432/appdb\n",
550550
},
551+
{
552+
Name: "secrets_uri_hex_outside_userinfo_decoys",
553+
Description: "The #808 boundary, which must stay EMPTY: hexadecimal values in every part of a URI " +
554+
"that is NOT user info. #808 exempts a structural candidate, a password or token in URI user " +
555+
"info, from unlabelledHexIdentifierCap, the 55 ceiling for a bare 32/40/64-hex value on a line " +
556+
"that names no secret word, because the URI grammar is that value's label. The exemption is " +
557+
"safe only while the grammar claims nothing else: the authority ends at the first `/ ? #`, so " +
558+
"a hex path segment (lines 1-2), a hex segment before an `@` in a path (line 3), a commit id " +
559+
"(line 4), a hex host label (line 5), a blob digest (line 6), a fragment (line 7) and a query " +
560+
"value (line 8) are never user info, and `git@` (line 9) is a user name that is not shaped " +
561+
"like a token. None of these is a candidate on main or after the fix.\n" +
562+
" The fix itself makes the issue's line report (`REDIS_URL=redis://default:<32 hex>@...` goes " +
563+
"from 55 LOW, confidence_ceiling 55, to 100 HIGH), so that positive, and the ceiling's own " +
564+
"ImageUniqueID case staying at 55, are pinned by the unit tests in " +
565+
"internal/validators/secrets/uri_userinfo_hex_test.go rather than here: a positive case would " +
566+
"move the metadata-key census.",
567+
Checks: []string{"SECRETS"},
568+
Input: "CACHE_URL=redis://cache.orders.invalid:6379/3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e\n" +
569+
"ASSET_URL=https://cdn.orders.invalid/assets/3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e/app.js\n" +
570+
"https://cdn.orders.invalid/img/3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e@2x.png\n" +
571+
"https://git.orders.invalid/orders/app/commit/8b1e4d7a2c5f9e0b3a6d8c1f4e7b0a9d2c5e8f1a\n" +
572+
"http://3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e.cache.orders.invalid:6379/0\n" +
573+
"https://registry.orders.invalid/v2/app/blobs/sha256:c4e7a1d9b3f6082e5a7c9d1b4f6e8a0c2d5b7e9f1a3c6e8b0d2f4a6c8e1b3d5f\n" +
574+
"REDIS_URL=redis://cache.orders.invalid:6379/0#3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e\n" +
575+
"https://api.orders.invalid/v1/objects?etag=3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e\n" +
576+
"ssh://git@git.orders.invalid/orders/8b1e4d7a2c5f9e0b3a6d8c1f4e7b0a9d2c5e8f1a.git\n",
577+
},
551578
{
552579
Name: "secrets_uri_userinfo_email_overlap",
553580
Description: "#766 matrix C, rendered by #773's rule. On `redis://default:<pw>@cache.orders-svc.io` " +

‎internal/goldencorpus/testdata/golden/secrets_uri_hex_outside_userinfo_decoys.csv‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Filename,Type,Confidence Level,Confidence %,Line Number,Text

‎internal/goldencorpus/testdata/golden/secrets_uri_hex_outside_userinfo_decoys.gitlab-sast.json‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
{
2+
"remediations": [],
3+
"scan": {
4+
"analyzer": {
5+
"id": "ferret-scan",
6+
"name": "Ferret Scan",
7+
"url": "https://github.com/awslabs/ferret-scan",
8+
"vendor": {
9+
"name": "Amazon Web Services"
10+
},
11+
"version": "0.0.0-development"
12+
},
13+
"end_time": "<TIMESTAMP>",
14+
"scanner": {
15+
"id": "ferret-scan",
16+
"name": "Ferret Scan",
17+
"url": "https://github.com/awslabs/ferret-scan",
18+
"vendor": {
19+
"name": "Amazon Web Services"
20+
},
21+
"version": "0.0.0-development"
22+
},
23+
"start_time": "<TIMESTAMP>",
24+
"status": "success",
25+
"type": "sast"
26+
},
27+
"version": "15.0.4",
28+
"vulnerabilities": []
29+
}

‎internal/goldencorpus/testdata/golden/secrets_uri_hex_outside_userinfo_decoys.json.json‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
{
2+
"results": []
3+
}

‎internal/goldencorpus/testdata/golden/secrets_uri_hex_outside_userinfo_decoys.junit.xml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<testsuites name="ferret-scan" tests="0" failures="0" errors="0" time="0.000">
3+
<testsuite name="security-scan" tests="0" failures="0" errors="0" time="0.000"></testsuite>
4+
</testsuites>

‎internal/goldencorpus/testdata/golden/secrets_uri_hex_outside_userinfo_decoys.redact_format_preserving.txt‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
=== REDACTED ===
2+
CACHE_URL=redis://cache.orders.invalid:6379/3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e
3+
ASSET_URL=https://cdn.orders.invalid/assets/3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e/app.js
4+
https://cdn.orders.invalid/img/3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e@2x.png
5+
https://git.orders.invalid/orders/app/commit/8b1e4d7a2c5f9e0b3a6d8c1f4e7b0a9d2c5e8f1a
6+
http://3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e.cache.orders.invalid:6379/0
7+
https://registry.orders.invalid/v2/app/blobs/sha256:c4e7a1d9b3f6082e5a7c9d1b4f6e8a0c2d5b7e9f1a3c6e8b0d2f4a6c8e1b3d5f
8+
REDIS_URL=redis://cache.orders.invalid:6379/0#3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e
9+
https://api.orders.invalid/v1/objects?etag=3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e
10+
ssh://git@git.orders.invalid/orders/8b1e4d7a2c5f9e0b3a6d8c1f4e7b0a9d2c5e8f1a.git
11+
=== FINDINGS (sorted) ===

‎internal/goldencorpus/testdata/golden/secrets_uri_hex_outside_userinfo_decoys.redact_simple.txt‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
=== REDACTED ===
2+
CACHE_URL=redis://cache.orders.invalid:6379/3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e
3+
ASSET_URL=https://cdn.orders.invalid/assets/3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e/app.js
4+
https://cdn.orders.invalid/img/3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e@2x.png
5+
https://git.orders.invalid/orders/app/commit/8b1e4d7a2c5f9e0b3a6d8c1f4e7b0a9d2c5e8f1a
6+
http://3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e.cache.orders.invalid:6379/0
7+
https://registry.orders.invalid/v2/app/blobs/sha256:c4e7a1d9b3f6082e5a7c9d1b4f6e8a0c2d5b7e9f1a3c6e8b0d2f4a6c8e1b3d5f
8+
REDIS_URL=redis://cache.orders.invalid:6379/0#3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e
9+
https://api.orders.invalid/v1/objects?etag=3f9a1c7e5b2d4f6a8c0e1b3d5f7a9c2e
10+
ssh://git@git.orders.invalid/orders/8b1e4d7a2c5f9e0b3a6d8c1f4e7b0a9d2c5e8f1a.git
11+
=== FINDINGS (sorted) ===

‎internal/goldencorpus/testdata/golden/secrets_uri_hex_outside_userinfo_decoys.sarif.json‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/refs/heads/main/sarif-2.1/schema/sarif-schema-2.1.0.json",
3+
"runs": [
4+
{
5+
"results": [],
6+
"tool": {
7+
"driver": {
8+
"informationUri": "https://github.com/awslabs/ferret-scan",
9+
"name": "Ferret Scan",
10+
"semanticVersion": "0.0.0-development",
11+
"version": "0.0.0-development"
12+
}
13+
}
14+
}
15+
],
16+
"version": "2.1.0"
17+
}

‎internal/goldencorpus/testdata/golden/secrets_uri_hex_outside_userinfo_decoys.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
No matches found.

0 commit comments

Comments
 (0)