v8.30.0

Changelog

• 6eaad03 0 to 5 - notes on recursive decoding (#1994)
• 09242ce Add new Looker client ID and client secret rules (#1947)
• c98e5e0 feat: add Airtable Personnal Access Token detection (#1952)
• 4ed0ca4 build: upgrade Go & alpine version (#1989)

v8.29.1

Changelog

• fb5d707 thats a paddlin
• 50493db feat: document stdout report path (#1990)

v8.29.0

Changelog

• ed65b65 Add trace log for skipped archive file when not enabled (#1961)
• c5ccbb9 Respect contexts with timeouts (#1948)
• 3821f30 Config min version (#1955)
• d223718 fix(config): validate rules when [extend] is used (#1592)
• 87d9629 feat: add Amazon Bedrock API key detection (#1935)
• 228396b Add GitHub Sponsors section and Discord link
• a82bc53 feat: improve regex to detect Sonar tokens with prefixes (#1931)

v8.28.0

Changelog

• 4fb4382 cant count
• b1c9c7e Composite rules (#1905)
• 72977e4 feat: add Anthropic API key detection (#1910)
• 7b02c98 fix(git): handle port (#1912)
• 2a7bcff dont prematurely calculate fragment newlines (#1909)
• bd79c3e feat(allowlist): promote optimizations (#1908)
• 7fb4eda Fix: CVEs on go and go crypto (#1868)
• a044b81 feat: add artifactory reference token and api key detection (#1906)
• bf380d4 silly
• f487f85 Update gitleaks.yml
• 958f55a add just like that, no leaks

Optimizations

#1909 waits to find newlines until a match. This ends up saving a boat load of time since before we were finding newlines for every fragment regardless if a rule matched or not.
#1908 promoted @rgmz excellent stopword optimization

Composite Rules (Multi-part or required Rules) #1905

In v8.28.0 Gitleaks introduced composite rules, which are made up of a single "primary" rule and one or more auxiliary or required rules. To create a composite rule, add a [[rules.required]] table to the primary rule specifying an id and optionally withinLines and/or withinColumns proximity constraints. A fragment is a chunk of content that Gitleaks processes at once (typically a file, part of a file, or git diff), and proximity matching instructs the primary rule to only report a finding if the auxiliary required rules also find matches within the specified area of the fragment.

Proximity matching: Using the withinLines and withinColumns fields instructs the primary rule to only report a finding if the auxiliary required rules also find matches within the specified proximity. You can set:

• withinLines: N - required findings must be within N lines (vertically)
• withinColumns: N - required findings must be within N characters (horizontally)
• Both - creates a rectangular search area (both constraints must be satisfied)
• Neither - fragment-level matching (required findings can be anywhere in the same fragment)

Here are diagrams illustrating each proximity behavior:

p = primary captured secret
a = auxiliary (required) captured secret
fragment = section of data gitleaks is looking at


    *Fragment-level proximity*               
    Any required finding in the fragment
          ┌────────┐                       
   ┌──────┤fragment├─────┐                 
   │      └──────┬─┤     │ ┌───────┐       
   │             │a│◀────┼─│✓ MATCH│       
   │          ┌─┐└─┘     │ └───────┘       
   │┌─┐       │p│        │                 
   ││a│    ┌─┐└─┘        │ ┌───────┐       
   │└─┘    │a│◀──────────┼─│✓ MATCH│       
   └─▲─────┴─┴───────────┘ └───────┘       
     │    ┌───────┐                        
     └────│✓ MATCH│                        
          └───────┘                        
                                           
                                           
   *Column bounded proximity*
   `withinColumns = 3`                    
          ┌────────┐                       
   ┌────┬─┤fragment├─┬───┐                 
   │      └──────┬─┤     │ ┌───────────┐   
   │    │        │a│◀┼───┼─│+1C ✓ MATCH│   
   │          ┌─┐└─┘     │ └───────────┘   
   │┌─┐ │     │p│    │   │                 
┌──▶│a│  ┌─┐  └─┘        │ ┌───────────┐   
│  │└─┘ ││a│◀────────┼───┼─│-2C ✓ MATCH│   
│  │       ┘             │ └───────────┘   
│  └── -3C ───0C─── +3C ─┘                 
│  ┌─────────┐                             
│  │ -4C ✗ NO│                             
└──│  MATCH  │                             
   └─────────┘                             
                                           
                                           
   *Line bounded proximity*
   `withinLines = 4`                      
         ┌────────┐                        
   ┌─────┤fragment├─────┐                  
  +4L─ ─ ┴────────┘─ ─ ─│                  
   │                    │                  
   │              ┌─┐   │ ┌────────────┐   
   │         ┌─┐  │a│◀──┼─│+1L ✓ MATCH │   
   0L  ┌─┐   │p│  └─┘   │ ├────────────┤   
   │   │a│◀──┴─┴────────┼─│-1L ✓ MATCH │   
   │   └─┘              │ └────────────┘   
   │                    │ ┌─────────┐      
  -4L─ ─ ─ ─ ─ ─ ─ ─┌─┐─│ │-5L ✗ NO │      
   │                │a│◀┼─│  MATCH  │      
   └────────────────┴─┴─┘ └─────────┘      
                                           
                                           
   *Line and column bounded proximity*
   `withinLines = 4`                      
   `withinColumns = 3`                    
         ┌────────┐                        
   ┌─────┤fragment├─────┐                  
  +4L   ┌└────────┴ ┐   │                  
   │            ┌─┐     │ ┌───────────────┐
   │    │       │a│◀┼───┼─│+2L/+1C ✓ MATCH│
   │         ┌─┐└─┘     │ └───────────────┘
   0L   │    │p│    │   │                  
   │         └─┘        │                  
   │    │           │   │ ┌────────────┐   
  -4L    ─ ─ ─ ─ ─ ─┌─┐ │ │-5L/+3C ✗ NO│   
   │                │a│◀┼─│   MATCH    │   
   └───-3C────0L───+3C┴─┘ └────────────┘   

v8.27.2

Changelog

• c7acf33 Merge branch 'master' of github.com:gitleaks/gitleaks
• 9faaa4a Add experimental allowlist optimizations (#1731)
• 79068b3 Detect Notion Public API Keys #1889 (#1890)

v8.27.1

Changelog

• 80468ef Merge branch 'master' of github.com:gitleaks/gitleaks
• ef82237 fix(atlassian): reduce false-positives for v1 pattern (#1892)
• 2463f11 Fix log suppresion issue (#1887)
• 6f251ee Added Heroku API Key New Version (#1883)
• 20f9a1d Add Platform Bitbucket (#1886)
• 722ce82 Add Platform Gitea (#1884)
• 79780b8 Merge branch 'master' of github.com:gitleaks/gitleaks
• c5683ca prevent default warn message when max-archive-depth not set (#1881)
• 0357c3c prevent default warn message when max-archive-depth not set

v8.27.0

Changelog

• 782f310 Archive support (#1872)
• 489d13c Update README.md
• d29ee55 Reduce aws-access-token false positives (#1876)
• 611db65 Set pass_filenames to false for Docker hook (#1850)
• 0589ae0 unicode decoding (#1854)
• 82f7e32 Diagnostics (#1856)
• f97a9ee chore: include decoder in debug log (#1853)

Got another @bplaxco release. Cheers!

Archive Scanning

Sometimes secrets are packaged within archive files like zip files or tarballs,
making them difficult to discover. Now you can tell gitleaks to automatically
extract and scan the contents of archives. The flag --max-archive-depth
enables this feature for both dir and git scan types. The default value of
"0" means this feature is disabled by default.

Recursive scanning is supported since archives can also contain other archives.
The --max-archive-depth flag sets the recursion limit. Recursion stops when
there are no new archives to extract, so setting a very high max depth just
sets the potential to go that deep. It will only go as deep as it needs to.

The findings for secrets located within an archive will include the path to the
file inside the archive. Inner paths are separated with !.

Example finding (shortened for brevity):

Finding:     DB_PASSWORD=8ae31cacf141669ddfb5da
...
File:        testdata/archives/nested.tar.gz!archives/files.tar!files/.env.prod
Line:        4
Commit:      6e6ee6596d337bb656496425fb98644eb62b4a82
...
Fingerprint: 6e6ee6596d337bb656496425fb98644eb62b4a82:testdata/archives/nested.tar.gz!archives/files.tar!files/.env.prod:generic-api-key:4
Link:        https://github.com/leaktk/gitleaks/blob/6e6ee6596d337bb656496425fb98644eb62b4a82/testdata/archives/nested.tar.gz

This means a secret was detected on line 4 of files/.env.prod. which is in
archives/files.tar which is in testdata/archives/nested.tar.gz.

Currently supported formats:

The compression
and archive
formats supported by mholt's archives package
are supported.

v8.26.0

Changelog

• 78eebac Percent/URL Decoding Support (#1831)
• 6f967ca fix(kubernetes): remove slow element from pat (#1848)
• 88f56d3 feat: identify slow file (#1479)
• 9609928 rm 1password detect test since we test it in cfg gen
• 23cb69f feat(rules): Add 1Password secret key detection (#1834)

Calling this one @bplaxco's release as he introduced a really clever method for mixed decoding without sacrificing too much performance. As I stated in his PR, I think he's either a wizard or some time traveling AI. Dude is wicked smaht

Anyways, Gitleaks now supports the following decoders: hex, percent(url enconding), and b64. It's relatively straight forward to add a new decoder so if you're motivated, community contributions are welcomed!

Here's an example:

~/code/gitleaks-org/gitleaks (master) cat decode.txt
text below
aGVsbG8sIHdvcmxkIQ%3D%3D%0A
text above
~/code/gitleaks-org/gitleaks (master) ./gitleaks dir decode.txt --max-decode-depth=2 --log-level=debug

    ○
    │╲
    │ ○
    ○ ░
    ░    gitleaks

4:08PM DBG using stdlib regex engine
4:08PM DBG unable to load gitleaks config from decode.txt/.gitleaks.toml since --source=decode.txt is a file, using default config
4:08PM DBG found .gitleaksignore file: .gitleaksignore
4:08PM DBG segment found: original=[29,38] pos=[29,38]: "%3D%3D%0A" -> "==\n"
4:08PM DBG segment found: original=[11,38] pos=[11,31]: "aGVsbG8sIHdvcmxkIQ==" -> "hello, world!"
4:08PM INF scanned ~50 bytes (50 bytes) in 1.5ms
4:08PM INF no leaks found

v8.25.1

Changelog

• d1c7759 fix(detect): test all allowlists (#1845)

Big thanks @rgmz

v8.25.0

Changelog

• 4451b45 feat(config): define multiple global allowlists (#1777) (cause for the minor bump change)
• 7fb21a4 feat(rules): Add Perplexity AI API key detection (#1825)
• f6193bc feat(gcp): increase rule entropy (#1840)
• 9bc7257 Adding clickhouse scanner (#1826)
• b6cc71a fix(baseline): work with --redact (#1741)
• cfdeb0d feat(rule): validate & sort rule when generating (#1817)