Skip to content

feat: add host, db and username to ExtraData for database detectors#4849

Open
mariocj89 wants to merge 1 commit into
trufflesecurity:mainfrom
mariocj89:add-db-host-username-extradata
Open

feat: add host, db and username to ExtraData for database detectors#4849
mariocj89 wants to merge 1 commit into
trufflesecurity:mainfrom
mariocj89:add-db-host-username-extradata

Conversation

@mariocj89
Copy link
Copy Markdown

@mariocj89 mariocj89 commented Mar 30, 2026

First time contributing to trufflehog, let me know if I should do anything differently. Thanks for your work ^^.

Fixes #4754 and helps with general triaging and identification for other database connection types :).

Description:

Populate ExtraData with parsed fields for all database connection string detectors (MongoDB, PostgreSQL, Redis, JDBC). This surfaces useful metadata about detected credentials.

The parsing logic already existed in each detector — this change exposes the extracted values in the result's ExtraData map alongside any pre-existing fields (rotation_guide, sslmode, etc.).

Checklist:

  • Tests passing (make test-community)?
  • Lint passing (make lint this requires golangci-lint)?

Manual test

Example file
mcorcherojim at PW0ACHMB in ~
$ cat /tmp/fake_secrets.txt
# MongoDB with database
mongodb://admin:SuperSecret123@mongo.prod.example.com:27017/customers

# PostgreSQL with database and sslmode
postgresql://dbadmin:P4ssw0rd!@pg-primary.internal:5432/analytics?sslmode=require

# Redis with username
redis://cacheuser:R3d1sP4ss@redis-cluster.example.com:6379/0

# JDBC MySQL
jdbc:mysql://appuser:MyS3cret@mysql-db.example.com:3306/orders

# JDBC PostgreSQL
jdbc:postgresql://etluser:Etl_Pass_99@warehouse.example.com:5432/datawarehouse

# JDBC SQL Server
jdbc:sqlserver://sqlbox.corp.local:1433;database=inventory;user=sa;password=S4_Admin!

# Azure Redis (no username)
myapp-cache.redis.cache.windows.net:6380,password=aB1cD2eF3gH4iJ5kL6mN7oP8qR9sT0uV1wX2yZ3a4b5,ssl=True,abortConnect=False

# MongoDB without database
mongodb://readonly:ViewOnly_2024@analytics.mongodb.example.com:27017
Example output

$ trufflehog filesystem --directory /tmp/fake_secrets.txt --no-verification 2>/dev/null
Found unverified result 🐷🔑❓
Detector Type: MongoDB
Decoder Type: PLAIN
Raw result: mongodb://readonly:ViewOnly_2024@analytics.mongodb.example.com:27017
Rotation_guide: https://howtorotate.com/docs/tutorials/mongo/
Host: analytics.mongodb.example.com:27017
Username: readonly
File: /tmp/fake_secrets.txt
Line: 23

Found unverified result 🐷🔑❓
Detector Type: MongoDB
Decoder Type: PLAIN
Raw result: mongodb://admin:SuperSecret123@mongo.prod.example.com:27017/customers
Rotation_guide: https://howtorotate.com/docs/tutorials/mongo/
Host: mongo.prod.example.com:27017
Username: admin
Database: customers
File: /tmp/fake_secrets.txt
Line: 2

Found unverified result 🐷🔑❓
Detector Type: Postgres
Decoder Type: PLAIN
Raw result: postgresql://etluser:Etl_Pass_99@warehouse.example.com:5432
Sslmode: <unset>
Host: warehouse.example.com
Username: etluser
Database: datawarehouse
File: /tmp/fake_secrets.txt
Line: 14

Found unverified result 🐷🔑❓
Detector Type: Postgres
Decoder Type: PLAIN
Raw result: postgresql://dbadmin:P4ssw0rd!@pg-primary.internal:5432
Sslmode: require
Host: pg-primary.internal
Username: dbadmin
Database: analytics
File: /tmp/fake_secrets.txt
Line: 5

Found unverified result 🐷🔑❓
Detector Type: JDBC
Decoder Type: PLAIN
Raw result: jdbc:sqlserver://sqlbox.corp.local:1433;database=inventory;user=sa;password=S4_Admin!
Host: sqlbox.corp.local:1433
Username: sa
Database: inventory
File: /tmp/fake_secrets.txt
Line: 17

Found unverified result 🐷🔑❓
Detector Type: JDBC
Decoder Type: PLAIN
Raw result: jdbc:mysql://appuser:MyS3cret@mysql-db.example.com:3306/orders
Host: tcp(mysql-db.example.com:3306)
Username: appuser
Database: orders
File: /tmp/fake_secrets.txt
Line: 11

Found unverified result 🐷🔑❓
Detector Type: JDBC
Decoder Type: PLAIN
Raw result: jdbc:postgresql://etluser:Etl_Pass_99@warehouse.example.com:5432/datawarehouse
Host: warehouse.example.com:5432
Username: etluser
Database: datawarehouse
File: /tmp/fake_secrets.txt
Line: 14

Note

Low Risk
Low risk: changes only add parsed metadata to Result.ExtraData for several database detectors and adjust tests accordingly, without altering matching patterns or verification logic.

Overview
Adds parsed connection metadata to findings for the JDBC, MongoDB, Postgres, and Redis detectors by populating Result.ExtraData with host, username, and (where applicable) database alongside existing fields like sslmode/rotation_guide.

For JDBC, parsing is now attempted even when verify=false so ExtraData is available for unverified results, while verification behavior remains the same (parse failures still skip only when verify=true). Tests are updated to ignore the new fields in integration diffs and new unit tests are added to assert the populated ExtraData for each detector.

Reviewed by Cursor Bugbot for commit ed0e490. Bugbot is set up for automated code reviews on this repo. Configure here.

@mariocj89 mariocj89 requested review from a team and Copilot March 30, 2026 16:23
@mariocj89 mariocj89 requested a review from a team as a code owner March 30, 2026 16:23
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 30, 2026

CLA assistant check
All committers have signed the CLA.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enhances database connection string detectors by surfacing parsed connection metadata (host, username, database) via Result.ExtraData, improving downstream triage and identification of detected credentials.

Changes:

  • Redis: populate ExtraData from the parsed Redis URL.
  • Postgres: add host, username, and database into ExtraData while preserving existing sslmode.
  • MongoDB + JDBC: expose parsed connection fields in ExtraData and add focused tests validating the new metadata.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
pkg/detectors/redis/redis.go Adds ExtraData population from parsed Redis URLs via a helper.
pkg/detectors/redis/redis_test.go Adds test coverage asserting host/username in ExtraData.
pkg/detectors/postgres/postgres.go Extends existing ExtraData to include host/username/database alongside sslmode.
pkg/detectors/postgres/postgres_test.go Adds tests validating new Postgres ExtraData fields and preserving sslmode.
pkg/detectors/mongodb/mongodb.go Preserves rotation_guide and adds host/user/database fields into ExtraData.
pkg/detectors/mongodb/mongodb_test.go Adds tests verifying MongoDB ExtraData fields are populated.
pkg/detectors/jdbc/jdbc.go Parses JDBC connection info to populate ExtraData even when verify=false.
pkg/detectors/jdbc/jdbc_test.go Adds tests covering ExtraData extraction and unsupported subprotocol behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/detectors/mongodb/mongodb_test.go
Comment thread pkg/detectors/redis/redis.go
Comment thread pkg/detectors/jdbc/jdbc.go
Comment thread pkg/detectors/redis/redis_test.go
Comment thread pkg/detectors/redis/redis.go
Comment thread pkg/detectors/jdbc/jdbc.go Outdated
@mariocj89 mariocj89 force-pushed the add-db-host-username-extradata branch 2 times, most recently from 2ee0783 to 252be3e Compare March 30, 2026 22:34
Copy link
Copy Markdown
Contributor

@MuneebUllahKhan222 MuneebUllahKhan222 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mariocj89,
Thank you for the contribution. This looks good to me, please update the integration tests to align with the changes in ExtraData.

@mariocj89
Copy link
Copy Markdown
Author

mariocj89 commented May 4, 2026

@MuneebUllahKhan222 does this still make sense given the recent addition of SecretParts? Are SecretParts returned and if so, does it make sense to return this in extraData?

Happy to go either direction:
a) SecretParts is for analysers, extraData for anything else and this PR still makes sense.
b) Things should all go towards SecretsParts, I'll update my pr to add if there is anything missing (need to check)

@MuneebUllahKhan222
Copy link
Copy Markdown
Contributor

@MuneebUllahKhan222 does this still make sense given the recent addition of SecretParts? Are SecretParts returned and if so, does it make sense to return this in extraData?

Happy to go either direction: a) SecretParts is for analysers, extraData for anything else and this PR still makes sense. b) Things should all go towards SecretsParts, I'll update my pr to add if there is anything missing (need to check)

I believe option (a) makes more sense. SecretParts is aimed at analysers, while ExtraData is used for other detector-specific metadata.

@mariocj89 mariocj89 force-pushed the add-db-host-username-extradata branch from 252be3e to 6ef0047 Compare May 5, 2026 07:25
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Reviewed by Cursor Bugbot for commit 6ef0047. Configure here.

Comment thread pkg/detectors/jdbc/jdbc.go
@mariocj89 mariocj89 force-pushed the add-db-host-username-extradata branch 2 times, most recently from cdaaa82 to c5dc121 Compare May 5, 2026 07:36
@mariocj89
Copy link
Copy Markdown
Author

@MuneebUllahKhan222 makes total sense! Everything in extraData should be non-sensitive info as well. Should be good for another review!

Copy link
Copy Markdown
Contributor

@MuneebUllahKhan222 MuneebUllahKhan222 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’ve added a few comments.

Additionally, we should account for changes in ExtraData within the integration tests. These fields can be safely excluded from comparisons by updating the test configuration as follows:

ignoreOpts := cmpopts.IgnoreFields(detectors.Result{}, "SecretParts", "ExtraData")

Comment thread pkg/detectors/jdbc/jdbc_test.go
Comment thread pkg/detectors/mongodb/mongodb_test.go Outdated
Comment thread pkg/detectors/postgres/postgres_test.go Outdated
@mariocj89 mariocj89 force-pushed the add-db-host-username-extradata branch from c5dc121 to 7cc5bc3 Compare May 5, 2026 10:55
Comment thread pkg/detectors/mongodb/mongodb_integration_test.go
Copy link
Copy Markdown
Contributor

@MuneebUllahKhan222 MuneebUllahKhan222 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. Just one non-blocking comment after that we will need a quick product-eng review before we can merge.

Comment thread pkg/detectors/mongodb/mongodb_test.go Outdated
@mariocj89 mariocj89 force-pushed the add-db-host-username-extradata branch from 7cc5bc3 to 2f3b64d Compare May 5, 2026 12:08
@MuneebUllahKhan222 MuneebUllahKhan222 added the review/product-eng Team integrations reviewed, awaiting product-eng review label May 5, 2026
@MuneebUllahKhan222
Copy link
Copy Markdown
Contributor

Hi @mariocj89,
Could you please resolve the conflicts so we can get this merged this week?

Populate ExtraData with parsed fields for all
database connection string detectors (MongoDB, PostgreSQL, Redis, JDBC).
This surfaces useful metadata about detected credentials.

The parsing logic already existed in each detector — this change
exposes the extracted values in the result's ExtraData map alongside
any pre-existing fields (rotation_guide, sslmode, etc.).
@mariocj89 mariocj89 force-pushed the add-db-host-username-extradata branch from 2f3b64d to ed0e490 Compare May 11, 2026 11:09
@mariocj89
Copy link
Copy Markdown
Author

@MuneebUllahKhan222 rebased!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/product-eng Team integrations reviewed, awaiting product-eng review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Include postgres parameters somewhere

5 participants