Audit log for blocked transactions missing Portion K population.

[jeremyjpj0916]

Currently setting:

SecAuditLogParts ABCJDHKZ

Based on table I should be seeing this kinda info:

Part letter	Description
A	Audit log header (mandatory)
B	Request headers
C	Request body
D	Reserved
E	Response body
F	Response headers
G	Reserved
H	Audit log trailer, which contains additional data
I	Compact request body alternative (to part C), which excludes files
J	Information on uploaded files (available as of version 2.6.0)
K	Contains a list of all rules that matched for the transaction
Z	Final boundary (mandatory)

On a given random blocked transaction I ended up getting this log:

/tmp/audit/20200213/20200213-2334 $ cat 20200213-233405-158163684568.835438

---FeKj7tVW---A--
[13/Feb/2020:23:34:05 +0000] 158163684568.835438 10.94.74.158 0 10.128.93.29 8443
---FeKj7tVW---B--
POST /auth/oauth2/token HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: multipart/form-data; boundary="----=_Part_135_1869748281.1581636845371"
MIME-Version: 1.0
Content-Length: 656
Host: gateway.company.com
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
X-Forwarded-For: 10.94.74.158
 
---FeKj7tVW---C--
 
------=_Part_135_1869748281.1581636845371
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: form-data; name="client_id"
 
XXXXXXXXXXXXXXXXXXXXXXX
------=_Part_135_1869748281.1581636845371
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: form-data; name="client_secret"
 
XXXXXXXXXXXXXXXXXXXXXXX
------=_Part_135_1869748281.1581636845371
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: form-data; name="grant_type"
 
client_credentials
------=_Part_135_1869748281.1581636845371--
 
 
---FeKj7tVW---D--
 
 
---FeKj7tVW---H--
ModSecurity: Warning. Matched "Operator `Rx' with parameter `^[\w\d/\.\-\+]+(?:\s?;\s?(?:boundary|charset)\s?=\s?['\"\w\d\.\-]+)?$' against variable `REQUEST_HEADERS:Content-Type' (Value: `multipart/form-data; boundary="----=_Part_135_1869748281.1581636845371"' ) [file "/usr/local/owasp-modsecurity-crs-3.2.0/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "870"] [id "920470"] [rev""] [msg "Illegal Content-Type header"] [data "multipart/form-data; boundary="----=_part_135_1869748281.1581636845371""] [severity "2"] [ver "OWASP_CRS/3.2.0"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/CONTENT_TYPE"] [tag "WASCTC/WASC-20"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/EE2"] [tag "PCI/12.1"] [hostname "10.128.93.29"] [uri "/auth/oauth2/token"] [unique_id "158163684568.835438"] [ref "v77,71t:lowercase"]
ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Ge' with parameter `5' against variable `TX:ANOMALY_SCORE' (Value: `5' ) [file "/usr/local/owasp-modsecurity-crs-3.2.0/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "79"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [data ""] [severity "2"] [ver ""] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "10.128.93.29"] [uri "/auth/oauth2/token"] [unique_id "158163684568.835438"] [ref ""]
 
---FeKj7tVW---J--
 
---FeKj7tVW---K--
 
---FeKj7tVW---Z--

Luckily Section H does have all the details from the tx it seems so audit log was bringing value because it seems like H is made up of two parts strung together Part 1: New info around the block that does NOT get printed out to NGINX stderr + Just the stderr print statement I see in typical logs when audit is disabled. Section H luckily does including the CRS error code: 920470 in that top half, which after reviewing the 3.2/master branch and the 3.3/dev branch of CRS it showed they had fixed up the regex of the rule in a way that my Content-Type header was no longer being blocked! Which was awesome to just patch that rule over and continue testing and finally generate a valid OAuth2.0 token not getting blocked by the WAF.

My ask here would be ModSec does parse and grab all the matching id's that lead to a block in section K cleanly printed, makes for easy copy paste searching in the repo's to figure out what rules to inspect for FP's and aligns with the current documentation on all the fields.

Note, I had to use the pending outstanding PR fix to capture blocked tx's in the audit log. Glad that was out there or I woulda been rekt, can't use a WAF without some form of audit inspection for blockings.

Version: Master branch right now of the ngx connector + libmodsec 3.0.4

[jeremyjpj0916]

Other Q's I have are what are all the values in the audit log reference file when in Concurrent if someone can link me to any further docs on some of the not straightforward values?

Example

gateway.company.com 10.96.3.174 - [14/Feb/2020:09:05:02 +0000] "GET /F5/status/derp?v=/bin/bash HTTP/1.1" 403 118 - "PostmanRuntime/7.6.1" 158167110219.631104 - /tmp/audit/20200214/20200214-0905/20200214-090502-158167110219.631104 0 2218.000000 md5:eb2855af2622ae9650492d24c13a9087

Mostly curious what the 118 is, the 0, and the 2218.00000(microsecond or millisecond to execute the checks?)

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[jeremyjpj0916]

still a nice to have thing working in the audit logs.

[victorhora]

Sorry @jeremyjpj0916. The "nostale" tag has been set for this one and it's now reopened. We'll get to this one when possible. Thank you

[zimmerle]

The K part of the audit log is missing the implementation -

https://github.com/SpiderLabs/ModSecurity/blob/a1a8c0fda77e24f59de637b725e818804cf02a10/src/transaction.cc#L1609-L1613

It is marked as TODO on the code.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[zimmerle]

ping.