Skip to content
This repository was archived by the owner on May 14, 2020. It is now read-only.

tx.paranoia_level on v3/master ignored #1127

Closed
theMiddleBlue opened this issue Jun 15, 2018 · 13 comments
Closed

tx.paranoia_level on v3/master ignored #1127

theMiddleBlue opened this issue Jun 15, 2018 · 13 comments
Labels
ModSec Issue related to ModSecurity

Comments

@theMiddleBlue
Copy link
Contributor

Hi,

I've an odd behavior with the last v3/master version (I've just pulled it). It seems that it ignores at all the setvar:tx.paranoia_level=1 and it matches many rules in PL2, 3 and 4.

My configuration:

SecAction \
  "id:900000,\
   phase:1,\
   nolog,\
   pass,\
   t:none,\
   setvar:tx.paranoia_level=1"

And on audit log I've many of these (with my previous version of v3/master it wasn't happen):

{
  "details": {
    "accuracy": "0",
    "rev": "",
    "ver": "",
    "reference": "v1839,1",
    "lineNumber": "18",
    "match": "Matched \"Operator `Eq' with parameter `0' against variable `MULTIPART_UNMATCHED_BOUNDARY' (Value: `2' )",
    "tags": [],
    "ruleId": "200004",
    "maturity": "0",
    "data": "",
    "severity": "0",
    "file": "/usr/local/openresty/nginx/conf/modsecurity_config/http-www.acselspa.it/http-www.acselspa.it"
  },
  "message": "Multipart parser detected a possible unmatched boundary."
},
{
  "details": {
    "accuracy": "9",
    "rev": "2",
    "ver": "OWASP_CRS/3.0.0",
    "reference": "o40,1o41,1o87,1o88,1o89,1o90,1o110,1o111,1o152,1o153,1o201,1o202,1o203,1o204,1o215,1o216,1o257,1o258,1o305,1o306,1o307,1o308,1o313,1o314,1o355,1o356,1o412,1o413,1o414,1o415,1o428,1o429,1o472,1o473,1v1839,474t:urlDecodeUni",
    "lineNumber": "1287",
    "match": "Matched \"Operator `ValidadeByteRange' with parameter `32-36,38-126' against variable `REQUEST_BODY' (Value: `------WebKitFormBoundaryWWc2IeyPuspjWwJ5\\x0d\\x0aContent-Disposition: form-data; name=\"action\"\\x0d\\x0 (476 characters omitted)' )",
    "tags": [
      "application-multi",
      "language-multi",
      "platform-multi",
      "attack-protocol",
      "OWASP_CRS/PROTOCOL_VIOLATION/EVASION",
      "paranoia-level/3"
    ],
    "ruleId": "920272",
    "maturity": "9",
    "data": "",
    "severity": "2",
    "file": "/usr/local/openresty/nginx/conf/owasp-modsecurity-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"
  },
  "message": "Invalid character in request (outside of printable chars below ascii 127)"
},
{
  "details": {
    "accuracy": "9",
    "rev": "2",
    "ver": "OWASP_CRS/3.0.0",
    "reference": "o40,1o41,1o62,1o72,1o73,1o79,1o86,1o87,1o88,1o89,1o90,1o110,1o111,1o152,1o153,1o174,1o184,1o185,1o191,1o200,1o201,1o202,1o203,1o204,1o215,1o216,1o257,1o258,1o279,1o289,1o290,1o296,1o304,1o305,1o306,1o307,1o308,1o313,1o314,1o355,1o356,1o377,1o387,1o388,1o394,1o411,1o412,1o413,1o414,1o415,1o428,1o429,1o472,1o473,1v1839,474t:urlDecodeUni",
    "lineNumber": "1348",
    "match": "Matched \"Operator `ValidadeByteRange' with parameter `38,44-46,48-58,61,65-90,95,97-122' against variable `REQUEST_BODY' (Value: `------WebKitFormBoundaryWWc2IeyPuspjWwJ5\\x0d\\x0aContent-Disposition: form-data; name=\"action\"\\x0d\\x0 (476 characters omitted)' )",
    "tags": [
      "application-multi",
      "language-multi",
      "platform-multi",
      "attack-protocol",
      "OWASP_CRS/PROTOCOL_VIOLATION/EVASION",
      "paranoia-level/4"
    ],
    "ruleId": "920273",
    "maturity": "9",
    "data": "",
    "severity": "2",
    "file": "/usr/local/openresty/nginx/conf/owasp-modsecurity-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"
  },
  "message": "Invalid character in request (outside of very strict set)"
},
{
  "details": {
    "accuracy": "8",
    "rev": "2",
    "ver": "OWASP_CRS/3.0.0",
    "reference": "o2,8o2,8v1931,19t:urlDecodeUni",
    "lineNumber": "1251",
    "match": "Matched \"Operator `Rx' with parameter `((?:[\\~\\!\\@\\#\\$\\%\\^\\&\\*\\(\\)\\-\\+\\=\\{\\}\\[\\]\\|\\:\\;\\\"\\'\\´\\�\\�\\`\\<\\>][^\\~\\!\\@\\#\\$\\%\\^\\&\\*\\(\\)\\-\\+\\=\\{\\}\\[\\]\\|\\:\\;\\\"\\'\\´\\�\\�\\`\\<\\>]*?){2})' against variable `ARGS:action' (Value: `wp-remove-post-lock' )",
    "tags": [
      "application-multi",
      "language-multi",
      "platform-multi",
      "attack-sqli",
      "OWASP_CRS/WEB_ATTACK/SQL_INJECTION",
      "WASCTC/WASC-19",
      "OWASP_TOP_10/A1",
      "OWASP_AppSensor/CIE1",
      "PCI/6.5.2",
      "paranoia-level/4"
    ],
    "ruleId": "942432",
    "maturity": "9",
    "data": "Matched Data: -remove- found within ARGS:action: wp-remove-post-lock",
    "severity": "4",
    "file": "/usr/local/openresty/nginx/conf/owasp-modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf"
  },
  "message": "Restricted SQL Character Anomaly Detection (args): # of special characters exceeded (2)"
}

Any idea? how can I do for pull the previous version of v3/master?

Thanks!

@dune73
Copy link
Contributor

dune73 commented Jun 15, 2018

Thank you for reporting @theMiddleBlue. And sorry for the inconvenience.

This sounds annoying indeed. Could you raise the ModSec DebugLogLevel and show us the detailed debug log for 920014, please? There we should see the actual paranoia level and the skipping to the end of the 920xxx rule file. Your alerts indicate this rule not working properly.

@theMiddleBlue
Copy link
Contributor Author

Hi @dune73,

Following a grep by 920014

[4] (Rule: 920014) Executing operator "Lt" with param "2" against TX:PARANOIA_LEVEL.

and here with 5 lines before and after:

# tail -n 0 -f /var/log/modsec-debug.log | grep -A5 -B5 '920014'
[4] Running [independent] (non-disruptive) action: setvar
[4] Rule returned 1.
[4] Executing chained rule.
[4] (Rule: 0) Executing operator "Within" with param "/proxy/ /lock-token/ /content-range/ /translate/ /if/" Was: "" against TX:regex(^HEADER_NAME_).
[4] Rule returned 0.
[4] (Rule: 920014) Executing operator "Lt" with param "2" against TX:PARANOIA_LEVEL.
[4] Rule returned 0.
[4] (Rule: 920200) Executing operator "Rx" with param "^bytes=((\d+)?\-(\d+)?\s*,?\s*){6}" against REQUEST_HEADERS:Range|REQUEST_HEADERS:Request-Range.
[4] Rule returned 0.
[4] (Rule: 920201) Executing operator "EndsWith" with param ".pdf" against REQUEST_BASENAME.
[4] Rule returned 0.

@csanders-git
Copy link
Contributor

csanders-git commented Jun 15, 2018

A user on irc reported the same thing. He assumed it was this commit owasp-modsecurity/ModSecurity@202a15b

@theMiddleBlue
Copy link
Contributor Author

theMiddleBlue commented Jun 15, 2018

I'm using this request for reproduce it:

curl -d 'ContentObjectAttribute_ezstring_data_text_436=Name+Name&ContentObjectAttribute_data_text_437=user%40gmail.com&ContentObjectAttribute_ezstring_data_text_2572=3478710812&ContentObjectAttribute_data_text_438=Message&ActionCollectInformation=Invia&ContentNodeID=78&ContentObjectID=80&ViewMode=full' 'http://127.0.0.1/content/action'

Audit Log:

ModSecurity: Warning. Matched "Operator `Rx' with parameter `^[\d.:]+$' against variable `REQUEST_HEADERS:Host' (Value: `127.0.0.1' ) [file "/usr/local/openresty/nginx/conf/owasp-modsecurity-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "733"] [id "920350"] [rev "2"] [msg "Host header is a numeric IP address"] [data "127.0.0.1"] [severity "4"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "9"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/IP_HOST"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] [hostname "172.17.0.1"] [uri "/content/action"] [unique_id "152906589237.129757"] [ref "o0,9v36,9"]
ModSecurity: Warning. Matched "Operator `ValidadeByteRange' with parameter `38,44-46,48-58,61,65-90,95,97-122' against variable `REQUEST_BODY' (Value: `ContentObjectAttribute_ezstring_data_text_436=Name+Name&ContentObjectAttribute_data_text_437=user%40 (193 characters omitted)' ) [file "/usr/local/openresty/nginx/conf/owasp-modsecurity-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "1348"] [id "920273"] [rev "2"] [msg "Invalid character in request (outside of very strict set)"] [data ""] [severity "2"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "9"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/EVASION"] [tag "paranoia-level/4"] [hostname "172.17.0.1"] [uri "/content/action"] [unique_id "152906589237.129757"] [ref "o4,1v197,9t:urlDecodeUnio50,1v151,293t:urlDecodeUni"]

Debug Log:

[4] Running [independent] (non-disruptive) action: setvar
[4] Rule returned 1.
[4] Executing chained rule.
[4] (Rule: 0) Executing operator "Within" with param "/proxy/ /lock-token/ /content-range/ /translate/ /if/" Was: "" against TX:regex(^HEADER_NAME_).
[4] Rule returned 0.
[4] (Rule: 920014) Executing operator "Lt" with param "2" against TX:PARANOIA_LEVEL.
[4] Rule returned 0.
[4] (Rule: 920200) Executing operator "Rx" with param "^bytes=((\d+)?\-(\d+)?\s*,?\s*){6}" against REQUEST_HEADERS:Range|REQUEST_HEADERS:Request-Range.
[4] Rule returned 0.
[4] (Rule: 920201) Executing operator "EndsWith" with param ".pdf" against REQUEST_BASENAME.
[4] Rule returned 0.

@csanders-git
Copy link
Contributor

As this appears in part to be a libmodsec problem @zimmerle would you like me to xpost this?

@theMiddleBlue
Copy link
Contributor Author

theMiddleBlue commented Jun 16, 2018

@csanders-git doing some tests, it seems that the latest working commit is owasp-modsecurity/ModSecurity@f928e44

@theMiddleBlue
Copy link
Contributor Author

Doing some tests, I've found the problem: v3/master ignore this variable when it's set using lowercase variable name...

Working configuration:

SecAction \
  "id:900000,\
   phase:1,\
   nolog,\
   pass,\
   t:none,\
   setvar:TX.PARANOIA_LEVEL=1"

Wrong configuration:

SecAction \
  "id:900000,\
   phase:1,\
   nolog,\
   pass,\
   t:none,\
   setvar:tx.paranoia_level=1"

Maybe this problem could affects any other SecRule on the CRS that set a lowercase variable name.

@lifeforms
Copy link
Contributor

lifeforms commented Jun 20, 2018

Hmmmm, that should not happen...

From the ModSecurity handbook:

screen shot 2018-06-20 at 12 47 47

@theMiddleBlue
Copy link
Contributor Author

Just tested more:

OK:

SecAction \
  "id:900000,\
   phase:1,\
   nolog,\
   pass,\
   t:none,\
   setvar:TX.PARANOIA_LEVEL=1"

KO:

SecAction \
  "id:900000,\
   phase:1,\
   nolog,\
   pass,\
   t:none,\
   setvar:tx.paranoia_level=1"

KO:

SecAction \
  "id:900000,\
   phase:1,\
   nolog,\
   pass,\
   t:none,\
   setvar:TX.paranoia_level=1"

KO:

SecAction \
  "id:900000,\
   phase:1,\
   nolog,\
   pass,\
   t:none,\
   setvar:TX.parANoia_LEvel=1"

@csanders-git
Copy link
Contributor

So if I had to guess that cfg for variable needs to be case insensitive @zimmerle

@lifeforms lifeforms added the ModSec Issue related to ModSecurity label Jun 20, 2018
@michaelgranzow-avi
Copy link

I think this bug has been introduced by owasp-modsecurity/ModSecurity@892beb5

Here, the lookup was changed from using std::multimap::equal_range with custom hash and equal functions (both case-insensitive) to std::string::compare, which is case-sensitive:

-        auto range = this->equal_range(var);
-        for (auto it = range.first; it != range.second; ++it) {
-            l->insert(l->begin(), new VariableValue(&m_name, &var, &it->second));
+        for (auto &a : *this) {
+            if (a.first.compare(0, var.size(), var) == 0) {
+                l->insert(l->begin(), new VariableValue(&m_name, &var, &a.second));
+            }

As a side-note, this change also seems to have turned a hash lookup into a linear search, so it may well have performance impact.

@victorhora
Copy link
Contributor

Thanks for the detailed reporting guys.

This is being handled at owasp-modsecurity/ModSecurity#1808. PR #1810 is up for evaluation to fix this issue. My tests went well and the buildbots are going fine for now. Would appreciate if you folks could test and report as well.

Thanks!

@victorhora
Copy link
Contributor

As pointed out by @zimmerle at owasp-modsecurity/ModSecurity#1808 (comment), this has been fixed by commit owasp-modsecurity/ModSecurity@d810de9. Thanks for your contribution @michaelgranzow-avi :)

I believe this one can be closed now.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
ModSec Issue related to ModSecurity
Projects
None yet
Development

No branches or pull requests

6 participants