Skip to content

V3/collection re fix #2107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed

V3/collection re fix #2107

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3334d2a
Regex search in collection data case insensitive fix
airween Feb 5, 2019
4e1b99a
Fixed regex case insensitive search bug in LMDB
airween Feb 5, 2019
640daa4
Added test cases
airween Feb 5, 2019
ea937cf
Increment OVECCOUNT value for bigger regex's
airween Feb 7, 2019
8040904
Removed pcre dependency from outside of regex class
airween Feb 19, 2019
886fe07
Merge remote-tracking branch 'upstream/v3/master' into v3/collection_…
airween Jun 1, 2019
d9c0f73
Added regression test to Makefile.am
airween Jun 1, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Makefile.am
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,7 @@ TESTS+=test/test-cases/regression/request-body-parser-xml-validade-dtd.json
TESTS+=test/test-cases/regression/rule-920120.json
TESTS+=test/test-cases/regression/rule-920200.json
TESTS+=test/test-cases/regression/rule-920274.json
TESTS+=test/test-cases/regression/rule-920450.json
TESTS+=test/test-cases/regression/secaction.json
TESTS+=test/test-cases/regression/sec_component_signature.json
TESTS+=test/test-cases/regression/secmarker.json
Expand Down
2 changes: 1 addition & 1 deletion src/collection/backend/in_memory-per_process.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ void InMemoryPerProcess::resolveRegularExpression(const std::string& var,
//std::string name = std::string(var, var.find(":") + 2,
// var.size() - var.find(":") - 3);
//size_t keySize = col.size();
Utils::Regex r(var);
Utils::Regex r(var, true);

for (const auto& x : *this) {
//if (x.first.size() <= keySize + 1) {
Expand Down
2 changes: 1 addition & 1 deletion src/collection/backend/lmdb.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -538,7 +538,7 @@ void LMDB::resolveRegularExpression(const std::string& var,
MDB_cursor *cursor;
size_t pos;

Utils::Regex r(var);
Utils::Regex r(var, true);

rc = mdb_txn_begin(m_env, NULL, 0, &txn);
lmdb_debug(rc, "txn", "resolveRegularExpression");
Expand Down
9 changes: 6 additions & 3 deletions src/utils/regex.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,18 +38,21 @@ namespace modsecurity {
namespace Utils {


Regex::Regex(const std::string& pattern_)
Regex::Regex(const std::string& pattern_, bool caseSensitive)
: pattern(pattern_.empty() ? ".*" : pattern_) {
const char *errptr = NULL;
int erroffset;
int flags = (PCRE_DOTALL|PCRE_MULTILINE);

m_pc = pcre_compile(pattern.c_str(), PCRE_DOTALL|PCRE_MULTILINE,
if (caseSensitive == true) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name of the variable is wrong: we add the CASELESS flag if caseSensitive is true. Either keep the name, pass false in the invocations of the constructor changed above and test for !caseSensitive here; or change the name to ignoreCase.

As written it will confuse countless readers :(

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank for your remark - you're right.

Because this PR contains so many unnecessary commit, and this name convention is really illogical, I created a new one - see #2297.

Now I close this PR. Thanks for all helps to everybody.

flags |= PCRE_CASELESS;
}
m_pc = pcre_compile(pattern.c_str(), flags,
&errptr, &erroffset, NULL);

m_pce = pcre_study(m_pc, pcre_study_opt, &errptr);
}


Regex::~Regex() {
if (m_pc != NULL) {
pcre_free(m_pc);
Expand Down
2 changes: 1 addition & 1 deletion src/utils/regex.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ class SMatch {

class Regex {
public:
explicit Regex(const std::string& pattern_);
explicit Regex(const std::string& pattern_, bool caseSensitive = false);
~Regex();

// m_pc and m_pce can't be easily copied
Expand Down
90 changes: 90 additions & 0 deletions test/test-cases/regression/rule-920450.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
[
{
"enabled":1,
"version_min":300000,
"title":"Testing setvar :: OWASP CRS id:920450 ('Translate' match)",
"client":{
"ip":"200.249.12.31",
"port":123
},
"server":{
"ip":"200.249.12.31",
"port":80
},
"request":{
"headers":{
"Translate": "test"
},
"uri":"/",
"method":"GET"
},
"response":{
"headers":{
"Date":"Mon, 13 Jul 2015 20:02:41 GMT",
"Last-Modified":"Sun, 26 Oct 2014 22:33:37 GMT",
"Content-Type":"text/html"
},
"body":[
"no need."
]
},
"expected":{
"http_code":400
},
"rules":[
"SecRuleEngine On",
"SecDefaultAction \"phase:2,deny,block,status:400,log\"",
"SecRule &TX:restricted_headers \"@eq 0\" \"id:901165,phase:1,pass,nolog,setvar:'tx.restricted_headers=/proxy/ /lock-token/ /content-range/ /translate/ /if/'\"",
"SecRule REQUEST_HEADERS_NAMES \"@rx ^.*$\" \"id:920450,phase:2,block,capture,t:none,t:lowercase,msg:'HTTP header is restricted by policy (%{MATCHED_VAR})',logdata:' Restricted header detected %{MATCHED_VAR}',ver:'OWASP_CRS/3.1.0',severity:'CRITICAL',setvar:'tx.header_name_%{tx.0}=/%{tx.0}/',chain",
"SecRule TX:/^HEADER_NAME_/ \"@within %{tx.restricted_headers}\" \"setvar:'tx.msg=%{rule.msg}',setvar:'tx.anomaly_score_pl1=+%{tx.critical_anomaly_score}',setvar:'tx.%{rule.id}-OWASP_CRS/POLICY/HEADERS_RESTRICTED-%{MATCHED_VAR_NAME}=%{MATCHED_VAR}'"
]
},
{
"enabled":1,
"version_min":300000,
"title":"Testing setvar :: OWASP CRS id:920450 ('Proxy-Connection' not match)",
"client":{
"ip":"200.249.12.31",
"port":123
},
"server":{
"ip":"200.249.12.31",
"port":80
},
"request":{
"headers":{
"Accept": "text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5",
"Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.7",
"Accept-Encoding": "gzip,deflate",
"Accept-Language": "en-us,en;q=0.5",
"Host": "localhost",
"Keep-Alive": "300",
"Proxy-Connection": "keep-alive",
"User-Agent": "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv"
},
"uri":"/",
"method":"GET"
},
"response":{
"headers":{
"Date":"Mon, 13 Jul 2015 20:02:41 GMT",
"Last-Modified":"Sun, 26 Oct 2014 22:33:37 GMT",
"Content-Type":"text/html"
},
"body":[
"no need."
]
},
"expected":{
"http_code":200
},
"rules":[
"SecRuleEngine On",
"SecDefaultAction \"phase:2,deny,block,status:400,log\"",
"SecRule &TX:restricted_headers \"@eq 0\" \"id:901165,phase:1,pass,nolog,setvar:'tx.restricted_headers=/proxy/ /lock-token/ /content-range/ /translate/ /if/'\"",
"SecRule REQUEST_HEADERS_NAMES \"@rx ^.*$\" \"id:920450,phase:2,block,capture,t:none,t:lowercase,msg:'HTTP header is restricted by policy (%{MATCHED_VAR})',logdata:' Restricted header detected %{MATCHED_VAR}',ver:'OWASP_CRS/3.1.0',severity:'CRITICAL',setvar:'tx.header_name_%{tx.0}=/%{tx.0}/',chain",
"SecRule TX:/^HEADER_NAME_/ \"@within %{tx.restricted_headers}\" \"setvar:'tx.msg=%{rule.msg}',setvar:'tx.anomaly_score_pl1=+%{tx.critical_anomaly_score}',setvar:'tx.%{rule.id}-OWASP_CRS/POLICY/HEADERS_RESTRICTED-%{MATCHED_VAR_NAME}=%{MATCHED_VAR}'"
]
}
]