my experience with Modsec for Nginx

[quenenni]

Hello,

Since lots of years, we are using Modsecurity with Apache and are very happy with it (and thanks a lot for this).

Recently, we configured a new server with a Nginx in front of apache in order to use Anubis in the middle.

I decided to finally use the Nginx version of Modsecurity.

Everything was perfect, except one thing.

When reloading (not even restarting) Nginx, our server load went from 0.5 to 25 and freeze for 20 to 30 seconds (I was not even able to strace what happens at that time because of the freeze).

The problem seems to come from the fact I have "modsecurity on;" in one generic vhost and on every vhost for a domain.
At the time of my tests, we had like +- 40 vhosts in Nginx (not that much as a full server will have several thousands vhosts).

When deactivating all the Modsecurity in our vhosts, the reload of Nginx came back to normal.

I noticed when relaoding Nginx that Nginx create new processes and then stop old processes.
And it's in the phase to stop old processes that they all go to 100% for 20-30 sec before closing.

My impression is that Modsec do things that deny Nginx to stop the processes.

Unfortunately, I can't give you more details as I quickly switched back to the Apache version of Modsec, but I wanted to let you know my experience.

Our setup:
Debian Bookworm 12.13
libmodsecurity3 3.0.9-1+deb12u1
Nginx 1.22.1-9+deb12u4

Here are exemples of vhosts (quite basic ones):

• generic vhost that Anubis contacts:

server {
    listen unix:/run/nginx.sock;

    server_name _;

    # Get the visiting IP from the TLS termination server
    set_real_ip_from unix:;
    real_ip_header X-Real-IP;

    modsecurity on;
    location / {
        proxy_pass http://127.0.0.1:80;
        modsecurity_rules_file /etc/nginx/modsecurity_includes.conf;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Http-Version $server_protocol;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-Port 443;
    }
    access_log /var/log/nginx/anubis_return_access.log withhost;
    error_log /var/log/nginx/anubis_return_error.log;
}
~ 

• vhost for every domains:

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    ssl_certificate /etc/letsencrypt/live/dev.XXX.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/dev.XXX.org/privkey.pem;

    location /.well-known/acme-challenge {
             root /var/www/letsencrypt;
    }

    server_name dev.XXX.org;

    modsecurity on;
    location / {
          proxy_pass http://127.0.0.1:80;
         modsecurity_rules_file /etc/nginx/modsecurity_includes.conf;
         proxy_set_header Host $host;
         proxy_set_header X-Forwarded-For  $remote_addr;
         proxy_set_header X-Forwarded-Proto https;
         proxy_set_header X-Forwarded-Port 443;
    }
}

Hope it will help you in some way.

Best

[airween]

Hi @quenenni,

The problem seems to come from the fact I have "modsecurity on;" in one generic vhost and on every vhost for a domain.

I'm really sorry but I don't see the difference: you have "modsecurity on;" in one generic vhost and on every vhost for a domain.

So you have many vhosts, and each vhosts have "modsecurity on;" directive, right?

When deactivating all the Modsecurity in our vhosts, the reload of Nginx came back to normal.

It's weird. I use many servers like this and never met with this problem...

Note, that I've never used Nginx (and Apache too) trough Unix socket... I'm afraid this could be the problem.

Our setup: Debian Bookworm 12.13 libmodsecurity3 3.0.9-1+deb12u1 Nginx 1.22.1-9+deb12u4

3.0.9 is a bit old, but I don't think the root cause is this.

Anyway, if you want to get a newer version, you might try our repository:

https://modsecurity.digitalwave.hu

Here are exemples of vhosts (quite basic ones):

You could try to put modsecurity on; into your http block:

...
http {
    ...
    ...
    modsecurity on;
    modsecurity_rules_file /etc/nginx/modsecurity_includes.conf;
    ...
}

and into that includes file you can include modsecurity.conf and the rules. And you can remove modsecurity on; from all of your vhosts.