Something changed recently about default apache configuration (FollowSymLinks disabled?)

[stronk7]

Hi,

we have detected recently that, when using the latest/current builds from March 28th, some applications that previously were running and working ok have stopped working with apache logs plenty of errors:

[core:error] [pid 1116] [client 172.21.0.6:50746] AH00037: Symbolic link not allowed or link target not accessible...

Is this an intended change and now we have to proceed to enable FollowSymLinks or maybe some unexpected change that needs to be fixed?

We have detected it in a few images:

• 8.1-bullseye digest: f2309205de24
• 8.0-bullseye digest: 1af3328638f3

But surely all them (apache) are the same.

Using the 8.1.16 ones or older 8.1.17 ones (before the very last ones commented above) it seems that the sym links are working ok.

Thanks for the nice work and ciao :-)

[tianon]

The images you've linked are CLI images and don't include Apache2 at all. 😅

Do you happen to have some more specific examples so we can investigate?

We didn't change anything related to this recently (that I recall), so my hunch is that Debian probably had an update to their apache2 packages which we use and keep fairly stock configuration from, and if that's the case, it was probably security-related (because they'd be otherwise unlikely to update it in the stable release).

[stronk7]

Oh, sorry I linked them incorrectly. The ones we are using are the apache ones:

• 8.1-apache-bullseye
• 8.0-apache-bullseye

We just have some soft links called "behatrun1", "behatrun2", "behatrun3" (we use that for parallel behat execution using the same codebase) within /var/www/html and exactly pointing to that directory, created with this command:

docker exec -t -w /var/www/html $WEBSERVER_CONTAINER .... bash -c "ln -s . \"behatrun1\""

(aka, a simple cd /var/www/html && ln -s . behatrun1 within the container)

With previous image versions, when we invoke http://localhost/behatrun1, it works perfectly. With the latest versions from few days ago, it does not work and apache error logs the Symbolic link not allowed message in the description.

I still have not looked what has changed (only suspicious commit that I've seen overlooking is one changing all the default permissions, not sure if it's related or no). But I've compared here some executions and the only difference in the whole stack (images, composer, codebase...) is the php image used.

That leaded me to think it's caused by some recent change in the images (the apache that comes with them or something when creating the images). They were working ok until 2-3 days ago, just today when we pulled the latest we found the consistent error.

If I find something else, I'll share it here. Hope it helps.

Ciao :-)

[tianon]

Quickest way to verify whether it's #1383 is to add --user www-data to your docker exec

[stronk7]

Hi,

I can confirm that, when --user www-data is used, the FollowSymLinks is working ok. But when no user is specified, and the links are created as root, is when FollowSymLinks stops working.

Still, it's not clear for me if that's because of #1383 or because apache changed something in their perms checks / defaults now being stricter.

I'll try to see in recent releases / CVEs if there is any change. In any case, those changes need to have happened over the last few days, because the image we were using (working "ok") previously was created March 27th. And the one that started to fail was March 30th).

That's how we came to here, after checking that the only difference was that one:

< moodlehq/moodle-php-apache:8.1 created:2023-03-30T07:43:55.510309374Z sha256:d4d260d12059ed7a8568ac985ca00624ad9286c85c8bdeabe502ea5f2f11f863
> moodlehq/moodle-php-apache:8.1 created:2023-03-27T07:41:34.41587614Z sha256:aea84238d975356c368680b68c30ffedef9f9941366301943ce0011edad1ed03

In the good side, all we have to do is to add the --user www-data to our scripts, but I can imagine this impacting out there noticeably (or maybe not, heh) in a bunch of default cases.

Ciao :-)

[stronk7]

Uhm...

so, looking to apache2 debian changelog (I imagine that's a good source, amend me if not, please):

https://launchpad.net/debian/+source/apache2/+changelog

It seems that their latest update to 2.4.56-1 happened on March 8th. And no changes since then.

So, if I'm interpreting that correctly, the previous PHP images already were using that apache debian package. Hence, it seems that #1383 is the ultimate case for this change of (previous) behaviour to happen and apache not accepting root's soft links any more.

Ciao :-)

[stronk7]

And, sort of confirmed...

I've just connected to the container while it was still installing the testing environment, have executed chmod 777 /var/www/html, and then all the tests (that use those soft links) have passed without problem (like they used to be).

So the 777 => 1777 modification (#1383) seems to be the one causing the change of behaviour, indeed.

So, the question remains... should we adapt our scripts to use --user www-data (or to change the perms back to 777)... or is this something that, maybe, is going to be reverted or fixed in the images themselves?

Ciao :-)

[tianon]

We do not plan to revert #1383 because it is generally a more secure way to accomplish the original intent of our 777, so I would suggest adapting your scripts to use --user (which makes them more defensive too 👍).

Alternatively, you could run your container itself with --user www-data and that will make all the docker exec invocations imply that automatically and might be a smaller change for you.

[tianon]

(see also "Running as an arbitrary user" under https://hub.docker.com/_/php)

[stronk7]

Thanks @tianon ! All ok for us.