mod_security-2.1.2 and sub requests and segmentation faults

When I moved from 1.8.7 to 2.1.1 I ran into a weird issue where sub requests were not being processed and blocks were not being performed if you were using mod_limitipconn. You can read about it here. During those days, I was running into intermittent segmentation faults with one vhost. I discovered that if I commented out the ErrorDocument for that particular vhost, the segfaults went away. I didn’t give much more thought on it, but when we just rolled out to 2.1.2, we received more of these… enough for me to start digging.

What I discovered, if a location was using a proxy (squid / dansguardian, etc) and posted a form that was using the multipart/form-data enctype, without supplying a upload file, the segfaults would occur. I immediately contacted mod_sec mailing list (I’m tellin’ ya, they have the best mailing list and responses to bugs!!!). I started working with one of the developers where we were trying to understand the cause.

The first step was to get a core dump and see what was up, and where exactly in the stack it was bombing. The core dump yielded very little info because of no debug CFLAGS. I was requested to add some debug flags (-g -ggdb) so we could get more useful core dumps. I thought this would be simple, just add it to CFLAGS in make.conf, and recompile. Well, not so easy, while I was recompiling, I was not seeing my flags in the compile output. Turns out that Apache modules inherit certain CFLAGS and don’t really use the make.conf. You can inject some CFLAGS by editing the ebuild. This means, we need to do a portage overlay on this ebuild.

mkdir -p /usr/local/portage/www-apache/mod_security cd /usr/local/portage/www-apache/mod_security cp /usr/portage/www-apache/mod_security/mod_security-2.1.2.ebuild . cp /usr/portage/www-apache/mod_security/files . -R vi mod_security-2.1.2.ebuild

Now let’s add our CFLAGS.

APXS2_ARGS="-DWITH_LIBXML2 -I/usr/include/libxml2 -lxml2 -S LIBEXECDIR=${S} -Wc,-O2 -Wc,-ggdb -c -o ${PN}2.so ${S}/apache2/*.c"

The area to pay attention to is -Wc,-O2 -Wc,-ggdb. After having some more output, they discovered the problem was with sub requests. Apparently, when they moved from 1.x to 2.x they put additional sub request handling and for some reason messes me up. I’m not quite sure why this seems to happen to me… I’d assume that everyone would be experiencing these segfaults under these circumstances.

They were quick on releasing a patch for me, and I’ll provide it here in case others are looking for a solution. You can download hanji2.diff patch here.

To apply the patch copy the patch to your overlay.

cp /path/to/patch /usr/local/portage/www-apache/mod_security/files/. cd /usr/local/portage/www-apache/mod_security ebuild mod_security-2.1.2.ebuild digest emerge -pv mod_security

You should see an output similar to this:

[ebuild R ] www-apache/mod_security-2.1.2 USE="-doc" 643 kB [1]

Total: 1 package (1 reinstall), Size of downloads: 643 kB Portage tree and overlays: [0] /usr/portage [1] /usr/local/portage

This diff removes sub request handling. I’m not sure what the future will hold for sub requests and mod_security. I’m waiting for an official patch release, so I can submit it to bugs, but in the meantime, this will fix the segfaults.