Using Varnish and iptables_recent to fend off Slowloris attacks on CentOS

As some may know, an old attack has been wrapped in a perl script called “Slowloris.pl”. Any script kiddie with a Linux box and a couple of Perl modules installed will be able to take down most webservers. The script works by sending a lot of requests, that do not finish, so the webserver becomes unable to process any “real” requests. I sat down and wrote a guide to use Varnish and iptables to block any Slowloris attacks against an Apache webserver. I hope this will prove helpful for anyone wanting to secure their web servers against this attack.

— Jes Kasper Klittum

Getting the sources

You can get the sources from the Varnish SourceForge site.
Find the download link on this page:
http://www.varnish-cache.org/
As of 10th of November 2010 this is what I did:

wget http://www.varnish-software.com/sites/default/files/varnish-2.1.4.tar.gz

Compiling Varnish

Make sure that gcc, autoconf, automake and pcre-devel are installed. If not installed run:
yum install gcc autoconf automake pcre-devel

Untar the package and enter the directory:
tar fxzv varnish-2.1.4.tar.gz
cd varnish-2.1.4/

Now compile the sources:
./configure
make
make install
cp redhat/varnish.initrc /etc/init.d/varnish
sed -i -e 's:exec="/usr/sbin/varnishd":exec="/usr/local/sbin/varnishd":' /etc/init.d/varnish
cp redhat/varnish.sysconfig /etc/sysconfig/varnish
groupadd -r varnish
useradd -r -g varnish -d /var/lib/varnish -s /sbin/nologin \
-c "Varnish http accelerator user" varnish
chkconfig --add varnish
chkconfig --level 345 varnish on
mkdir -p /var/lib/varnish/

Make a symlink from /etc/varnish to /usr/local/etc/varnish

ln -s /usr/local/etc/varnish /etc/varnish

Edit /etc/sysconfig/varnish to match this:
DAEMON_OPTS="-a :80 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-u varnish -g varnish \
-s file,/var/lib/varnish/varnish_storage.bin,1G"

Configure Apache

You need to make apache listen to another port than 80. To do this, you must change the directives Listen, ServerName, NameVirtualHost and all “virtualhost *:80” enties in the apache configuration.

I did this:
root@www conf]# grep -A2 Varnish /etc/httpd/conf/httpd.conf
#Varnish change
#Listen 80
Listen 8080
--
#Varnish change
#ServerName 127.0.0.1:80
ServerName 127.0.0.1:8080
--
#Varnish change
#NameVirtualHost *:80
NameVirtualHost *:8080

#Varnish change
#

Configure Varnish

Now, apache is now listening to another port, and our firewall denies untrust access to this port – what we need to do now, is get Varnish to filter the packets comning in on port 80 and forward then to port 8080 on 127.0.0.1

cat /etc/varnish/default.vcl
#-e This is a basic VCL configuration file for varnish. See the vcl(7)
#man page for details on VCL syntax and semantics.
#
#Default backend definition. Set this to point to your content
#server.
#
backend armada {
.host = "127.0.0.1";
.port = "8080";
}

#
# Define what to do when receiving a request;
# - Send all requests to the backend "armada"
# - Remove X-Forwarded-For if it is already set for some reason
# - Add a new X-Forwarded-For header, set it to the clients IP (for logging on Apache)
#
sub vcl_recv {
set req.backend = armada;
remove req.http.X-Forwarded-For;
set req.http.X-Forwarded-For = client.ip;
}

#
# Set timeout for all objects to 0 seconds. This defeats the purpose of running a cache,
# but this is for the added security only.
# This was the easiest way for me to preserve logging of all requests.
#
sub vcl_fetch {
set obj.ttl = 0s;
}

Fixing Apache logging

All IPs will be logged as coming from 127.0.0.1 if you do not use the X-Forwarded-For for the Log IP. There are different approaches to this.
Best approach is to install mod_rpaf; reverse proxy add forward
This requires httpd-devel to be installed. This can be obtained through yum.

cd /root
wget http://stderr.net/apache/rpaf/download/mod_rpaf-0.6.tar.gz
tar fxzv mod_rpaf-0.6.tar.gz
cd mod_rpaf-0.6
apxs -i -c -n mod_rpaf-2.0.so mod_rpaf-2.0.c

Create a file /etc/httpd/conf.d/rpaf.conf
cat /etc/httpd/conf.d/rpaf.conf
LoadModule rpaf_module modules/mod_rpaf-2.0.so
RPAFenable On
RPAFproxy_ips 127.0.0.1

Limit number og connections from one IP

Using iptables recent, we can make sure that any IP will only be allowed to make, f.ex., 20 connections in 5 seconds – otherwise they get blocked.

iptables -N Slowloris
iptables -A INPUT -p tcp --dport 80 -m state --state NEW -j Slowloris
iptables -A Slowloris -m recent --set --name APACHE
iptables -A Slowloris -m recent --update --seconds 5 --hitcount 20 --name APACHE -j DROP
iptables-save > /etc/sysconfig/iptables
chkconfig iptables on

Starting it all up

/etc/init.d/httpd stop
/etc/init.d/varnish start
/etc/init.d/httpd start

Kudos

These guides were used to compose this walk-through:
http://wiki.tyk.nu/index.php/Using_Varnish_to_protect_Apache_against_slowloris#Configuring_Varnish
http://developer.mindtouch.com/User:PeteE/Varnish_Installation
http://varnish.projects.linpro.no/
http://maxgarrick.com/reverse-proxy-with-nginx
http://www.ducea.com/2006/06/28/using-iptables-to-block-brute-force-attacks/

2 thoughts on “Using Varnish and iptables_recent to fend off Slowloris attacks on CentOS

  1. André Cardoso

    Hi.
    I followed your guide, but when I attack the server with slowloris and try to access a page through the browser, I just get a 200 OK, with empty content — a blank page! So it seems that my varnish is not stopping slowloris!

    Can you give me some help?

  2. Administrator Forfatter

    Hi André,

    It appears to be a problem with Varnish, when it is recieving more than 4000 requests per second. This URL states so anyway:

    http://osdir.com/ml/web.varnish.misc/2007-10/msg00003.html

    4000+ requests/s is a very high number, and it is way beyond where Apache gives up, så my solution should still be valid in most cases. However, you may be able to tweak Varnish to handle this in some way – however, this must be up others to test. And please report back if successful.

Skriv et svar

Din e-mailadresse vil ikke blive publiceret. Krævede felter er markeret med *