Prefiltering log events with rsyslog to greatly reduce fail2ban CPU load

Assume you have a service (web, mail, lb etc.) and you want to watch the logs of this service with fail2ban and take actions if specific log patterns occur. If your service is busy and have a high log volume/high log traffic, fail2ban can quickly eats up much cpu while trying to match regular expressions against the loglines.

Before you trying to tune your fail2ban failregex in every detail or wasting some time trying other methods to lower the CPU usage of fail2ban, take a look at rsyslog. With rsyslog, you can prefilter your logs for fail2ban and write the filtered events to a separate logfile. Then you only watch this logfile (which should be much smaller and has lower activity) with fail2ban. This should significantly reduce the CPU usage of fail2ban.

Here i use HAProxy (which is logging to syslog) as example service. Your service should be able to log to syslog (if not, look at the rsyslog “imfile” feature).

HAProxy config:

global
    log 127.0.0.1   local0
    log 127.0.0.1   local1 notice

 
/etc/rsyslog.d/49-haproxy.conf:

if $syslogfacility-text == 'local0' and $msg contains ' 418 ' then  -/var/log/haproxy/haproxy-filtered.log
local0.*                        -/var/log/haproxy/haproxy0a.log
& ~
local1.*                        -/var/log/haproxy/haproxy1a.log
& ~

Here I prefilter a HTTP status code (418) from the HAProxy access log and write the matched lines to a separate logfile. I use “contains” (no complex regex matching, just simple string matching) here for best efficiency. It doesn’t matter if some log lines are catched unintentionally because of the simple string matching. Fail2ban inspects the filtered logfile anyway with a specific and detailed regular expression.

rsyslog is a high performance logging software and can manage high log traffic with minimal cpu impact. With the help of rsyslog, you can use fail2ban even on services with a high log volume and/or a high log activity.

Leave a Reply