Linux: Anonymize IP logs with syslog-ng

By using a replacement for the default system logger, syslog-ng, you can selectively stop IPs from being written to log files, for example on a mail server.


If you’re on a Debian based system, you can simply apt-get the package:

apt-get install syslog-ng

Syslog-NG allows you to rewrite content using regular expressions. Add the following two rewrite rules to /etc/syslog-ng/syslog-ng.conf to replace IPv4 and IPv6 addresses by [REDACTED] and [REDACTED6]. (the regexp for IPv6 hasn’t been tested extensively yet).

rewrite r_ip {
subst('\b(1?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\.(1?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\.(1?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\.(1?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\b',
"\[REDACTED\]", value("MESSAGE"), type("pcre"), flags("global"));
subst('\b((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25
[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?',
"\[REDACTED6\]", value("MESSAGE"), type("pcre"), flags("global"));
};

You can now use this rule within syslog-ng.conf to selectively anonymize logs, for example:
log { source(s_src); filter(f_mail); rewrite(r_ip); destination(d_mail); };

Debian Squeeze

Debian Squeeze comes with syslog-ng 3.1, which does not support regexps longer than 1024. You can extend the limit and build your own new .deb:

apt-get install build-essential fakeroot devscripts
apt-get build-dep syslog-ng
apt-get source syslog-ng
rm syslog-ng_*.deb
cd syslog-ng-*
sed -i -e "s/#define MAX_REGEXP_LEN 1024/#define MAX_REGEXP_LEN 2048/"
src/cfg-lex.c
debuild -rfakeroot -uc -us
cd ..
dpkg -i syslog-ng_*.deb

Later versions do not have this problem.

General syslog advice

Think about using a tmpfs for /var/log. Edit the logrotate config to shred old logs (just add the “shred” keyword to it), and lower the default 4 week retention.

Leave a Reply

Your email address will not be published. Required fields are marked *