Sievelog is a message dispatcher inspired by swatch but more focused on the following problems:

• Having a clean syntax
• Directing a raw syslog stream to different files based on content
• Mailing out alerts based on content
• Being fast

Sievelog's syntax is as simple as <regex> -> /some/file.

Here's an example that would drop all smbd messages from the server named monkey and drop all nmbd messages into their own file:

"monkey smbd" -> /dev/null
"monkey nmbd" -> /sievelog/netbios

Sievelog is meant to be executed through a pipe by a syslog server. In classic BSD syslog, you'd say something like:

*.debug  | /path/to/sievelog /path/to/sieve.rules

and now sievelog would get all messages and filter them according to the rules in /path/to/sieve.rules.

An example of a functioning syslog-ng config (running in production) is:

source src {
        tcp(port(1514));
};

destination sievelog {
        program("/usr/local/bin/python -O /root/sievelog/sievelog -P /var/run/sievelog-stats -o /root/swatch/swatch.sv"
        template(forwarded));
};

log {
        source(src);
        destination(sievelog);
};

The -o option tells sievelog to activate the adaptive ruleset optimizer (ARO).

Sievelog has a ruleset optimizer built into it that can drop CPU usage significantly at the cost of breaking ruleset order. The ARO bubble-sorts the rules based on how frequently they're triggered so that the most commonly triggered rules get tried first. The effects are profound. During development we fed a ruleset (2000+ entries) and message stream (1500+ msg/sec) to sievelog that pegged a modern processor and caused lots of messages to be dropped. After we wrote and activated the ARO, CPU usage dropped to 1-3%. When sievelog's restarted in our environment, the ARO takes about 15 - 45 seconds to converge to an optimal ordering.

Ok, that's great, but doesn't this mean that we can't rely on rules being evaluated in the order they're written anymore? Yes. To cope with that, there's a feature of the language that lets you specify compound rules, whose actions are a list of guard/action pairs. The contents of a compound rule won't be touched by the ARO, even though the compound rule itself will be moved around.

The -P option tells sievelog that it should dump the ARO counter table into the named file when you send it a SIGINFO. This makes it very easy to see if there are legacy sievelog rules which aren't matching anything anymore.

Q: Ok, so what platforms does sievelog support?

A: Any Unix-like system running Python 2.5 or higher.

Q: ...and what syslog servers?

A: Any syslog server that supports pipes as an output method

Q: How do I get started?

A: Clone the sievelog mercurial repo, take a look at the test.sv file, bust up your own rules, check it with sievelog -t, and wire it up to a syslog server.