Repel http :BL

Repel http:BL is used with the Apache webserver or as a batch filter to identify friendly search engines and detect malicious web bots such as email address harvesters and comment spammers. The rewritemap filter accesses the DNS blacklist registry compiled by Project Honeypot to reliably determine the type and threat level of robots visiting your server.

Uses

Repel can for example be used to:

• Prevent malicious bots from accessing the websites.
• Redirect harvester and comment spammers to a honey pot.
• Use strict verification like CAPTCHA only on suspected harvesters and comment spammers rather than all visitors.
• Tailor content to search engines to facilitate proper indexing.

Repel can also be used batch from a terminal/shell to build blacklists from suspected IP addresses. The output is in Apache rewritemap format with the original address first on the line.

Requirements

Repel requires Python (compatible with version 3.x and 2.x back to 2.3.5). The distribution includes instructions about how to obtain a free key to use the Project Honeypot blacklist.

Open Source Libraries

The Repel repository includes open source Python libraries that might be useful for other projects:

• async.py - a library for asynchronous/concurrent execution, mapping and iteration. Repel uses it to perform multiple DNS queries in parallel and process the results in the order of the requests.
• dnsbl.py - retrieve DNSBL info for IP addresses.