DLucene: Full-text Search/Indexing for D

Summary

This is a port of the Apache Lucene full text search/indexing system to the D programming language, based initially on the existing Lucene.Net port. Through the use of D specific techniques along with tight integration to the underlying OS, it is hoped that excellent performance with a very small runtime footprint can be achieved.

Goals

In addition to a compact D library, a simple, reuseable daemon program is invisaged that will have no external dependencies, and can be controlled over a pipe or TCP using simplistic commands encapsulated in cPickle, serialize(), or HTTP.

• Load time, run time, memory usage and throughput to outperform Lucene.
• Binary compatibility with Lucene indices.
• Python and PHP wrapper classes, fast codecs for cPickle and serialize().

Blue sky:

• Implement Google-style boolean query parser.
• Switch to Tango or at least support it.
• Support GDC to allow easy Debian packaging.
• Support a ReSTful HTTP API ala. CouchDB.

License

Please note that the code does not conform to the Apache 2.0 license at present. This is my fault in two ways:

• I stripped the boilerplate from each source file, in contravention to the license.
• My modifications to the original Lucene/Lucene.Net code are not yet explicitly licensed.

To make that clearer: this software is not currently open source software, and potentially illegal to distribute. I will fix this situation when I have more time.

Status (2007-11-23)

• Development is taking place in a private Subversion repository that is mirrored here.
• An RSS feed of commits is available.
• A technical TODO list is available.
• None of this code has been tested thoroughly yet.

Beyond what is listed in the table below, the following tasks remain:

• Convert C#-style documentation, find a way to pull into DocBook.
• Create regression tests.