Sparrow is a really fast lightweight queue written in Ruby that speaks memcache. That means you can use Sparrow with any memcached client library (Ruby or otherwise).

Sparrow keeps messages in memory, but persists them to disk, using Sqlite, when the queue is shutdown.

Basic tests shows that Sparrow processes messages at a rate of around 1660 per second. The load Sparrow can cope with increases exponentially as you add to the cluster. Sparrow also takes advantage of eventmachine, which uses a non-blocking io, offering great performance.

Sparrow's speed greatly increases when using the included client.

Sparrow comes with built in support for daemonization and clustering. Also included are example libraries and clients. For example:

require 'memcache' m = MemCache.new('127.0.0.1:11212') m['queue_name'] = '1' # Publish to queue m['queue_name'] #=> 1 Pull next msg from queue m['queue_name'] #=> nil m.delete('queue_name) # Delete queue

Messages are deleted as soon as they're read and the order you add messages to the queue probably won't be the same order when they're removed.

Additional memcached commands that are supported are: flush_all # Deletes all queues version quit The memcached commands 'add', and 'replace' just call 'set'.

Command line options are: ```

Usage: sparrow [-b path] [-h host] [-p port] [-P file] [-d] [-k port] [-l file] [-e] sparrow --help sparrow --version

Configuration: -b, --base PATH Path to queue data store. (default: /var/spool/sparrow)

Network: -h, --host HOST Specify host (default: 0.0.0.0) -p, --port PORT Specify port (default: 11212)

Daemonization: -P, --pid FILE save PID in FILE when using -d option. (default: /var/run/sparrow.log) -d, --daemon Daemonize mode -k, --kill PORT Kill specified running daemons - leave blank to kill all.

Logging: -l, --log [FILE] Path to print debugging information. -e, --debug Run in debug mode (default: false)

Miscellaneous: -?, --help Display this usage information. -v, --version Display version ```

Prerequisites are eventmachine and the sqlite3-ruby gem.

The daemonization won't work on Windows.

Check out the code: svn checkout http://sparrow.googlecode.com/svn/trunk/ sparrow

Install the gem: sudo gem install sparrow

Client

Also included is a Sparrow client with advanced clustering capabilities and fallbacks. If the Sparrow servers aren't available, queuing can fallback to Amazon SQS (you'll need the SQS gem if you're using this).

```

class MyQueue < MQueue::Queue def on_message(args) puts "Received msg with args: #{args.inspect}" end end

SQS.access_key_id = 'YOURACCESSKEYID' SQS.secret_access_key = 'YOURSECRETACCESSKEY'

servers = [ MQueue::Protocols::Sparrow.new({:host => 'localhost', :port => 11212, :weight => 1}), MQueue::Protocols::SQS.new({:weight => 0}) ]

MyQueue.servers = servers MyQueue.publish 'test' MyQueue.run

MyQueue.publish 'test'

MyQueue.run ```

Sparrow was inspired by Twitter's Starling