BuildInstructions

How to build the Lounge

Updated Sep 4, 2009 by kevin.a....@gmail.com

Before you start...

At Meebo, we deploy the lounge via RPMS, so the installation process is heavily biased towards building an RPM and then installing that RPM (as opposed to doing a much saner configure/make/make install sort of process). On the upside, it makes this a bit easier to deploy to a production environment. The downside is that you have to screw around with RPMs to make sure all the files end up in the right places.

This is especially painful if you're on a debian-based system instead of a redhat-based one. (CentOS versus Ubuntu, for instance) In that case, alien works fine to convert/install these packages.

This also assumes you have an instance of CouchDB up and listening on port 5984. If you want to use a different port, that's fine, you'll just need to modify the shards.conf file (more on that later).

Build

check out a copy of the source, then build the various components:

couchdb

You need to apply a small patch to the couchdb sources. The patch enables design-only replication. The lounge uses design-only replication to synchronize your design documents and views across all shards in the cluster.

If you are on Debian or Ubuntu, try the patched .debs in the Downloads section. They were built against Ubuntu 9.04 (jaunty) on i386 and amd64, but they should work with Debian testing and maybe Ubuntu 8.10 as well. (Let us know if you have dependency trouble)

If you aren't, you'll have to build CouchDB. Grab sources from http://couchdb.apache.org/downloads.html and then:

tar zxvf apache-couchdb-0.9.0.tar.gz
cd apache-couchdb-0.9.0
patch -p1 < (PATH_TO_COUCHDB_LOUNGE)/couchdb/designonly_replication.patch
./configure && make
sudo make install

dumbproxy

dumbproxy is nginx with a custom proxying module. The proxying module depends on json-c, a library availabe at http://oss.metaparadigm.com/json-c. Additionally, I've built 64-bit rpm and deb packages for this and made them available in the downloads section of this project (http://couchdb-lounge.googlecode.com/files/json-c_0.7-7_amd64.deb, http://couchdb-lounge.googlecode.com/files/json-c-0.7-6.x86_64.rpm)

cd dumbproxy/trunk
./build_rpm.sh
cd rpms/<your_processor_type>
sudo <rpm -i | alien -di> lounge-dumbproxy-1.0.0*.rpm
sudo /etc/init.d/dumbproxy start

python-lounge

this is a collection of python modules used by smartproxy and the other python components

cd python-lounge
python setup.py install

smartproxy

this is a python/twisted daemon that handles sharding/partitioning of the views You'll need a couple of other python bits working for the daemon to work properly:

Twisted (v8.1+) (apt: python-twisted, yum: Twisted) -- twisted provides the asynchronous networking framework we use to handle requests without blocking.
python-cjson (v1.0.5+) (apt/yum: python-cjson) -- fast json parser

cd smartproxy/trunk
make
cd dist
sudo <rpm -i | alien -di> lounge-smartproxy-1.0-0.noarch.rpm
sudo /etc/init.d/smartproxy start

replicator

The Lounge replicator has two goals:

Keep design documents synchronized across all shards so you can run views over your whole cluster.
Replicate data from master shards to slave shards for redundancy.

See InternalReplication for more details. The replicator is easy to set up. Just copy replicator/replication_notifier.py somewhere, e.g.,

sudo install -m 755 replicator/replication_notifier.py /var/lounge/lib/

Then add this block to your CouchDB default.ini file:

[update_notification]
replicator = /usr/bin/python /var/lounge/lib/replication_notifier.py

Restart couch. You can use ps to confirm that replication_notifier.py is running.

Is it working?

So, assuming you have a couch instance running on this box (or, if you have a couple of other boxes lying around, running on those), we can now set up the sharding arrangement.

Read ShardsConfandYou for details on how to set up your shards.

By default, the cluster will have two shards on two nodes, both pointing at localhost:5984. This is good enough to get you rolling on one machine.

Testing with Futon

Head over to TestingOneNode

Comment by bondarenko.roman@gmail.com, Sep 3, 2009

Hi! I have a question about 'replicator' (replication_notifier.py). Suppose, I have 3 nodes (instances of CouchDB). Each node placed on separate host (node1 on host1, node2 on host2, node3 on host3). I think, I must place the 'replicator' to each host and configure each local.ini with:

[update_notification]
replicator = /usr/bin/python /var/lounge/lib/replication_notifier.py

Is it right? Or I should do it only for first node?

(sorry for my English :)

Comment by project member srlind...@gmail.com, Sep 3, 2009

@bondarenko.roman: You are correct, you'll want to do that for each node in the cluster.

Comment by bondarenko.roman@gmail.com, Sep 4, 2009

@srlindsay: Thanks!

And next 2 questions:

Can I use more than 1 dumbproxy inside my cluster? (e.g. 1 dumbproxy on each host; all instances use an equal config settings)? Or it can cause problems?
And same question for the smartproxy (1 smartproxy on each host and equal configs for all instances)?

Comment by project member srlind...@gmail.com, Sep 4, 2009

@bondarenko.roman: As long as the shards.conf is the same, you can have as many smart and dumb proxies as you want -- for high availability reasons, you'll definitely want more than one of each running.

Comment by bondarenko.roman@gmail.com, Sep 5, 2009

@srlindsay: Thanks for your answers.

Comment by jma...@translab.its.uci.edu, Sep 29, 2009

Interested in trying this out, but I'd like to use couchdb trunk or couchdb 0.10.

The patch failed when applied to the trunk. What are the plans, if any, for upgrading to a more recent version of couchdb?

Comment by project member kevin.a....@gmail.com, Sep 29, 2009

@jmarca: We're planning to try to add support for 0.10 as soon as it officially drops ... which should be any minute now.

Comment by jma...@translab.its.uci.edu, Oct 13, 2009

@kevin.a.ferguson, now that 0.10 has dropped, I've of course already moved on to using 0.11 git version because the view building is better, supposedly. But when you upgrade lounge for 0.10, I'm planning on setting up a test environment.

Comment by project member kevin.a....@gmail.com, Oct 15, 2009

@jmarca: It's a moving target :)

I hope to have Lounge for 0.10 ready this weekend-- we're pretty eager to upgrade our own cluster.

Comment by klimp...@gmail.com, Oct 15, 2009

Can you guys post the instructions to build .deb files?

Comment by project member kevin.a....@gmail.com, Oct 16, 2009

@klimpong -- as soon as we figure out how to :)

Seriously, it's clearly way too hard to install the lounge right now, which is preventing people from experimenting with it and giving feedback. So we should probably bump the priority of improving the packaging.

Comment by project member kevin.a....@gmail.com, Oct 17, 2009

Hey @jmarca, if you want to get the latest svn version it should work with CouchDB 0.10.0. It has not been as battle-tested as the 0.9.0 edition so please report any troubles.

Comment by bchesn...@gmail.com, Oct 20, 2009

that would be good to have a full requirement lists and maybe a full makefile. It's hard to install all the stuff pn ubuntu.

Comment by project member srlind...@gmail.com, Oct 20, 2009

bchesneau: yeah, agreed. I did a bunch of work this weekend, rearranging things to make it work more like a coherent project rather than a collection of subprojects. Assuming you have json-c installed, you should be able to build everything with the usual 'configure, make, make install' build process. Since that process now works, I can build a .deb package out of it. I still need to get all the default paths in the right places so that the .deb will work without tweaking, but it's really close to being a 'dpkg -i' and forget sort of install.

Comment by bchesn...@gmail.com, Oct 21, 2009

thanks , saw the deb on github let me know when you need test

Comment by timcoffm...@gmail.com, Nov 4, 2009

I need to echo the comments above re: the Lounge is too hard to install and the frustration factor is dissuading usage.

I have been trying for days to get it installed. Nearly ready to give up!

On platform el5/x86_64, I've moved to json-c 0.9 when Lounge wouldn't build with json-c 0.7. With json-c 0.9, the rpm has but, but it won't install.

Comment by timcoffm...@gmail.com, Nov 6, 2009

I have successfully built, installed, and am running nginx-lounge (i.e. dumbproxy). I accomplised this by skipping the RPM process entirely and using "sudo make install" instead.

I think python-lounge installed correctly. There's not really a verification step for that.

Now I am stuck on the Twisted component (i.e. smartproxy). It won't start, but I'm not sure why.

[tcoffman]$ sudo /etc/init.d/smartproxyd start
Starting smartproxy: /bin/bash: /usr/bin/twistd -r epoll -y /etc/lounge/smartproxy.tac --pidfile=/var/run/pid/smartproxyd.pid --logfile=/var/log/lounge/smartproxyd.log -u 0 -g 0: No such file or directory
                                                           [FAILED]
[tcoffman]$ sudo /usr/bin/twistd -r epoll -y /etc/lounge/smartproxy.tac --pidfile=/var/run/pid/smartproxyd.pid --logfile=/var/log/lounge/smartproxyd.log -u 0 -g 0

Failed to load application: No module named lounge.prefs
[tcoffman]$

► Sign in to add a comment