What: Distributed Memory Allocation Framework
Who: Alex Merritt, Jeff Young

For use with Jeff Young's Oncilla Project.

-- Prerequisites --

* gcc (obviously)
* openmpi, libopenmpi
* python, scons
* perl >=5.9.4 (for gccfilter)
* libregexp-common-perl (for gccfilter)
* libgetopt-argvfile-perl (for gccfilter)

-- Build --

The command is simply:

    $ scons [-Q] [filter=1] [dbg=1]

This will produce a binary 'oncillamem' to bin/.

'filter' argument - when enabled, will cause all gcc output to pass through a
colorizing filter for easier identification of errors.

'dbg' enables debug symbols to be built in for use with GDB

To clean:

    $ scons -c [-Q]

-- Execution of OncillaMem--

For an easy run (on octane*):

    bin/oncillamem bin/nodefile


bin/nodefile should contain the DNS of each node this code will execute on, one
per line. The first machine is where rank0 (the master) will be spawned.

Edit bin/nodefile to list the machines on which the daemons should spawn so that
bin/launch will spawn them appropriately.

To enable debug/verbose output, define the environment variable 'OCM_VERBOSE' to
be anything (the code just checks if it exists, not the value it is set to for
now).

    OCM_VERBOSE=1 ./oncillamem

-- Execution of test applications--

Applications must include oncillamem.h and need to
link against the Oncillmem library, libocm. Currently, you
will need to set your LD_LIBRARY_PATH to point to the 
library path in the Oncilla main directory, e.g., 
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ONCILLA_ROOT/lib

-- Using the API --

TODO

apps must link against liboncillamem & include oncillamem.h
maybe have these install to the system?

-- DEV NOTES --

if a process dies... the daemon perhaps should detect this and deallocate all
memory that was allocated to it

library must perform part of the allocation request, as malloc etc cannot be
used if created in another process, such as the daemon

the daemon may instruct the library to carry out certain things using the MQ
module. eg an allocation that happens locally, the daemon may inject messages
instructing the library to perform an allocation, return the pointer value to
the daemon (forwarding it to rank0) then injecting another message to release
the application