To INSTALL or Use MemX now, go to Usage

The MemX system, developed in the Operating Systems and Networks (OSNET) Lab at Binghamton University, provides a mechanism to virtualize the collective memory resources of machines in a cluster with zero modifications to your application or operating system.

Features:

1. Auto-discovery and pooling of unused memory of nodes across physical or virtual machine networks.
2. Fully transparent, linux kernel-space implementation (2.6.XX).
3. No centralized servers. No userland components. (aside from the application).
4. Completely transparent access to memory clients (no application or OS modifications).
5. Live client state transfers (allows you to shutdown a client and reattach a block device on a new host or virtual machine). Sort of like a self-migrating block device without VMs
6. Live server shutdowns. Allows you to migrate server memory of individual hosts without disconnecting clients. (Self-dispersing servers).

MemX uses about 6000 lines of kernel code for all of the above features.

Coming features:

1. Page-sharing
2. RAID-like reliability
3. Multiple-client read support (Please note that MemX is not a DSM system requiring message passing and/or coherency/consistency protocols.)

How it Works:

In MemX, a linux module is inserted into the vanilla kernel of every participating machine in the cluster, turning each machine into either a client or a server.

The client module implements an optional block device. (The client can also be accessed from within the kernel). This device can be used either for mmap(), for swapping, for filesystems. It's your choice.

Node Discovery:

1. Servers announce (broadcast) themselves and their load statistics to every machine in the cluster at regular intervals. These announcements allow clients to make decisions for page allocation and retrieval across all available servers.
2. Clients accordingly accept block I/O requests, either from the local linux swap-daemon or the file system layer, and service the I/O requests from cluster-wide memory.

Advantages of our approach:

1. We get the benefits of a kernel-space implementation, without changing the virtual memory system.
2. The network protocol stack is bypassed because we don't need routing, fragmentation, nor a transport layer.
3. Cluster-wide memory is managed and virtualized across a low-latency, high-bandwidth interconnect.
4. Depending on the state of the cluster, any client or server can be made to join or leave the system at will.
5. Any workstation with a vanilla linux kernel can be used to run any legacy memory-intensive application.

Previous Work:

The following is a list of other remote memory systems that are similar in spirit but do not satisfy the goals that we do:

(Refer to "How it Works" for explanation of terms used)

Classification

Publications

MemX: Supporting Large Memory Applications in Xen Virtual Machines, SlidesIn Proc. of November, 2007 Second International Workshop on Virtualization Technology in Distributed Computing (VTDC07). A workshop in cunjunction with Super Computing 2007, Reno, Nevada. M. Hines and K. Gopalan