Compiling and using ramzswap

I'm trying on an CentOS 4 box (2.6.9) and found that it actually needs Linux kernel 2.6.17+ for the mutex API used.

Please add comments regarding compilation issues only. For other things, please use mailing list and issue tracker.

Hi, is there a way how to find out the efficiency? like compress ratio of data stored in that virtual swap disc? Or do you have any statistic on usual efficiency?

tiborb95: /proc/compcache gives idea on compression efficiency and /proc/tlsfinfo for storage stats. For other data, just explore this site.

Hello,

compilation on 2.6.20.3-ubuntu1 (on powerpc g4) succeeds with the following warnings;

CC M? /usr/src/compcache-0.4/sub-projects/allocators/tlsf-kmod/tlsf.o

/usr/src/compcache-0.4/sub-projects/allocators/tlsf-kmod/tlsf.c: In function `tlsf_destroy_memory_pool': /usr/src/compcache-0.4/sub-projects/allocators/tlsf-kmod/tlsf.c:448: warning: implicit declaration of function `pr_warning'

But using compcache fails with the following messages in dmesg;

3360.755026 tlsf: Unknown symbol pr_warning 3360.877490 compcache: Unknown symbol tlsf_free 3360.877732 compcache: Unknown symbol tlsf_malloc 3360.877891 compcache: Unknown symbol tlsf_destroy_memory_pool 3360.878040 compcache: Unknown symbol tlsf_create_memory_pool

Ok to fix the tlsf "unknown symbol" problem,

just add the following line to sub-projects/allocators/tlsf-kmod/tlsf.c

#include "compat.h"

Fixed in SVN rev #170. Thanks for pointing this out.

i compiled it on 2.6.9 but there is no mutex.h in it ...i found its there in 2.6.24 but not in 2.6.9 but you wrote it will compile well on all 2.6.

I tried patching 2.6.27.8 with 2.6.26.patch and I get this: drivers/block/compcache.c: In function ‘compcache_size_setup’: drivers/block/compcache.c:465: error: implicit declaration of function ‘strtoul’

Paczesiowa: see http://code.google.com/p/compcache/issues/detail?id=16 for the fix. I will update the patch soon.

this debian package provides an initscript to start compcache at boot time: http://groups.google.com/group/kurumix/web/compcache-initscript_0.1-1_i386.deb everybody is invited to test, is GPL.

Fabulous tool! I have prepared an install target for the Makefile and a init-script to have it fully installed in 5 minutes on my Ubuntu. http://gatopeichs.pbwiki.com/

If i compile kernel with JFFS2 filesystem support, then you modules warn about dublicate symbols lzo

Your comments?

> If i compile kernel with JFFS2 filesystem support, then you modules warn about dublicate symbols lzo

LZO modules are also distributed with compcache. So, if you try to reload these, you will get this warning. With 'use_compcache.sh' script you won't get these warning as it will silently skip reloading these modules.

> LZO modules are also distributed with compcache.

LZO algorithm already exist in 2.6.28+

dixlor: LZO is distributed to support old distro versions that still do not ship these (can't think of any names right now but still ...). Thats why use_compcache.sh script is there which will ignore these shipped modules in case they are already there.

Was able to compile on gOS based off of Ubuntu Hardy with 2.6.24. No runs, no drips, no errors. One "gotcha" seems to be that you have to compile as root. Sudo Make doesn't work right but make as root does.

linuxrebel: you need not compile as root - in fact thats a bad thing to do in general. Looks like you extracted files as root - do it all as normal user. You need to be root only to run (un)'use_compcache.sh' script.

Hi Nitin! When will you try to commit it to mainline?

marcodiegomesquita: Not sure... Pushing to mainline looks too time consuming!

Don't forget that you can always do the following:

make -C /lib/modules/2.6.26-1-openvz-686/build SUBDIRS=/usr/src/compcache-0.5.2 modules modules_install

If you do that however, you need to make sure you change a few INSMOD to $MODPROBE_BIN in the use_compcache.sh for xvmalloc and compcache modules. After using depmod -a however.

rcj4747: effort to push it to mainline has already been started: http://lkml.org/lkml/2009/3/20/196

I'm glad you found it useful. Thanks.

why there is no kernel patch equivalent to 0.5.3 ?

Paczesiowa: why waste time maintaining equivalent kernel patch when its available as module.

it was supposed to be faster when compiled in kernel. and it is the only thing that needs module support on my laptop now.

Paczesiowa: its really a big overhead for me to maintain equivalent kernel patch for every release. Also, there should not be any speed advantages when its compiled in kernel. Hopefully someday it will be merged with mainline Linux kernel.

Is there a plan to support swap devices that are file based e.g. /opt/file.swp as backing store?

> Is there a plan to support swap devices that are file based e.g. /opt/file.swp as backing store?

Its planned but low priority.

Please add to this documentation whether the backing swap device should be mounted as swap (swapon) by the kernel, or not. Whether it will be completely managed by compcache module. Thanks.

I have posted two debian binary packages. They are intended to make life easier (I hope :-) for compcache users on Debian boxes.

The first one is called compcache-source:

http://groups.google.com/group/kurumix/web/compcache-source_0.5.2-1_all.deb

and is a debianization of the compcache 0.5.2 source code. You can install the package and compilate the modules as follows:

# dpkg -i compcache-source_0.5.2-1_all.deb # tar -C /usr/src -xjvf compcache.tar.bz2 # cd /usr/src/linux # make-kpkg favourite options? modules

The second one is called compcache-utils and includes utility tools to manage compcache: the initscript /etc/init.d/compcache with its default file /etc/defaults/compcache which allows to start compcache at boot-time.

It includes also the script /usr/bin/compcache to enable/disable the compressed swap device. The script accepts the same sintaxis as use_compcache.sh, and has many command-line options. Probably has many bugs, also :-)

The package compcache-utils depends on this package:

http://groups.google.com/group/kurumix/web/shsh_0.1-1_i386.deb

Therefore, install this one first.

I have nothing to do with the Debian foundation. However, the binary packages are Debian-policy compliant, no any policy violation detected by Lintian, neither warnings messages.

For bugs, or feedback, please e-mail me.

I'm sorry, I forget this link in my previous post:

http://groups.google.com/group/kurumix/web/compcache-tools_0.5.2-1_i386.deb

troesmix: Thanks for this effort!

Does it support multiple swap partitions as backing_swap, by the way? Ie., I have /dev/sba3 and /dev/sdb3. Or would you need to keep the 2nd swap at a lower priority?

> Does it support multiple swap partitions as backing_swap, by the way? Ie., I have /dev/sba3 and /dev/sdb3. Or would you need to keep the 2nd swap at a lower priority?

No, it does not support multiple backing_swap partitions.

However, the support for creating multiple ramzswap devices is currently under development. When this is done, you will be able to use individual physical swap partitions/files as backing swap for different ramzswap devices.

Hi, Is it possible to use this feature without enabling swaping to any hard-disk? You see, on most embedded devices, memory swap feature is turned off, there is no flash partion used as swap disk. Thanks!

gzwu2000: Yes, its possible to use ramzswap without any physical swap partition.

Hi, great concept, which genuinely seems to extend the point at which my machine starts thrashing.

Is there any benefit to having ramzswap manage a backing device at the moment? Itś not clear in the documentation.

I would expect for this to be useful, ramzswap would kick least-recently-used compressed pages out to the backing device, to make room for new pages. Looking at the code, it seems to never kick any compressed pages out to the backing device, and if ramzswap becomes full, it simply forwards further requests on to the backing swap device. I think you get exactly the same result if you just swapon your disk swap as a seperate lower-priority swap device?

I guess this requires a compressed swap filesystem of some kind - I believe there is such a project?

Do you plan to implement this?

> Is there any benefit to having ramzswap manage a backing device at the moment? Itś not clear in the documentation.

Yes, there is. As you mentioned yourself, any uncompressible pages are sent to backing swap. When physical swap and ramzswap are used separately, then uncompressible pages will have to be stored in memory -- we will not be able to forward these two any lower priority swap. Currently, this is the only benefit of having physical disk as backing swap. LRU kind of eviction is planned feature.

> I guess this requires a compressed swap filesystem of some kind - I believe there is such a project? Do you plan to implement this?

No compressed FS is required - raw management of disk sectors is much faster as it allows bypassing VFS layer and I/O scheduler.

BTW, better place to discuss these is the project mailing list.

Just noting something from the multiple_rzs branch

the use_ramzswap.sh script does not work anymore. Basically you can go through the manual instructions and pass the number of devices to ramzswap.ko as follows:

sudo insmod ramzswap.ko NUM_DEVICES=1

Then you can use the sub-projects/rzscontrol to do the following:

rzscontrol /dev/ramzswap0 --init --memlimit_kb=524288 --backing_swap=/special/or/file

Which works pretty damn well :D

mwsealey: just fyi, multiple_rzs branch still does not support file as backing swap. Support for multiple ramzswap devices is still pretty much untested. Thanks for testing out this branch :)

nitin, no problem! I was curious how far it had gotten - of course backing swap never gets used here (I have a little too much RAM in my VM for safety) so I did not notice that it actually does not work, but I am very very interested in the use of files as backing swap (especially as I like to keep my systems with as few partitions as possible).

I am also only using one ramzswap device so far - I don't know what your design decision is for supporting multiple swap devices. My sole reason for using this on my VMs is because swapping really slows them down and latency of a VM to a virtual disk file (which may not be fixed size or defragmented or even hosted over iSCSI or other networking :) is too high for the tasks I want to do.

But I will also be very interested in this on production, consumer hardware (~512MB net devices) at some point. If it works for a 3GHz quad-core 64-bit (4GB RAM) compiler environment inside a VM then I think it is stress tested enough for a ~1GHz ARM7 :)

> I am also only using one ramzswap device so far - I don't know what your design decision is for supporting multiple swap devices.

Some reasons to support multiple ramzswap devices:

• Its useful when file is supported as backing swap. Swap files are naturally more flexible than creating partitions, so its expected we can have more than one of them.

• It helps scalability: each ramzswap device has its own xvmalloc pool, compression buffers etc. So, using multiple rzs devices is better than creating a single mammoth rzs device.

• Implementing multiple rzs devices requires major changes to code. So, its better to do it now when the code is relatively simple.

Nitin,

So multiple ramzswap devices means - one in RAM, one on swap partition, one on file kind of multiple? I was thinking multiple swap devices in RAM but if the intention is to have the storage medium different for each device that makes a lot more sense now :D

I am very very very interested in the ability to have an in-memory ramzswap plus extending swap files with compressed data on the fly - okay so you cannot resize swap on the fly but a swapoff/swapon and some file expansion or truncation dynamically with some helper scripts or "balloon" modules or so would come in very handy to save space on disk while keeping up with the swap requirements of a system.

Obviously using partitions means you fail if you use too much memory.. it is very difficult to resize a partition. Using files you can expand the file and get more swap even if you need to find a low-use period to do the expansion.

With compression, less data is written to the swap devices so on slow SSD devices (SD card or NOR) this becomes extremely valuable.

That basically covers my needs, and it looks like you already have them in development, so I am very pleased. I would appreciate a nudge by email or so if you need ANYTHING tested on ARM, x86 or PowerPC platforms :D

> So multiple ramzswap devices means - one in RAM, one on swap partition, one on file kind of multiple? I was thinking multiple swap devices in RAM but if the intention is to have the storage medium different for each device that makes a lot more sense now :D

Yes, multiple rzs devices means each can have separate backing medium - partition, file, or none.

> That basically covers my needs, and it looks like you already have them in development, so I am very pleased. I would appreciate a nudge by email or so if you need ANYTHING tested on ARM, x86 or PowerPC platforms :D

Thanks! The project needs good testing on ARM.  Issue #33  shows that its currently unstable on ARM. Still clueless as to whats going wrong on this platform...

Hi nitin,

Following other compression systems on Linux and awaiting an official mainlined Squashfs LZMA, I noticed this:

http://git.kernel.org/?p=linux/kernel/git/pkl/squashfs-2.6.git;a=blobdiff;f=fs/squashfs/cache.c;h=b40e37bed743ce7a11145bd5e21137353e3c2bcd;hp=f15048b862a6d242f58231be438fdab0c51f0f51;hb=4e35105a99ea0781951a859d5b3957927eb23fe5;hpb=c064983b63a1ae30ed7d12cf19935fc3074802a4

In his rationale for using the cache and fragmentation issues etc. I wondered, is this potentially a candidate for more wide use of xvmalloc? I remember when you submitted to LKML

By the way I will have patchsets for 2.6.26+ (up to 30) ramzswap and also multiple_rzs branches for people to mess around with (I am disappointed that Jaunty shipped with "compcache" and not "ramzswap", for example) because I am doing a lot of work bringing a certain patch bundle up and running progressively on a new hardware.. if anyone is really interested in these, I may do them "properly" and not hack them.. is there any interest at all?

Hi mwsealey,

> In his rationale for using the cache and fragmentation issues etc. I wondered, is this potentially a candidate for more wide use of xvmalloc? I remember when you submitted to LKML

I just had a quick look at this code but could not see a clear use case for xvmalloc. Perhaps I should look more carefully.

> By the way I will have patchsets for 2.6.26+ (up to 30) ramzswap and also multiple_rzs branches for people to mess around with (I am disappointed that Jaunty shipped with "compcache" and not "ramzswap", for example) because I am doing a lot of work bringing a certain patch bundle up and running progressively on a new hardware.. if anyone is really interested in these, I may do them "properly" and not hack them.. is there any interest at all?

Appreciate your effort but what is the need to maintain patches when you have everything available as out-of-tree kernel modules? I'm soon going make multiple_rzs branch available as out-of-tree modules too (0.6-rc1) -- still it needs more cleanups, testing and completing the help page: http://code.google.com/p/compcache/wiki/CompilingAndUsingNew

A related project - now very defunct was mergemem.

This simply did the opposite of copy-on-write - if a page became identical after being written to (say two shells that are started and perform the same initialisation in some memory structures, so a page again becomes identical) it's merged. http://mergemem.ist.org/

I used this with fairly good results on my 486 laptops at the time.

This would seem a logical addition to the project.

> A related project - now very defunct was mergemem.

This seems same as now active "KSM" (Kernel Shared Memory) project: http://lwn.net/Articles/329123/ This merges identical pages just like mergemem does.

Also, you might be interested in a project that merges almost identical pages: Difference Engine: Harnessing Memory Redundancy in Virtual Machines PDF: http://www.usenix.org/events/osdi08/tech/full_papers/gupta/gupta.pdf

I have encountered a bug while attempting insmod with both the memlimit_kb and backing_swap parameters.

This is encountered on a G1, so may be related to the insmod version or hardware. This is using the patched version of .5.x that was in the  issue 33 . http://code.google.com/p/compcache/issues/detail?id=33&can=1

The instructions above to enable a backing_swap with user defined memlimit (my settings plugged in): insmod ramzswap.ko memlimit_kb=10240 backing_swap=/dev/block/mmcblk0p3

results in a ramzswap as if no parameters. That is a ramzswap of 25% of available space with no backing_swap.

However, when the parameters are reversed: insmod ramzswap.ko backing_swap=/dev/block/mmcblk0p3 memlimit_kb=10240

the results are as expected.

Correction to above. memlimit_kb has no effect. It is defaulting to memlimit_kb of 15% when backing_swap is first.