Introduction
zram disks (/dev/zramX) are generic in-memory R/W block devices. Here we measure I/O performance of these devices and compare it to plain ramdisks.
Details
I/O performance was measured using 'dd' command
Common configuration for ram and zram disks case:
- Tests were run natively (not in a VM)
- Host OS: Fedora 11 x86_64
- Host configuration: CPU: Intel Core 2 Duo (T9600, 2.8GHz), RAM: 4G
- Boot into runlevel 3 and turn off crond, atd, cpuspeed
ram disk case
- Create 2G ram disk using ramdisk_size boot param
- mkfs.ext4 /dev/ram0
- mount /dev/ram0 /mnt/other
zram disk case
- Create 2G zram disk using zramconfig
- mkfs.ext4 /dev/zram0
- mount /dev/zram0 /mnt/other
Read benchmark
1G file was copied to (z)ramdisk which was then synchronously read into /dev/null using different block sizes.
if [ -z "$1" ]; then
echo "Missing file to read into /dev/null"
exit 1
else
FI="$1"
fi
if [ -z "$2" ]; then
echo "Missing file to dump output"
exit 1
else
FO="$2"
fi
for B in 64 128 256 512 1K 2K 4K 8K 16K 32K 64K 128K 256K 512K 1M; do
echo "$B" | tee -a "$FO"
echo 1 > /proc/sys/vm/drop_caches
echo 1 > /proc/sys/vm/drop_caches
sleep 1
dd if="$FI" of=/dev/null bs="$B" iflag=sync 2>&1 | tee -a "$FO"
done
Write benchmark
First make sure that test (512M) file was completely cached. Then synchronously write this file to (z)ramdisk.
if [ -z "$1" ]; then
echo "Missing file to be written"
exit 1
else
FI="$1"
fi
# make sure input file is cached
dd if="$FI" of=/dev/null
dd if="$FI" of=/dev/null
dd if="$FI" of=/dev/null
dd if="$FI" of=/dev/null
for B in 64 128 256 512 1K 2K 4K 8K 16K 32K 64K 128K 256K 512K 1M; do
echo "$B" | tee -a "$FO"
dd if="$FI" of=`basename "$FI"` bs="$B" oflag=sync 2>&1 | tee -a "$FO"
done
Results
Read Speed (MB/sec)
BlockSize ramdisk zramdisk
64 90.3 88.6
128 176 156
256 300 245
512 483 354
1K 695 450
2K 890 520
4K 1024 567
8K 1126.4 586
16K 1126.4 598
32K 1126.4 601
64K 1126.4 603
128K 1126.4 604
256K 1228.8 605
512K 1228.8 607
1M 1126.4 604
Write Speed (MB/sec)
BlockSize ramdisk zramdisk
64 7.3 2.9
128 13.7 5.5
256 23.7 9.7
512 37.5 15.6
1K 52.9 24.8
2K 67 29.3
4K 79.1 41.8
8K 136 58.8
16K 218 87.4
32K 310 114
64K 393 133
128K 454 147
256K 481 156
512K 500 159
1M 403 160
Is the read/write performance ceiling a function of how fast the CPU can decompress a page or some other limit?
I would have thought decompressing ram-to-ram LZO would have gotten a bit better performance on a 2.8GHz processor than 160MB/s. Could this be directly attributed to the block layer?
> Is the read/write performance ceiling a function of how fast the CPU can decompress a page or some other limit?
Even if CPUs were insanely fast, we would be limited by how fast data can be pumped from main memory to CPU.
Other factors:
I came to know about this last point only recently: LZO compressor performance seems to be highly dependent on the kind of data being compressed. See: http://denisy.dyndns.org/lzo_vs_lzjb/ I forgot what kind of data I used in this benchmark and that can be also be one of the reasons of seemingly slow 160MB/s.
(I hope wiki won't mess with formatting)
> (I hope wiki won't mess with formatting)
it did.
I have to find the research paper, but some people found a major improvement by replacing the LZO compressor with one tuned for the type of data you send to swap.
Even if zram's overhead is between 50% and 70%, it is still much faster than disk access. Since zram's main usefulness is to act as a compressed block device for swap storage, it seems a good idea to compare it against bare disk access.
An even better idea would be to show it working (and improving performance) in real world cases. I know these tests have already been done, but they're getting old by now.
@earlcolby: It would be interesting to explore LZO alternates and would be great if you can dig out those papers.
@marcodiegomesquita: Yes, comparison against spinning disks would be interesting but before I do lot of performance study again, I'm planning some of the badly missing features first. One of them being ability to write out zram (compressed) pages to backing disk/file instead of pinning its pages in memory as we currently do. This change is expected to cause major code changes and will again outdate any performance numbers.
I wonder what kind of algorithms policy would be good to decide which compressed page should go to the backing medium. Sending a compressed page that holds many uncompressed pages may not be a good idea since it has a high probability of been needed soon. Pages that are frequently used also are not good candidates to go to the backing medium.
What about some kind of speculative prefetch/posfetch that sends pages to backing medium whenever the system is idle (so that time is not wasted when the page is chosen to go the backing device) or copies pages from the backing medium to the main memory whenever there is enough free ram and the system is idle (so that no time is wasted when the page is accessed)?
The PDF I was thinking of is found at http://www.cs.umass.edu/~yannis/compression.pdf
Hope i have added something useful.
What the difference between to add vm.swappiness = 50 in /etc/sysctl.conf and zram??? right now i use zram and look greate!!!
Anyone tested with LZO 2.06 or LZ4? Also what kind of coding gain?