How Fast is Redis?Redis includes the redis-benchmark utility that simulates SETs/GETs done by N clients at the same time sending M total queries (it is similar to the Apache's ab utility). Below you'll find the full output of the benchmark executed against a Linux box. - The test was done with 50 simultaneous clients performing 100000 requests.
- The value SET and GET is a 256 bytes string.
- The Linux box is running Linux 2.6, it's Xeon X3320 2.5Ghz.
- Text executed using the loopback interface (127.0.0.1).
Results: about 110000 SETs per second, about 81000 GETs per second. Latency percentiles./redis-benchmark -n 100000
====== SET ======
100007 requests completed in 0.88 seconds
50 parallel clients
3 bytes payload
keep alive: 1
58.50% <= 0 milliseconds
99.17% <= 1 milliseconds
99.58% <= 2 milliseconds
99.85% <= 3 milliseconds
99.90% <= 6 milliseconds
100.00% <= 9 milliseconds
114293.71 requests per second
====== GET ======
100000 requests completed in 1.23 seconds
50 parallel clients
3 bytes payload
keep alive: 1
43.12% <= 0 milliseconds
96.82% <= 1 milliseconds
98.62% <= 2 milliseconds
100.00% <= 3 milliseconds
81234.77 requests per second
====== INCR ======
100018 requests completed in 1.46 seconds
50 parallel clients
3 bytes payload
keep alive: 1
32.32% <= 0 milliseconds
96.67% <= 1 milliseconds
99.14% <= 2 milliseconds
99.83% <= 3 milliseconds
99.88% <= 4 milliseconds
99.89% <= 5 milliseconds
99.96% <= 9 milliseconds
100.00% <= 18 milliseconds
68458.59 requests per second
====== LPUSH ======
100004 requests completed in 1.14 seconds
50 parallel clients
3 bytes payload
keep alive: 1
62.27% <= 0 milliseconds
99.74% <= 1 milliseconds
99.85% <= 2 milliseconds
99.86% <= 3 milliseconds
99.89% <= 5 milliseconds
99.93% <= 7 milliseconds
99.96% <= 9 milliseconds
100.00% <= 22 milliseconds
100.00% <= 208 milliseconds
88109.25 requests per second
====== LPOP ======
100001 requests completed in 1.39 seconds
50 parallel clients
3 bytes payload
keep alive: 1
54.83% <= 0 milliseconds
97.34% <= 1 milliseconds
99.95% <= 2 milliseconds
99.96% <= 3 milliseconds
99.96% <= 4 milliseconds
100.00% <= 9 milliseconds
100.00% <= 208 milliseconds
71994.96 requests per second Notes: changing the payload from 256 to 1024 or 4096 bytes does not change the numbers significantly (but reply packets are glued together up to 1024 bytes so GETs may be slower with big payloads). The same for the number of clients, from 50 to 256 clients I got the same numbers. With only 10 clients it starts to get a bit slower. You can expect different results from different boxes. For example a low profile box like Intel core duo T5500 clocked at 1.66Ghz running Linux 2.6 will output the following: ./redis-benchmark -q -n 100000
SET: 53684.38 requests per second
GET: 45497.73 requests per second
INCR: 39370.47 requests per second
LPUSH: 34803.41 requests per second
LPOP: 37367.20 requests per second Another one using a 64 bit box, a Xeon L5420 clocked at 2.5 Ghz: ./redis-benchmark -q -n 100000
PING: 111731.84 requests per second
SET: 108114.59 requests per second
GET: 98717.67 requests per second
INCR: 95241.91 requests per second
LPUSH: 104712.05 requests per second
LPOP: 93722.59 requests per second
|
Sign in to add a comment
My test on an oldish dual core Linux box with one kernel-based RAID1 volume and an average load of 0.1-0.2
Ludo: cool! I could basically handle my largest applications with a single server instead of the MySQL cluster I'm running currently :)
Dell E520 desktop with 1 Core2 cpu and 4GB RAM
I modified Redis in order to send multi-buffer (but small) replies in a single TCP packet. Results updated in this page.
I did a bit of testing on the Amazon EC2 virtual servers. The results are "a bit" disappointing for the small instance... (but it's a VPS)
P.S. If it's of any interest I have prepared a script that can be used to do the benchmarking on EC2 (launches the instances, downloads from svn, compiles, benchmarks, ...).
Fedora Base 32 bit ( ami-5647a33f )
Small ( m1.small )
High-CPU Medium ( c1.medium )
Fedora Base 64 bit ( ami-2547a34c )
Large ( m1.large )
Extra Large ( m1.xlarge )
High-CPU Extra Large ( c1.xlarge )
Hi, I've tested using PHP client.
In a Dell Inspiron 6400 (Core Duo T2250 @1.73GHz, 1 GB RAM). With Ubuntu 8.04 (linux 2.6)
I've generated 1000000 (1M) entries id=text, and took over 1'20" (1 minute 20 seconds)
With 100000 (100K) entries took 9 seconds, as I expected.
I've made the same test in MySQL in local, and the results are similar.
The redis-benchmarks:
SET
57.12% <= 0 milliseconds 99.76% <= 1 milliseconds 99.83% <= 2 milliseconds 99.86% <= 3 milliseconds 99.89% <= 4 milliseconds 99.93% <= 5 milliseconds 99.93% <= 6 milliseconds 100.00% <= 7 milliseconds 100.00% <= 203 milliseconds 78994.47 requests per second
GET
52.85% <= 0 milliseconds 99.35% <= 1 milliseconds 99.68% <= 2 milliseconds 99.79% <= 3 milliseconds 99.86% <= 4 milliseconds 99.86% <= 5 milliseconds 99.93% <= 6 milliseconds 100.00% <= 7 milliseconds 100.00% <= 8 milliseconds 100.00% <= 203 milliseconds 70926.24 requests per second
INCR
49.58% <= 0 milliseconds 99.41% <= 1 milliseconds 99.71% <= 2 milliseconds 99.81% <= 3 milliseconds 99.89% <= 4 milliseconds 99.90% <= 5 milliseconds 99.90% <= 6 milliseconds 99.96% <= 7 milliseconds 100.00% <= 8 milliseconds 100.00% <= 202 milliseconds 66757.68 requests per second
LPUSH
53.13% <= 0 milliseconds 99.56% <= 1 milliseconds 99.74% <= 2 milliseconds 99.81% <= 3 milliseconds 99.85% <= 4 milliseconds 99.86% <= 5 milliseconds 99.92% <= 6 milliseconds 100.00% <= 7 milliseconds 100.00% <= 203 milliseconds 74128.98 requests per second
LPOP
50.45% <= 0 milliseconds 99.40% <= 1 milliseconds 99.65% <= 2 milliseconds 99.69% <= 3 milliseconds 99.83% <= 4 milliseconds 99.86% <= 6 milliseconds 99.96% <= 7 milliseconds 99.97% <= 8 milliseconds 99.99% <= 16 milliseconds 100.00% <= 17 milliseconds 100.00% <= 205 milliseconds 67026.14 requests per second
root@inspiron6400:/opt/redis-beta-6# ./redis-benchmark -n 100000
SET
39.43% <= 0 milliseconds 96.91% <= 1 milliseconds 98.56% <= 2 milliseconds 99.23% <= 3 milliseconds 99.63% <= 4 milliseconds 99.72% <= 5 milliseconds 99.83% <= 6 milliseconds 99.92% <= 7 milliseconds 99.95% <= 8 milliseconds 100.00% <= 11 milliseconds 74295.69 requests per second
GET
46.01% <= 0 milliseconds 98.75% <= 1 milliseconds 99.42% <= 2 milliseconds 99.51% <= 3 milliseconds 99.72% <= 4 milliseconds 99.79% <= 5 milliseconds 99.82% <= 6 milliseconds 99.89% <= 7 milliseconds 99.89% <= 10 milliseconds 99.93% <= 11 milliseconds 99.95% <= 20 milliseconds 99.96% <= 21 milliseconds 100.00% <= 22 milliseconds 100.00% <= 208 milliseconds 59560.45 requests per second
INCR
42.83% <= 0 milliseconds 97.54% <= 1 milliseconds 98.76% <= 2 milliseconds 99.32% <= 3 milliseconds 99.48% <= 4 milliseconds 99.63% <= 5 milliseconds 99.79% <= 6 milliseconds 99.81% <= 7 milliseconds 99.85% <= 9 milliseconds 99.89% <= 11 milliseconds 99.92% <= 12 milliseconds 99.94% <= 14 milliseconds 99.96% <= 15 milliseconds 99.99% <= 26 milliseconds 100.00% <= 27 milliseconds 100.00% <= 208 milliseconds 57013.11 requests per second
LPUSH
54.74% <= 0 milliseconds 99.32% <= 1 milliseconds 99.70% <= 2 milliseconds 99.83% <= 3 milliseconds 99.86% <= 4 milliseconds 99.87% <= 5 milliseconds 99.89% <= 6 milliseconds 99.97% <= 7 milliseconds 99.98% <= 16 milliseconds 100.00% <= 17 milliseconds 100.00% <= 205 milliseconds 73338.71 requests per second
LPOP
45.98% <= 0 milliseconds 99.43% <= 1 milliseconds 99.88% <= 2 milliseconds 99.96% <= 3 milliseconds 99.96% <= 5 milliseconds 100.00% <= 7 milliseconds 100.00% <= 208 milliseconds 68448.32 requests per second
I think the PHP client is not properly tuned.
If any is interested I send to him the code I've used.
Hello jhernandis, I don't have numbers about the PHP client performances but I need more data about your tests. The PHP script, the average length of the values SET and so on. Note that if you are adding keys with a single connection you can't expect the same performance of the benchmarks, since they simulate 50 simultaneous clients, so the time spent in the 'round trip time' of request-reply is amortized. If you need to set a lot of keys from a single connection and make it very fast you should use pipelining instead. Thanks!
@luca: thanks for this benchmarks. The small seems a bit too slow... indeed
@antirez, I've sent you the code of my tests (I've supossed your mail is antirez AT gmail.com, don't?)
I've used a file containing 1000 phrases from "lorem ipsum" for data, and a for($i=0; $i<1000000M $i++) { $r->set('id' . $i, $data$datapos++?);.......
@jhernandis: thanks I'll look at the code later, but I think the numbers you got are more or less ok:
1000000 sets /80 sec = 12500 query/sec, that is what you will get more or less using the benchmark with '-c 1', add to this a bit of PHP overhead and the sentences that are a bit bigger, and what you get is that you are actually measuring more the round trip time that Redis performances! :)
In order to compare this numbers with MySQL what you really need is this:
Use a client to perform continuous writes, and N clients performing random reads. Under load Redis will have performances similar to the vanilla benchmark, while MySQL will start to degrade and get a lot slower.
Another test you may want to do is to use pipelining to send N queries, with N big (for example 1000 or 10000) and then get N replies, and so on. This will bring you again the numbers you see in the benchmarks.
@antirez: Thanks for your explain. I'll test again using the enviroment that you describes.
amd 2.1GHz(single core), 2G RAM
redis-benchmark -q -n 100000
ruby bench.rb
Rehearsal --------------------------------------- set 3.220000 0.710000 3.930000 ( 5.794546) ------------------------------ total: 3.930000sec user system total real set 2.970000 0.670000 3.640000 ( 5.281302)@impactplayr: that's a bit slow, what kind of OS? This is what I get under Linux emulated under vmware in a macbook (Inte Core Duo CPU T8300 @ 2.40GHz):
Another benchmark against a 64 bit Linux box, Xeon L5420 clocked at 2.5 Ghz:
@antirez Ubuntu 8.10 32bit
A slightly aging machine, I suppose: Dual AMD 285 Opteron, 16GB ram. (tbh, I had expected more)
Ubuntu 8.04 LTS x64
Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz
MemTotal?: 8189780 kB
redis-0.091
I tried ./redis-benchmark -n 1000000 (note the extra zero)
"top", "1" shows:
Mem: 8189780k total, 1043512k used, 7146268k free, 95896k buffers Swap: 9968292k total, 0k used, 9968292k free, 625096k cached
@maxdemarzi: very interesting! Especially the CPU usage bit. Should I guess all this time spent in software interrupts is about the loopback interface?
Btw it's worth to note that every benchmark against loopback must consume 100% of CPU in total otherwise it means there are idle times to remove in the code somewhere.
Now with version 0.091:
conf1:
conf2 changes only "shareobjects" to "yes"
It appears that the new option, shareobjects, does give a bit of a performance boost.
I did put some critical comments on using Redis on EC2: http://michalfrackowiak.com/blog:redis-performance
On OS X on a 2.66GHz Core 2 Duo MacBook? Pro I get:
$ redis-benchmark -q -n 100000 PING: 28646.62 requests per second SET: 27887.62 requests per second GET: 25755.09 requests per second INCR: 25712.85 requests per second LPUSH: 27951.09 requests per second LPOP: 25544.32 requests per second
Seems very slow (well, not really, but in comparison to other numbers given here), but perhaps is an OS X thing. I'll be using it in production on Linux anyway :)
Hello pcooper, indeed under Mac OS X the benchmark shows numbers that are very far compared to Linux. My best guess is that this is a not very optimized loopback interface implementation but I'm not sure about it, tests using a real interface against Linux and Mac OS X in similar conditions are really needed in order to verify this guess.
For comparisons against MySQL: http://colinhowe.wordpress.com/2009/04/27/redis-vs-mysql/
Why nobody considers testing sets performance ?? The most goal I'm looking at redis it's ability to operate on sets.. but I've got terrible result.. it does 1000 SADDs in about 40 sec! on Core Duo E6850 3.00GHz
why it's so much slooow? anybody else experienced that ?
For me, operations with sets are only a little bit slower then with lists or strings.
you're right, my excuse.. something wrong with my redis API, I've put sets tests into redis-benchmark and it worked quite well:
hi,
have anybody made benchmark with keep alive disabled?
i've run benchmark:
./redis-benchmark -c 100 -n 10000 -d 200 -k 0 -l
and i've got a lot of:
Connect: connect: Cannot assign requested address
Connect: connect: Cannot assign requested address
Connect: connect: Cannot assign requested address
Connect: connect: Cannot assign requested address
is it normal?
@sebpaa: try this: "sudo sysctl -w net.inet.tcp.msl=1000" if you are using a Mac. In Linux instead use the following: "echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse"
great, thx (i didnt see that hint while running benchmark... )
I have done some benchmarking using Redis 1.0.2 on: Amazon EC2 Flexiscale Slichost (albeit a tiny vm)
I have compiled it all into a Google Docs spreadsheet and done some cost analysis for both request throughput and available storage. Please let me know if you find it useful:
http://spreadsheets.google.com/ccc?key=0AhcHKeq_S228dDE3QzlES2V3RHhCbmh5MDlQVjdhc1E&hl=en
PS. It would be great to put some bare-metal results on there. If someone could send me benchmarks for 1.0.2 using: ./redis-benchmark -n 100000 ./redis-benchmark -n 10000 -d 200
...along with the server spec and monthly cost of their hosting, I would be happy to add it to the analysis.
I have just written a blog post analysing the above results:
http://porteightyeight.com/2009/11/09/redis-benchmarking-on-amazon-ec2-flexiscale-and-slicehost/