We moved to Redis.io!

My test on an oldish dual core Linux box with one kernel-based RAID1 volume and an average load of 0.1-0.2

./redis-benchmark -n 100000


*** SET TEST ***
== 100003 requests completed in 1.83 seconds
== 50 parallel clients
== 3 bytes payload
== keep alive: 1

36.67% <= 0 milliseconds
99.85% <= 1 milliseconds
99.95% <= 2 milliseconds
99.96% <= 3 milliseconds
99.98% <= 5 milliseconds
100.00% <= 6 milliseconds
100.00% <= 210 milliseconds

== 54527.26 requests per second


*** GET TEST ***
== 100000 requests completed in 2.45 seconds
== 50 parallel clients
== 3 bytes payload
== keep alive: 1

0.09% <= 0 milliseconds
79.16% <= 1 milliseconds
99.80% <= 2 milliseconds
99.83% <= 3 milliseconds
99.88% <= 4 milliseconds
99.90% <= 5 milliseconds
99.90% <= 6 milliseconds
99.94% <= 8 milliseconds
99.95% <= 9 milliseconds
99.95% <= 10 milliseconds
99.97% <= 26 milliseconds
100.00% <= 27 milliseconds

== 40733.20 requests per second

Ludo: cool! I could basically handle my largest applications with a single server instead of the MySQL cluster I'm running currently :)

Dell E520 desktop with 1 Core2 cpu and 4GB RAM

*** SET TEST ***
== 100001 requests completed in 0.80 seconds
== 50 parallel clients
== 3 bytes payload
== keep alive: 1

60.44% <= 0 milliseconds
99.97% <= 1 milliseconds
99.99% <= 2 milliseconds
99.99% <= 3 milliseconds
100.00% <= 4 milliseconds
100.00% <= 5 milliseconds

== 125629.40 requests per second


*** GET TEST ***
== 100001 requests completed in 1.42 seconds
== 50 parallel clients
== 3 bytes payload
== keep alive: 1

50.83% <= 0 milliseconds
99.97% <= 1 milliseconds
99.99% <= 2 milliseconds
99.99% <= 3 milliseconds
100.00% <= 4 milliseconds
100.00% <= 5 milliseconds
100.00% <= 207 milliseconds

== 70373.68 requests per second


*** INCR TEST ***
== 100000 requests completed in 1.19 seconds
== 50 parallel clients
== 3 bytes payload
== keep alive: 1

40.67% <= 0 milliseconds
100.00% <= 1 milliseconds
100.00% <= 2 milliseconds

== 84033.61 requests per second

I modified Redis in order to send multi-buffer (but small) replies in a single TCP packet. Results updated in this page.

I did a bit of testing on the Amazon EC2 virtual servers. The results are "a bit" disappointing for the small instance... (but it's a VPS)

P.S. If it's of any interest I have prepared a script that can be used to do the benchmarking on EC2 (launches the instances, downloads from svn, compiles, benchmarks, ...).

Fedora Base 32 bit ( ami-5647a33f )

• 1.7 GB memory
• 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit)
• 160 GB instance storage (150 GB plus 10 GB root partition)
• 32-bit platform
• I/O Performance: Moderate

SET: 9263.55 requests per second
GET: 8840.95 requests per second
INCR: 8607.33 requests per second
LPUSH: 9144.11 requests per second
LPOP: 8813.68 requests per second

High-CPU Medium ( c1.medium )

• 1.7 GB of memory
• 5 EC2 Compute Units (2 virtual cores with 2.5 EC2 Compute Units each)
• 350 GB of instance storage
• 32-bit platform
• I/O Performance: Moderate

SET: 20068.23 requests per second
GET: 19342.36 requests per second
INCR: 19264.11 requests per second
LPUSH: 19872.81 requests per second
LPOP: 19290.12 requests per second

Fedora Base 64 bit ( ami-2547a34c )

• 7.5 GB memory
• 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
• 850 GB instance storage (2×420 GB plus 10 GB root partition)
• 64-bit platform
• I/O Performance: High

SET: 44209.55 requests per second
GET: 43802.02 requests per second
INCR: 29943.43 requests per second
LPUSH: 30308.39 requests per second
LPOP: 28438.73 requests per second

Extra Large ( m1.xlarge )

• 15 GB memory
• 8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each)
• 1,690 GB instance storage (4×420 GB plus 10 GB root partition)
• 64-bit platform
• I/O Performance: High

SET: 46932.43 requests per second
GET: 44705.86 requests per second
INCR: 44702.73 requests per second
LPUSH: 47688.13 requests per second
LPOP: 46279.50 requests per second

High-CPU Extra Large ( c1.xlarge )

• 7 GB of memory
• 20 EC2 Compute Units (8 virtual cores with 2.5 EC2 Compute Units each)
• 1690 GB of instance storage
• 64-bit platform
• I/O Performance: High

SET: 62114.91 requests per second
GET: 60756.38 requests per second
INCR: 58789.54 requests per second
LPUSH: 60425.98 requests per second
LPOP: 56660.06 requests per second

Hi, I've tested using PHP client.

In a Dell Inspiron 6400 (Core Duo T2250 @1.73GHz, 1 GB RAM). With Ubuntu 8.04 (linux 2.6)

I've generated 1000000 (1M) entries id=text, and took over 1'20" (1 minute 20 seconds)

With 100000 (100K) entries took 9 seconds, as I expected.

I've made the same test in MySQL in local, and the results are similar.

The redis-benchmarks:

SET

1. 0007 requests completed in 1.27 seconds

50 parallel clients 3 bytes payload keep alive: 1

57.12% <= 0 milliseconds 99.76% <= 1 milliseconds 99.83% <= 2 milliseconds 99.86% <= 3 milliseconds 99.89% <= 4 milliseconds 99.93% <= 5 milliseconds 99.93% <= 6 milliseconds 100.00% <= 7 milliseconds 100.00% <= 203 milliseconds 78994.47 requests per second

GET

1. 0006 requests completed in 1.41 seconds

50 parallel clients 3 bytes payload keep alive: 1

52.85% <= 0 milliseconds 99.35% <= 1 milliseconds 99.68% <= 2 milliseconds 99.79% <= 3 milliseconds 99.86% <= 4 milliseconds 99.86% <= 5 milliseconds 99.93% <= 6 milliseconds 100.00% <= 7 milliseconds 100.00% <= 8 milliseconds 100.00% <= 203 milliseconds 70926.24 requests per second

INCR

1. 0003 requests completed in 1.50 seconds

50 parallel clients 3 bytes payload keep alive: 1

49.58% <= 0 milliseconds 99.41% <= 1 milliseconds 99.71% <= 2 milliseconds 99.81% <= 3 milliseconds 99.89% <= 4 milliseconds 99.90% <= 5 milliseconds 99.90% <= 6 milliseconds 99.96% <= 7 milliseconds 100.00% <= 8 milliseconds 100.00% <= 202 milliseconds 66757.68 requests per second

LPUSH

1. 0000 requests completed in 1.35 seconds

50 parallel clients 3 bytes payload keep alive: 1

53.13% <= 0 milliseconds 99.56% <= 1 milliseconds 99.74% <= 2 milliseconds 99.81% <= 3 milliseconds 99.85% <= 4 milliseconds 99.86% <= 5 milliseconds 99.92% <= 6 milliseconds 100.00% <= 7 milliseconds 100.00% <= 203 milliseconds 74128.98 requests per second

LPOP

1. 0003 requests completed in 1.49 seconds

50 parallel clients 3 bytes payload keep alive: 1

50.45% <= 0 milliseconds 99.40% <= 1 milliseconds 99.65% <= 2 milliseconds 99.69% <= 3 milliseconds 99.83% <= 4 milliseconds 99.86% <= 6 milliseconds 99.96% <= 7 milliseconds 99.97% <= 8 milliseconds 99.99% <= 16 milliseconds 100.00% <= 17 milliseconds 100.00% <= 205 milliseconds 67026.14 requests per second

root@inspiron6400:/opt/redis-beta-6# ./redis-benchmark -n 100000

SET

1. 0002 requests completed in 1.35 seconds

50 parallel clients 3 bytes payload keep alive: 1

39.43% <= 0 milliseconds 96.91% <= 1 milliseconds 98.56% <= 2 milliseconds 99.23% <= 3 milliseconds 99.63% <= 4 milliseconds 99.72% <= 5 milliseconds 99.83% <= 6 milliseconds 99.92% <= 7 milliseconds 99.95% <= 8 milliseconds 100.00% <= 11 milliseconds 74295.69 requests per second

GET

1. 0002 requests completed in 1.68 seconds

50 parallel clients 3 bytes payload keep alive: 1

46.01% <= 0 milliseconds 98.75% <= 1 milliseconds 99.42% <= 2 milliseconds 99.51% <= 3 milliseconds 99.72% <= 4 milliseconds 99.79% <= 5 milliseconds 99.82% <= 6 milliseconds 99.89% <= 7 milliseconds 99.89% <= 10 milliseconds 99.93% <= 11 milliseconds 99.95% <= 20 milliseconds 99.96% <= 21 milliseconds 100.00% <= 22 milliseconds 100.00% <= 208 milliseconds 59560.45 requests per second

INCR

1. 0001 requests completed in 1.75 seconds

50 parallel clients 3 bytes payload keep alive: 1

42.83% <= 0 milliseconds 97.54% <= 1 milliseconds 98.76% <= 2 milliseconds 99.32% <= 3 milliseconds 99.48% <= 4 milliseconds 99.63% <= 5 milliseconds 99.79% <= 6 milliseconds 99.81% <= 7 milliseconds 99.85% <= 9 milliseconds 99.89% <= 11 milliseconds 99.92% <= 12 milliseconds 99.94% <= 14 milliseconds 99.96% <= 15 milliseconds 99.99% <= 26 milliseconds 100.00% <= 27 milliseconds 100.00% <= 208 milliseconds 57013.11 requests per second

LPUSH

1. 0034 requests completed in 1.36 seconds

50 parallel clients 3 bytes payload keep alive: 1

54.74% <= 0 milliseconds 99.32% <= 1 milliseconds 99.70% <= 2 milliseconds 99.83% <= 3 milliseconds 99.86% <= 4 milliseconds 99.87% <= 5 milliseconds 99.89% <= 6 milliseconds 99.97% <= 7 milliseconds 99.98% <= 16 milliseconds 100.00% <= 17 milliseconds 100.00% <= 205 milliseconds 73338.71 requests per second

LPOP

1. 0003 requests completed in 1.46 seconds

50 parallel clients 3 bytes payload keep alive: 1

45.98% <= 0 milliseconds 99.43% <= 1 milliseconds 99.88% <= 2 milliseconds 99.96% <= 3 milliseconds 99.96% <= 5 milliseconds 100.00% <= 7 milliseconds 100.00% <= 208 milliseconds 68448.32 requests per second

I think the PHP client is not properly tuned.

If any is interested I send to him the code I've used.

Hello jhernandis, I don't have numbers about the PHP client performances but I need more data about your tests. The PHP script, the average length of the values SET and so on. Note that if you are adding keys with a single connection you can't expect the same performance of the benchmarks, since they simulate 50 simultaneous clients, so the time spent in the 'round trip time' of request-reply is amortized. If you need to set a lot of keys from a single connection and make it very fast you should use pipelining instead. Thanks!

@luca: thanks for this benchmarks. The small seems a bit too slow... indeed

@antirez, I've sent you the code of my tests (I've supossed your mail is antirez AT gmail.com, don't?)

I've used a file containing 1000 phrases from "lorem ipsum" for data, and a for($i=0; $i<1000000M $i++) { $r->set('id' . $i, $data[$datapos++]);.......

@jhernandis: thanks I'll look at the code later, but I think the numbers you got are more or less ok:

1000000 sets /80 sec = 12500 query/sec, that is what you will get more or less using the benchmark with '-c 1', add to this a bit of PHP overhead and the sentences that are a bit bigger, and what you get is that you are actually measuring more the round trip time that Redis performances! :)

In order to compare this numbers with MySQL what you really need is this:

Use a client to perform continuous writes, and N clients performing random reads. Under load Redis will have performances similar to the vanilla benchmark, while MySQL will start to degrade and get a lot slower.

Another test you may want to do is to use pipelining to send N queries, with N big (for example 1000 or 10000) and then get N replies, and so on. This will bring you again the numbers you see in the benchmarks.

@antirez: Thanks for your explain. I'll test again using the enviroment that you describes.

amd 2.1GHz(single core), 2G RAM

redis-benchmark -q -n 100000

PING: 30445.05 requests per second
SET: 36594.59 requests per second
GET: 28957.16 requests per second
INCR: 27976.23 requests per second
LPUSH: 31125.74 requests per second
LPOP: 27297.68 requests per second

ruby bench.rb

Rehearsal ---------------------------------------
set   3.220000   0.710000   3.930000 (  5.794546)
------------------------------ total: 3.930000sec

          user     system      total        real
set   2.970000   0.670000   3.640000 (  5.281302)

@impactplayr: that's a bit slow, what kind of OS? This is what I get under Linux emulated under vmware in a macbook (Inte Core Duo CPU T8300 @ 2.40GHz):

$ ./redis-benchmark -q -n 100000
PING: 41178.68 requests per second
SET: 40962.73 requests per second
GET: 43805.08 requests per second
INCR: 42428.09 requests per second
LPUSH: 40488.66 requests per second
LPOP: 41003.28 requests per second

Another benchmark against a 64 bit Linux box, Xeon L5420 clocked at 2.5 Ghz:

PING: 111731.84 requests per second
SET: 108114.59 requests per second
GET: 98717.67 requests per second
INCR: 95241.91 requests per second
LPUSH: 104712.05 requests per second
LPOP: 93722.59 requests per second

@antirez Ubuntu 8.10 32bit

# ./redis-benchmark -q -n 100000
 PING : 49263.05 requests per second
  SET : 46729.44 requests per second
  GET : 43649.50 requests per second
 INCR : 39654.78 requests per second
LPUSH : 42019.32 requests per second
 LPOP : 40883.07 requests per second

A slightly aging machine, I suppose: Dual AMD 285 Opteron, 16GB ram. (tbh, I had expected more)

Ubuntu 8.04 LTS x64

Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz

MemTotal?: 8189780 kB

redis-0.091

I tried ./redis-benchmark -n 1000000 (note the extra zero)

 || PING   ||  78021.45 requests per second  ||  completed in 12.83 seconds || 
 || SET    ||  76167.72 requests per second  ||  completed in 13.16 seconds || 
 || GET    ||  70358.41 requests per second  ||  completed in 14.23 seconds || 
 || INCR   ||  67526.51 requests per second  ||  completed in 14.86 seconds || 
 || LPUSH  ||  73850.38 requests per second  ||  completed in 13.54 seconds || 
 || LPOP   ||  66432.14 requests per second  ||  completed in 15.06 seconds || 

"top", "1" shows:

 || Cpu0   ||  14.3%us ||  49.7%sy ||   0.0%ni ||   0.0%id ||   0.0%wa ||   0.0%hi ||  36.0%si ||   0.0%st || 
 || Cpu1   ||   0.0%us ||   0.3%sy ||   0.0%ni ||  99.7%id ||   0.0%wa ||   0.0%hi ||   0.0%si ||   0.0%st || 
 || Cpu2   ||   0.0%us ||   0.0%sy ||   0.0%ni || 100.0%id ||   0.0%wa ||   0.0%hi ||   0.0%si ||   0.0%st || 
 || Cpu3   ||  12.7%us ||  49.7%sy ||   0.0%ni ||   0.0%id ||   0.0%wa ||   0.0%hi ||  37.7%si ||   0.0%st || 

Mem: 8189780k total, 1043512k used, 7146268k free, 95896k buffers Swap: 9968292k total, 0k used, 9968292k free, 625096k cached

@maxdemarzi: very interesting! Especially the CPU usage bit. Should I guess all this time spent in software interrupts is about the loopback interface?

Btw it's worth to note that every benchmark against loopback must consume 100% of CPU in total otherwise it means there are idle times to remove in the code somewhere.

Now with version 0.091:

conf1:

daemonize yes
pidfile /var/run/redis.pid
port 6379
bind 127.0.0.1
timeout 300
save 900 1
dir ./
logfile /dev/null
databases 16
glueoutputbuf yes
shareobjects no

conf2 changes only "shareobjects" to "yes"

It appears that the new option, shareobjects, does give a bit of a performance boost.

I did put some critical comments on using Redis on EC2: http://michalfrackowiak.com/blog:redis-performance

On OS X on a 2.66GHz Core 2 Duo MacBook? Pro I get:

$ redis-benchmark -q -n 100000 PING: 28646.62 requests per second SET: 27887.62 requests per second GET: 25755.09 requests per second INCR: 25712.85 requests per second LPUSH: 27951.09 requests per second LPOP: 25544.32 requests per second

Seems very slow (well, not really, but in comparison to other numbers given here), but perhaps is an OS X thing. I'll be using it in production on Linux anyway :)

Hello pcooper, indeed under Mac OS X the benchmark shows numbers that are very far compared to Linux. My best guess is that this is a not very optimized loopback interface implementation but I'm not sure about it, tests using a real interface against Linux and Mac OS X in similar conditions are really needed in order to verify this guess.

For comparisons against MySQL: http://colinhowe.wordpress.com/2009/04/27/redis-vs-mysql/

Why nobody considers testing sets performance ?? The most goal I'm looking at redis it's ability to operate on sets.. but I've got terrible result.. it does 1000 SADDs in about 40 sec! on Core Duo E6850 3.00GHz

why it's so much slooow? anybody else experienced that ?

For me, operations with sets are only a little bit slower then with lists or strings.

you're right, my excuse.. something wrong with my redis API, I've put sets tests into redis-benchmark and it worked quite well:

./redis-benchmark -q -n 100000

SET: 35473.05 requests per second
GET: 55145.54 requests per second
INCR: 35481.73 requests per second
LPUSH: 35152.90 requests per second
LPOP: 40093.39 requests per second
SADD: 36719.53 requests per second
SREM: 34859.93 requests per second
PING: 50529.29 requests per second

hi,

have anybody made benchmark with keep alive disabled?

i've run benchmark:

./redis-benchmark -c 100 -n 10000 -d 200 -k 0 -l

and i've got a lot of:

Connect: connect: Cannot assign requested address

Connect: connect: Cannot assign requested address

Connect: connect: Cannot assign requested address

Connect: connect: Cannot assign requested address

is it normal?

@sebpaa: try this: "sudo sysctl -w net.inet.tcp.msl=1000" if you are using a Mac. In Linux instead use the following: "echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse"

great, thx (i didnt see that hint while running benchmark... )

I have done some benchmarking using Redis 1.0.2 on: Amazon EC2 Flexiscale Slichost (albeit a tiny vm)

I have compiled it all into a Google Docs spreadsheet and done some cost analysis for both request throughput and available storage. Please let me know if you find it useful:

http://spreadsheets.google.com/ccc?key=0AhcHKeq_S228dDE3QzlES2V3RHhCbmh5MDlQVjdhc1E&hl=en

PS. It would be great to put some bare-metal results on there. If someone could send me benchmarks for 1.0.2 using: ./redis-benchmark -n 100000 ./redis-benchmark -n 10000 -d 200

...along with the server spec and monthly cost of their hosting, I would be happy to add it to the analysis.

I have just written a blog post analysing the above results:

http://porteightyeight.com/2009/11/09/redis-benchmarking-on-amazon-ec2-flexiscale-and-slicehost/

My benchmark result for Mac OSX 10.5

centurydaily-lm:redis sabhinav$ ./redis-benchmark -q -n 100000 SET: 30442.01 requests per second GET: 23855.24 requests per second INCR: 27178.53 requests per second LPUSH: 30552.23 requests per second LPOP: 24934.68 requests per second PING: 32311.79 requests per second LPUSH (again, in order to bench LRANGE): 32306.85 requests per second LRANGE (first 100 elements): 4253.33 requests per second LRANGE (first 300 elements): 1377.75 requests per second LRANGE (first 450 elements): 900.67 requests per second LRANGE (first 600 elements): 685.02 requests per second

01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Error writing to client: Broken pipe 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:07 . Client closed connection 01 Dec 20:40:08 . DB 0: 3 keys (0 volatile) in 4 slots HT. 01 Dec 20:40:08 . 0 clients connected (0 slaves), 11991936 bytes in use, 0 shared objects 01 Dec 20:40:13 . DB 0: 3 keys (0 volatile) in 4 slots HT. 01 Dec 20:40:13 . 0 clients connected (0 slaves), 11991936 bytes in use, 0 shared objects 01 Dec 20:40:18 . DB 0: 3 keys (0 volatile) in 4 slots HT. 01 Dec 20:40:18 . 0 clients connected (0 slaves), 11991936 bytes in use, 0 shared objects

Why a poor performance on LRANGE, i m not sure. Also connections reported as broken pipes.

Otherwise a kick ass server.

Hey acharnock,

Awesome benchmarks. One thing to keep in mind when comparing Amazon to someone like slicehost is that Slicehost allows for bursting. So, if other people on the machine are not using resources you will get a boost in performance. This makes it hard to truly measure because as soon as the server becomes busy all of a sudden your slicehost server won't be performing top notch.

> Why a poor performance on LRANGE, i m not sure.

Why? It's obvious that server can't return hundreds of records as fast as single record.

redis-benchmark -n 1000000

SET

1. 00009 requests completed in 6.75 seconds

50 parallel clients 3 bytes payload keep alive: 1

66.92% <= 0 milliseconds 99.80% <= 1 milliseconds 99.93% <= 2 milliseconds 99.94% <= 3 milliseconds 99.96% <= 4 milliseconds 99.97% <= 5 milliseconds 99.98% <= 7 milliseconds 99.99% <= 8 milliseconds 99.99% <= 15 milliseconds 100.00% <= 16 milliseconds 100.00% <= 32 milliseconds 148127.53 requests per second

GET

1. 00003 requests completed in 7.05 seconds

50 parallel clients 3 bytes payload keep alive: 1

65.08% <= 0 milliseconds 99.93% <= 1 milliseconds 99.93% <= 2 milliseconds 99.97% <= 4 milliseconds 99.97% <= 5 milliseconds 99.97% <= 6 milliseconds 99.99% <= 7 milliseconds 100.00% <= 8 milliseconds 141804.17 requests per second

INCR

1. 00047 requests completed in 9.39 seconds

50 parallel clients 3 bytes payload keep alive: 1

54.45% <= 0 milliseconds 98.89% <= 1 milliseconds 99.93% <= 2 milliseconds 99.96% <= 4 milliseconds 99.97% <= 5 milliseconds 99.97% <= 6 milliseconds 99.99% <= 7 milliseconds 100.00% <= 8 milliseconds 106501.27 requests per second

LPUSH

1. 00007 requests completed in 6.62 seconds

50 parallel clients 3 bytes payload keep alive: 1

67.16% <= 0 milliseconds 99.94% <= 1 milliseconds 99.95% <= 2 milliseconds 99.98% <= 4 milliseconds 99.98% <= 5 milliseconds 99.99% <= 7 milliseconds 100.00% <= 8 milliseconds 150944.45 requests per second

LPOP

1. 00001 requests completed in 6.97 seconds

50 parallel clients 3 bytes payload keep alive: 1

65.47% <= 0 milliseconds 99.93% <= 1 milliseconds 99.93% <= 2 milliseconds 99.93% <= 3 milliseconds 99.97% <= 4 milliseconds 99.98% <= 5 milliseconds 100.00% <= 7 milliseconds 100.00% <= 8 milliseconds 143369.33 requests per second

PING

1. 00006 requests completed in 5.72 seconds

50 parallel clients 3 bytes payload keep alive: 1

71.63% <= 0 milliseconds 99.95% <= 1 milliseconds 99.95% <= 3 milliseconds 99.98% <= 4 milliseconds 99.99% <= 5 milliseconds 100.00% <= 7 milliseconds 100.00% <= 8 milliseconds 174734.58 requests per second

Wow...Redis does not perform well when virtualized! Here's some simple benchmark results between an 8GB, 4 vCPU monster Cloud Server (RackSpace?) and...a 1GB, 1vCPU VirtualBox? VM running on my underpowered laptop. Both are running Ubuntu 9.10 x64. Not only are neither anywhere close to the expected performance, but my weakling laptop actually beats the bigger, arguably more capable Cloud Server!

RackSpace? Cloud Server

root@Redis:~/redis-1.1.95-beta# ./redis-benchmark -q -n 100000 SET: 32900.00 requests per second GET: 23049.54 requests per second INCR: 19276.88 requests per second LPUSH: 19455.25 requests per second LPOP: 19531.25 requests per second PING: 21834.06 requests per second LPUSH (again, in order to bench LRANGE): 20618.56 requests per second LRANGE (first 100 elements): 4500.27 requests per second LRANGE (first 300 elements): 1366.31 requests per second LRANGE (first 450 elements): 1047.67 requests per second LRANGE (first 600 elements): 762.72 requests per second

root@Redis:~/redis-1.02# ./redis-benchmark -q -n 100000 SET: 29242.40 requests per second GET: 28901.73 requests per second INCR: 27551.79 requests per second LPUSH: 28901.73 requests per second LPOP: 27247.96 requests per second PING: 30395.14 requests per second

VirtualBox? on localhost

john@ubuntu:~/redis-1.1.95-beta$ ./redis-benchmark -q -n 100000 SET: 44822.95 requests per second GET: 43535.05 requests per second INCR: 41084.63 requests per second LPUSH: 42955.32 requests per second LPOP: 43572.98 requests per second PING: 47460.84 requests per second LPUSH (again, in order to bench LRANGE): 45808.52 requests per second LRANGE (first 100 elements): 7945.75 requests per second LRANGE (first 300 elements): 1849.93 requests per second LRANGE (first 450 elements): 1204.59 requests per second LRANGE (first 600 elements): 854.87 requests per second

it seems everyone in here is using the redis-benchmark utility to do benchmarks. if you look carefully into it's source code, it's doing some pretty strange stuff - using same values for each query for example. there's a benchmark at sourceforge that shows much darker numbers in 1000's range and falling fast with database saturation, but something is most probably wrong with it too.. at least it's tests are supposedly based on real-life use-case: http://sourceforge.net/projects/dbbenchmark/

if somebody can comment on redis performance on real applications with real traffic, please do it. a long thread based on "./redis-benchmark" runs and no real tests seems really weird.

@equoblog

Hello!

This is why you should reconsider your statements:

a) Redis-benchmark can SET/GET random keys. Just use the "-r" option. For instance "-r 1000000" means use random keys, in the range 0-999999. Performances are exactly the same:

antirez@wwwsmuvi2:/tmp/redis-1.2.0$ ./redis-benchmark -q
SET: 78818.90 requests per second
GET: 74103.70 requests per second
INCR: 60987.80 requests per second
LPUSH: 77542.64 requests per second
LPOP: 76953.85 requests per second
PING: 69041.38 requests per second
LPUSH (again, in order to bench LRANGE): 78140.62 requests per second

antirez@wwwsmuvi2:/tmp/redis-1.2.0$ ./redis-benchmark -q -r 1000000000
SET: 73588.23 requests per second
GET: 71000.00 requests per second
INCR: 67567.57 requests per second
LPUSH: 78273.44 requests per second
LPOP: 77519.38 requests per second
PING: 76419.85 requests per second
LPUSH (again, in order to bench LRANGE): 80725.80 requests per second

You can try with different "-r" values and what you'll get will be the same results? Why? Read "b"

b) With in-memory databases things are very different. Operations of very different types are going to be the same speed. Did you noticed that PING is the same speed as LPUSH? Yes, this is why the networking overhead is so big compared to the fast in memory operations that almost all the time is spent in I/O and request parsing and handling.

c) Most benchmarks you'll read not showing the same performances as Redis-benchmark are broken in one this two ways: 1) they are single threaded. So what they measure is Round Trip Time, not performances. 2) they use databases that don't fit in memory (but without the brand new Virtual Memory feature in Redis Git) so the OS is swapping like mad.

d) There are real-world usage of Redis showing that with 200/300 requests/second the CPU usage is mostly 0%. Guess what?

So please, before to claim that Redis is not as fast as I cliam, try to design your own test with good methodology and show us that I and redis-benchmark are wrong.

Sorry for the aggressive tone of this email but I think that if you was not able to check that there was a "-r" option in the redis-benchmark utility your research was very poor and still you are here claiming that Redis can handle 1000 requests/second in the real world at best.

@jazzman007 Redis can actually perform well on a VPS, here is an example running on 512MB RAM with 8vCPUS:

~/redis-1.2.0# ./redis-benchmark -q
SET: 60283.14 requests per second
GET: 58847.06 requests per second
INCR: 58479.53 requests per second
LPUSH: 59218.93 requests per second
LPOP: 57188.57 requests per second
PING: 58894.12 requests per second
LPUSH (again, in order to bench LRANGE): 59218.93 requests per second
LRANGE (first 100 elements): 10810.81 requests per second
LRANGE (first 300 elements): 3357.96 requests per second
LRANGE (first 450 elements): 2217.79 requests per second
LRANGE (first 600 elements): 1654.81 requests per second

@antirez Thanks for explanation, and don't take it the wrong way, like Redis immensely, though confused and disappointed with the existing benchmarking tools for it. Checked the -r options already before writing :) The point was about values that are being set in the benchmark. You're basically SETting a lot of keys to the same value. That's not a use case from real life even remotely. And what's with the GET's ? All of them return the same value, the value itself being very small. Basically, the benchmark works on a hyper-artificial toy task and provides numbers that are, it seems, at least partially misleading. A benchmark on some real-life workload would be much more interesting. The one on sourceforge claims to use a real workload and gets very low performance on that job from both MySQL and Redis. Both the starting performance and the slowdown with DB saturation seems to be similar, with Redis being slower on average.

Can you please update the benchmark code to work closer to "real-life" scenarios or explain why on the sf benchmark Redis is not flying as it should ?

Thanks.

p.s. your tone was quite appropriate, hope that there is some misconfiguration that can explain the slowdown.

getting some interesting performance on my win32 machine :) - just compiled it with cygwin and the machine is doing like a million other things at the same time though, would be interesting to find out what the bottleneck on win32 is.

$ ./redis-benchmark.exe  -q
SET: 4399.56 requests per second
GET: 4328.58 requests per second
INCR: 4532.16 requests per second
LPUSH: 4709.39 requests per second
LPOP: 4478.53 requests per second
PING: 4153.65 requests per second
LPUSH (again, in order to bench LRANGE): 4458.63 requests per second
LRANGE (first 100 elements): 553.92 requests per second
...

CPU : I7 920 / RAM : 12 Gb / HDD : 2 Tb / OS : Ubuntu 9.04

./redis-benchmark -q
SET: 141028.17 requests per second
GET: 138888.89 requests per second
INCR: 119059.52 requests per second
LPUSH: 153907.70 requests per second
LPOP: 136986.30 requests per second
PING: 151772.73 requests per second
LPUSH (again, in order to bench LRANGE): 156421.88 requests per second
LRANGE (first 100 elements): 14367.82 requests per second
LRANGE (first 300 elements): 3762.23 requests per second
LRANGE (first 450 elements): 2413.13 requests per second
LRANGE (first 600 elements): 1765.54 requests per second

test on sheevaplug computer (http://www.globalscaletechnologies.com/t-sheevaplugdetails.aspx#component):

SET: 5128.21 requests per second
GET: 4897.16 requests per second
INCR: 4555.81 requests per second
LPUSH: 5336.18 requests per second
LPOP: 4856.73 requests per second
PING: 5701.25 requests per second
LPUSH (again, in order to bench LRANGE): 5333.33 requests per second
LRANGE (first 100 elements): 580.73 requests per second
LRANGE (first 300 elements): 204.17 requests per second
LRANGE (first 450 elements): 132.83 requests per second
LRANGE (first 600 elements): 98.45 requests per second

CPU: I7 920 / RAM : 8 Gb DDR3 / OS : Ubuntu 9.04 64Bit

./redis-benchmark -n 100000 -q
SET: 144517.34 requests per second
GET: 132100.39 requests per second
INCR: 111982.08 requests per second
LPUSH: 144534.69 requests per second
LPOP: 140056.03 requests per second
PING: 139100.14 requests per second
LPUSH (again, in order to bench LRANGE): 146854.62 requests per second
LRANGE (first 100 elements): 15048.91 requests per second
LRANGE (first 300 elements): 4257.86 requests per second
LRANGE (first 450 elements): 2804.34 requests per second
LRANGE (first 600 elements): 2114.61 requests per second

PS: http://hetzner.com @ 49€ / month, since some days also as 89€ for 24GB

@sminty83: beware! hetzner does not have ECC RAM!

On a Linode720:

$ ./redis-benchmark -q
SET: 48800.00 requests per second
GET: 44453.33 requests per second
INCR: 42739.32 requests per second
LPUSH: 44092.51 requests per second
LPOP: 44488.89 requests per second
PING: 43107.76 requests per second
LPUSH (again, in order to bench LRANGE): 41670.83 requests per second
LRANGE (first 100 elements): 5924.17 requests per second
LRANGE (first 300 elements): 1936.86 requests per second
LRANGE (first 450 elements): 1245.95 requests per second
LRANGE (first 600 elements): 738.28 requests per second

Freshly compiled redis from git Fedora 12, kernel 2.6.32.9-70.fc12.x86_64 AMD Athlon(tm) II X4 620 Processor, 2600 MHz

$ ./redis-benchmark -q -n 100000
PING: 72732.37 requests per second
PING (multi bulk): 72098.05 requests per second
SET: 70274.07 requests per second
GET: 71942.45 requests per second
INCR: 66181.34 requests per second
LPUSH: 70972.32 requests per second
LPOP: 71994.96 requests per second
SADD: 76863.95 requests per second
SPOP: 77590.38 requests per second
LPUSH (again, in order to bench LRANGE): 71581.96 requests per second
LRANGE (first 100 elements): 8046.35 requests per second
LRANGE (first 300 elements): 2155.50 requests per second
LRANGE (first 450 elements): 1263.79 requests per second
LRANGE (first 600 elements): 921.06 requests per second

It looks significantly slower than the Core i7 920 machines, which is no surprise, but the margin is more than I would have guessed.

Running on a Ubuntu 8.10 inside a VMWare Fusion

1 virtual processor on a Core2Duo? 2.4 GHz Macbook Pro

512 MB of RAM available to Ubuntu:

SET: 5316.15 requests per second 
GET:4832.13 requests per second
INCR: 5262.74 requests per second
LPUSH: 6432.05 requests per second
LPOP: 5444.08 requests per second
PING: 5383.21 requests per second
LPUSH (again, in order to bench LRANGE): 6360.18 requests per second
LRANGE (first 100 elements): 602.41 requests per second
LRANGE (first 300 elements): 273.10 requests per second
LRANGE (first 450 elements): 173.37 requests per second
LRANGE (first 600 elements): 118.54 requests per second

It's definitely slow compared to other benchmarks, but it's running inside a VM, and I'm more than happy with the result :)

Hello dmitriid,

thanks for the feedback, btw that's strange, with my macobook pro 13" under VMWare fusion I get 50k operations second...

Cheers, Salvatore

New to Redis & liking it; kudos & thanks!

Just started profiling the redis-server using profiling tools (intel) on my old linux laptop (core2-duo, x60s); Thought I'd post the results of a few hours worth of testing the load.. since this is a thread with most of the data.

I used the following commands -

./redis-server ./redis-benchmark -r 100000 (both running on same laptop)

+ results in ~20k tps range for the (set..lpush) tests, 600 for LRange etc. + collected samples for 360s (with repeated runs of redis-benchmark)

Here are some (early) findings for this microbenchmark from redis-server side -

+ libc: int_malloc & int_free dominates the test (65% of 180k tick samples)

Further since we are interested in looking into, redis-server (~30% of ticks, balance 5% in kernel)

+ glueReplyBuffersIfNeeded (with a CPI of ~2) consumes 14% of clockticks with 7% retired (~2 CPI) + sdslen: It didnot make sense to see sdslen in second @ 12% clockticks

(with a ~2.8 CPI to boot) - Tool err? I'll try oprofile one of these days.

+ zmalloc, zfree dominate callgraphs & instructions retired as expected. (Further microbenchmarking, ie, zmalloc vs malloc - the increment_used_memory (with thread safety) seems to have added 5-10% cost. The function call accounts for 4% of the IR.) Not compared with tcmalloc, etc.

Looking to experiment with larger 'real' datasets, so bear with current microbenchmark effort for now. They are bound to have a few anamolies. Overall, good first impressions. thanks for making this easy to build, test, integrate & run sample benchmarks exercising all the imp workloads. Let me know if you need further experimentation/profiling on a particular workload/dataset of interest. ciao.

One point is worth mentioning: on multi-CPU boxes, redis benchmark is sensitive to process placement. This may lead to inconsistent results if process placement is not enforced. When the redis benchmark is launched, you may have different results depending on which CPU core the OS schedules client and server processes on.

Here is an example using two boxes: the first one is a 24-cores AMD Istanbul machine (4*6 cores, no hyperthreading), the second one is a 32-cores Intel Nehalem EX machine (4*8 cores, plus hyperthreading).

Here are the results when process placement is enforced using the taskset command (OS is Linux SLES10):

AMD Opteron Istanbul - same CPU, same core

SET: 25740.61 requests per second
GET: 23894.86 requests per second
INCR: 38595.14 requests per second
LPUSH: 42408.82 requests per second
LPOP: 40209.09 requests per second
PING: 44345.89 requests per second
LPUSH (again, in order to bench LRANGE): 42229.73 requests per second
LRANGE (first 100 elements): 8070.37 requests per second
LRANGE (first 300 elements): 2939.62 requests per second
LRANGE (first 450 elements): 1973.98 requests per second
LRANGE (first 600 elements): 1496.36 requests per second

AMD Opteron Istanbul - same CPU, different core

SET: 53706.23 requests per second
GET: 41000.41 requests per second
INCR: 77954.80 requests per second
LPUSH: 76525.62 requests per second
LPOP: 76757.48 requests per second
PING: 76583.46 requests per second
LPUSH (again, in order to bench LRANGE): 76584.99 requests per second
LRANGE (first 100 elements): 10775.86 requests per second
LRANGE (first 300 elements): 3296.85 requests per second
LRANGE (first 450 elements): 1994.77 requests per second
LRANGE (first 600 elements): 1223.15 requests per second

AMD Opteron Istanbul - different CPUs

SET: 45290.31 requests per second
GET: 36245.02 requests per second
INCR: 67165.88 requests per second
LPUSH: 66806.95 requests per second
LPOP: 66458.47 requests per second
PING: 66319.62 requests per second
LPUSH (again, in order to bench LRANGE): 66503.99 requests per second
LRANGE (first 100 elements): 10002.00 requests per second
LRANGE (first 300 elements): 2945.94 requests per second
LRANGE (first 450 elements): 1847.44 requests per second
LRANGE (first 600 elements): 1156.39 requests per second

Intel Nehalem EX - same CPU, same core, same thread

SET: 23164.23 requests per second
GET: 22436.62 requests per second
INCR: 33288.95 requests per second
LPUSH: 37921.88 requests per second
LPOP: 36153.29 requests per second
PING: 40128.41 requests per second
LPUSH (again, in order to bench LRANGE): 34566.20 requests per second
LRANGE (first 100 elements): 5247.42 requests per second
LRANGE (first 300 elements): 1923.63 requests per second
LRANGE (first 450 elements): 1349.07 requests per second
LRANGE (first 600 elements): 1060.91 requests per second

Intel Nehalem EX - same CPU, same core, different thread

SET: 37106.87 requests per second
GET: 34317.09 requests per second
INCR: 55991.04 requests per second
LPUSH: 65322.66 requests per second
LPOP: 64600.78 requests per second
PING: 64645.77 requests per second
LPUSH (again, in order to bench LRANGE): 65105.47 requests per second
LRANGE (first 100 elements): 6800.41 requests per second
LRANGE (first 300 elements): 2456.88 requests per second
LRANGE (first 450 elements): 1609.40 requests per second
LRANGE (first 600 elements): 1181.42 requests per second

Intel Nehalem EX - same CPU, different core

SET: 52631.58 requests per second
GET: 52250.26 requests per second
INCR: 88888.89 requests per second
LPUSH: 91075.59 requests per second
LPOP: 89384.27 requests per second
PING: 90337.85 requests per second
LPUSH (again, in order to bench LRANGE): 89215.88 requests per second
LRANGE (first 100 elements): 8714.01 requests per second
LRANGE (first 300 elements): 2932.29 requests per second
LRANGE (first 450 elements): 1920.68 requests per second
LRANGE (first 600 elements): 1432.64 requests per second

Intel Nehalem EX - different CPU

SET: 41288.19 requests per second
GET: 38988.30 requests per second
INCR: 63412.18 requests per second
LPUSH: 64278.28 requests per second
LPOP: 62276.46 requests per second
PING: 62907.55 requests per second
LPUSH (again, in order to bench LRANGE): 63253.64 requests per second
LRANGE (first 100 elements): 8431.79 requests per second
LRANGE (first 300 elements): 2834.31 requests per second
LRANGE (first 450 elements): 1866.12 requests per second
LRANGE (first 600 elements): 1401.62 requests per second

As expected, the best combination is "same CPU, but different core", since it benefits from the L3 cache available on recent CPUs ...

Regards, Didier.

Intel i7 930 o/c at 3.35GHz w/12GB ram:

$ redis-benchmark -q -n 100000
SET: 158730.16 requests per second
GET: 147711.97 requests per second
INCR: 134770.89 requests per second
LPUSH: 158480.19 requests per second
LPOP: 157480.31 requests per second
PING: 168360.28 requests per second
LPUSH (again, in order to bench LRANGE): 171548.89 requests per second
LRANGE (first 100 elements): 19425.21 requests per second
LRANGE (first 300 elements): 5324.25 requests per second
LRANGE (first 450 elements): 3511.36 requests per second
LRANGE (first 600 elements): 2544.92 requests per second

antirez: in your VM setup, are you using a multi-core VM or a single-core VM. I'm using VirtualBox? with 1 virtual core on my 2.54 ghz Core 2 duo, and I get similar results as dmitriid.

Intel® Core™ i7-920 Quad-Core / 8 GB DDR3 RAM FreeBSD 8.0 amd64, ZFS, Redis 1.2.4

#redis-benchmark -q -n 100000
SET: 101840.12 requests per second
GET: 101629.06 requests per second
INCR: 88653.37 requests per second
LPUSH: 108353.20 requests per second
LPOP: 87275.74 requests per second
PING: 106056.20 requests per second
LPUSH (again, in order to bench LRANGE): 101113.24 requests per second
LRANGE (first 100 elements): 9351.91 requests per second
LRANGE (first 300 elements): 3423.13 requests per second
LRANGE (first 450 elements): 2253.42 requests per second
LRANGE (first 600 elements): 1612.51 requests per second

memcached lose...

On a 4-core, 8GB memory unit setup by RailsMachine? (VM). A bit surprised at the specs, asking them for more details on their VM setup right now.

 ./redis-benchmark -q -n 100000

PING: 26476.04 requests per second
PING (multi bulk): 25786.49 requests per second
SET: 25297.24 requests per second
GET: 25342.12 requests per second
INCR: 24801.59 requests per second
LPUSH: 25322.87 requests per second
LPOP: 25113.01 requests per second
SADD: 25458.25 requests per second
SPOP: 26232.95 requests per second
LPUSH (again, in order to bench LRANGE): 29256.88 requests per second
LRANGE (first 100 elements): 16545.50 requests per second
LRANGE (first 300 elements): 4614.67 requests per second
LRANGE (first 450 elements): 2180.64 requests per second
LRANGE (first 600 elements): 1361.80 requests per second

On:

• Processor Name: Intel Core 2 Duo
• Processor Speed: 2.26 GHz
• Number Of Processors: 1
• Total Number Of Cores: 2
• L2 Cache: 3 MB
• Memory: 4 GB

I am getting:

• SET: 22542.83 requests per second
• GET: 20080.69 requests per second
• INCR: 19770.36 requests per second
• LPUSH: 22481.91 requests per second
• LPOP: 19950.33 requests per second
• PING: 24350.13 requests per second
• LPUSH (again, in order to bench
• LRANGE): 21333.19 requests per second
• LRANGE (first 100 elements): 4234.42 requests per second
• LRANGE (first 300 elements): 1579.93 requests per second
• LRANGE (first 450 elements): 1023.45 requests per second
• LRANGE (first 600 elements): 775.34 requests per second

Is this OK! or do I need to change some settings?

Gogrid VM : 4Gb RAM : 4 CPU : Centos 5.5 32bit : redis-1.2.6

root@ggw-devdb01 /usr/src/redhat/SOURCES/redis-1.2.6 14:29:28 # ./redis-benchmark -q -n 100000
SET: 63099.05 requests per second
GET: 59846.20 requests per second
INCR: 53167.46 requests per second
LPUSH: 63497.78 requests per second
LPOP: 60168.47 requests per second
PING: 61741.98 requests per second
LPUSH (again, in order to bench LRANGE): 61462.82 requests per second
LRANGE (first 100 elements): 5953.09 requests per second
LRANGE (first 300 elements): 1923.93 requests per second
LRANGE (first 450 elements): 1235.22 requests per second
LRANGE (first 600 elements): 891.97 requests per second

HW + OS

• 8 x Quad-Core AMD Opteron(tm) Processor 8374 HE
• 32GB RAM
• Gentoo Base System release 1.12.11
• SEAGATE ST3146707LC, SW RAID 10

BENCHMARK

./redis-benchmark -q -n 100000
PING: 52303.87 requests per second
PING (multi bulk): 52220.37 requests per second
SET: 51361.58 requests per second
GET: 51151.41 requests per second
INCR: 49310.16 requests per second
LPUSH: 50994.90 requests per second
LPOP: 50505.05 requests per second
SADD: 50530.57 requests per second
SPOP: 52854.65 requests per second
LPUSH (again, in order to bench LRANGE): 50942.94 requests per second
LRANGE (first 100 elements): 10792.14 requests per second

Why i have so bad results?

HW + OS

• Intel(R) Pentium(R) 4 CPU 3.20GHz (3192.01-MHz 686-class CPU) (two CPU)
• real memory = 4296015872 (4097 MB)
• ad4: 238475MB <Seagate ST3250824AS 3.AAH> at ata2-master UDMA100 SATA
• FreeBSD mercurio.madrid.loeda.net 8.0-STABLE FreeBSD 8.0-STABLE #1:

BENCHMARK

mercurio# redis-benchmark -q -n 100000
SET: 21712.55 requests per second
GET: 20726.01 requests per second
INCR: 19814.74 requests per second
LPUSH: 22594.76 requests per second
LPOP: 20924.88 requests per second
PING: 23436.27 requests per second
LPUSH (again, in order to bench LRANGE): 22232.33 requests per second
LRANGE (first 100 elements): 3981.61 requests per second
LRANGE (first 300 elements): 1457.57 requests per second
LRANGE (first 450 elements): 980.66 requests per second
LRANGE (first 600 elements): 730.92 requests per second

I wonder if instead of standard hashed REDIS would use Judy array (Speed comparison: http://judy.sourceforge.net/examples/Judy_hashing.pdf Project: http://judy.sourceforge.net/) or Trie (http://en.wikipedia.org/wiki/Trie) would make it even faster. There are libraries in C so it should be easier to write a wrapper around them and try how they work.

I am particularly intrigued by trie structure that would make very efficient retrieving "something:" keys where is whatsoever string.

What is speed of HGET/HSET? Should I consider it the same as of GET/SET?

Linode 2048 running Debian 5.0 32-bit (kernal 2.6.18.8)

> ./redis-benchmark -q           
PING: 45871.56 requests per second
PING (multi bulk): 40889.80 requests per second
SET: 45908.26 requests per second
GET: 37747.17 requests per second
INCR: 42557.45 requests per second
LPUSH: 42918.45 requests per second
LPOP: 42440.68 requests per second
SADD: 41242.80 requests per second
SPOP: 43672.49 requests per second
LPUSH (again, in order to bench LRANGE): 41218.11 requests per second
LRANGE (first 100 elements): 6422.61 requests per second
LRANGE (first 300 elements): 2100.40 requests per second
LRANGE (first 450 elements): 1153.40 requests per second
LRANGE (first 600 elements): 736.70 requests per second

2x Westmere 4-core, SLES11 Server running on Core0 Client on Core1-Core7

1
PING: 210568.42 requests per second
PING (multi bulk): 206613.64 requests per second
SET: 184842.88 requests per second
GET: 181818.17 requests per second
INCR: 155763.23 requests per second
LPUSH: 167224.08 requests per second
LPOP: 173913.05 requests per second
SADD: 183150.19 requests per second
SPOP: 210978.91 requests per second
LPUSH (again, in order to bench LRANGE): 152207.00 requests per second
LRANGE (first 100 elements): 24260.07 requests per second
LRANGE (first 300 elements): 6798.10 requests per second
LRANGE (first 450 elements): 4333.13 requests per second
LRANGE (first 600 elements): 3246.75 requests per second

2
PING: 205379.88 requests per second
PING (multi bulk): 201209.25 requests per second
SET: 196078.44 requests per second
GET: 187269.67 requests per second
INCR: 166666.66 requests per second
LPUSH: 175131.36 requests per second
LPOP: 178571.42 requests per second
SADD: 175746.92 requests per second
SPOP: 207933.47 requests per second
LPUSH (again, in order to bench LRANGE): 197240.62 requests per second
LRANGE (first 100 elements): 24857.07 requests per second
LRANGE (first 300 elements): 6860.12 requests per second
LRANGE (first 450 elements): 4402.18 requests per second
LRANGE (first 600 elements): 3278.69 requests per second

3
PING: 208807.94 requests per second
PING (multi bulk): 205342.92 requests per second
SET: 179857.92 requests per second
GET: 179533.22 requests per second
INCR: 156494.52 requests per second
LPUSH: 169491.53 requests per second
LPOP: 175438.59 requests per second
SADD: 179211.45 requests per second
SPOP: 211448.20 requests per second
LPUSH (again, in order to bench LRANGE): 195694.72 requests per second
LRANGE (first 100 elements): 22326.41 requests per second
LRANGE (first 300 elements): 6864.83 requests per second
LRANGE (first 450 elements): 4321.71 requests per second
LRANGE (first 600 elements): 3263.81 requests per second

4
PING: 136070.75 requests per second
PING (multi bulk): 132450.33 requests per second
SET: 130549.61 requests per second
GET: 128042.25 requests per second
INCR: 116550.12 requests per second
LPUSH: 123304.56 requests per second
LPOP: 124689.53 requests per second
SADD: 120336.95 requests per second
SPOP: 137746.55 requests per second
LPUSH (again, in order to bench LRANGE): 128369.71 requests per second
LRANGE (first 100 elements): 23375.41 requests per second
LRANGE (first 300 elements): 6565.99 requests per second
LRANGE (first 450 elements): 4272.41 requests per second
LRANGE (first 600 elements): 3194.58 requests per second

5
PING: 139691.34 requests per second
PING (multi bulk): 134772.23 requests per second
SET: 105374.08 requests per second
GET: 125944.58 requests per second
INCR: 114942.53 requests per second
LPUSH: 124534.25 requests per second
LPOP: 125629.40 requests per second
SADD: 118343.19 requests per second
SPOP: 135869.56 requests per second
LPUSH (again, in order to bench LRANGE): 120627.27 requests per second
LRANGE (first 100 elements): 22815.42 requests per second
LRANGE (first 300 elements): 6680.47 requests per second
LRANGE (first 450 elements): 4237.11 requests per second
LRANGE (first 600 elements): 3160.36 requests per second

6
PING: 138138.12 requests per second
PING (multi bulk): 132100.39 requests per second
SET: 125157.70 requests per second
GET: 125471.77 requests per second
INCR: 116009.28 requests per second
LPUSH: 119760.48 requests per second
LPOP: 126103.41 requests per second
SADD: 124853.93 requests per second
SPOP: 139666.20 requests per second
LPUSH (again, in order to bench LRANGE): 127389.80 requests per second
LRANGE (first 100 elements): 26681.16 requests per second
LRANGE (first 300 elements): 6768.19 requests per second
LRANGE (first 450 elements): 4349.34 requests per second
LRANGE (first 600 elements): 3322.59 requests per second

7
PING: 137555.70 requests per second
PING (multi bulk): 131578.95 requests per second
SET: 123001.23 requests per second
GET: 125313.29 requests per second
INCR: 115340.26 requests per second
LPUSH: 125470.52 requests per second
LPOP: 123762.38 requests per second
SADD: 124223.60 requests per second
SPOP: 137954.48 requests per second
LPUSH (again, in order to bench LRANGE): 128534.70 requests per second
LRANGE (first 100 elements): 24084.78 requests per second
LRANGE (first 300 elements): 6450.36 requests per second
LRANGE (first 450 elements): 4240.52 requests per second
LRANGE (first 600 elements): 3187.35 requests per second

Results are incredibly promising... I'll try to do some "real world" stress test with billions of records over the upcoming week and I will share the results. Right now I'm pretty impressed...

• Processor Name: Intel(TM) Core 2 Duo
• Processor Speed: 2.53 GHz
• Number Of Processors: 2
• L2 Cache: 3 MB
• Memory: 2 GB
• OS: Ubuntu 10.04 Lucid 32bit

• SET: 65756.74 requests per second
• GET: 59605.48 requests per second
• INCR: 65402.22 requests per second
• LPUSH: 53522.20 requests per second
• LPOP: 52323.22 requests per second
• PING: 73166.79 requests per second
• LPUSH (again, in order to bench LRANGE): 60720.10 requests per second
• LRANGE (first 100 elements): 6494.77 requests per second
• LRANGE (first 300 elements): 2137.76 requests per second
• LRANGE (first 450 elements): 1399.44 requests per second
• LRANGE (first 600 elements): 970.22 requests per second

CPU: AMD Opteron x8-6128
Speed:	2000MHz
RAM:	16.0GB
CPUs:	2 Physical CPUs
Cores:	16 Total Cores
RAID:	RAID 10

ubuntu@experimental:~$ redis-benchmark -q -n 100000 
PING: 42610.57 requests per second
PING (multi bulk): 42519.13 requests per second
SET: 43384.81 requests per second
GET: 42035.31 requests per second
INCR: 42482.16 requests per second
LPUSH: 41912.83 requests per second
LPOP: 42216.97 requests per second
SADD: 42593.27 requests per second
SPOP: 42559.15 requests per second
LPUSH (again, in order to bench LRANGE): 43444.83 requests per second
LRANGE (first 100 elements): 11977.60 requests per second
LRANGE (first 300 elements): 2636.92 requests per second
LRANGE (first 450 elements): 1446.68 requests per second
LRANGE (first 600 elements): 947.60 requests per second

On an Acer Aspire ONE A150L netbook.

CPU:    Intel(R) Atom(TM) CPU N270
Speed:  1.60GHz
Cores:  1
RAM:    1GiB
Distro: Arch Linux
Kernel: 2.6.35-ARCH
Redis:  2.0.0

% redis-benchmark -q -n 100000
PING: 17080.09 requests per second
PING (multi bulk): 16147.26 requests per second
SET: 16781.34 requests per second
GET: 16556.20 requests per second
INCR: 15797.95 requests per second
LPUSH: 15497.13 requests per second
LPOP: 15880.58 requests per second
SADD: 16401.67 requests per second
SPOP: 16705.36 requests per second
LPUSH (again, in order to bench LRANGE): 15862.94 requests per second
LRANGE (first 100 elements): 2002.68 requests per second
LRANGE (first 300 elements): 657.06 requests per second
LRANGE (first 450 elements): 428.64 requests per second
LRANGE (first 600 elements): 317.87 requests per second

Linode 512 - Redis 2.0.2

CPU

4 cores:

vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           L5520  @ 2.27GHz
stepping        : 5
cpu MHz         : 2266.746
cache size      : 8192 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu de tsc msr pae cx8 sep cmov pat clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nonstop_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt hypervisor lahf_lm
bogomips        : 4533.49
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual

PING: 29203.21 requests per second
PING (multi bulk): 27645.94 requests per second
SET: 27934.92 requests per second
GET: 26717.07 requests per second
INCR: 27991.60 requests per second
LPUSH: 30566.32 requests per second
LPOP: 27542.00 requests per second
SADD: 28360.75 requests per second
SPOP: 29886.73 requests per second
LPUSH (again, in order to bench LRANGE): 28208.74 requests per second
LRANGE (first 100 elements): 9971.08 requests per second
LRANGE (first 300 elements): 3131.65 requests per second
LRANGE (first 450 elements): 1976.13 requests per second
LRANGE (first 600 elements): 1511.21 requests per second

Here is benchmark on an iMac:

CPU: Intel Core 2 Duo, 2.66GHz 
RAM: 2Go 1067Mhz DDR3 
Distro: Mac Os X 10.6.4 (Snow Leopard) 
Kernel: Darwin 10.4.0 
Redis: 2.0.2

$ ./redis-benchmark -q -n 100000
PING: 37174.72 requests per second
PING (multi bulk): 35206.62 requests per second
SET: 35076.81 requests per second
GET: 35280.42 requests per second
INCR: 34429.95 requests per second
LPUSH: 34381.57 requests per second
LPOP: 34925.63 requests per second
SADD: 33724.21 requests per second
SPOP: 37009.99 requests per second
LPUSH (again, in order to bench LRANGE): 32460.56 requests per second
LRANGE (first 100 elements): 8010.25 requests per second
LRANGE (first 300 elements): 2824.06 requests per second
LRANGE (first 450 elements): 1831.70 requests per second
LRANGE (first 600 elements): 1313.35 requests per second

CPU: Intel® Core™ i7-980X Hexa-Core
Speed:  3,33 GHz
RAM:    24GB DDR 3
CPUs:   1 Physical CPUs
Cores:  6 Total Cores / 12 Threads
RAID:   RAID 0
DISKS: 4 SSD 128GB with RAID 0

./redis-benchmark -q -n 100000
PING: 166673.33 requests per second
PING (multi bulk): 172125.66 requests per second
SET: 160256.41 requests per second
GET: 167507.53 requests per second
INCR: 156987.44 requests per second
LPUSH: 162866.44 requests per second
LPOP: 160022.41 requests per second
SADD: 164216.73 requests per second
SPOP: 169209.81 requests per second
LPUSH (again, in order to bench LRANGE): 160256.41 requests per second
LRANGE (first 100 elements): 24857.07 requests per second
LRANGE (first 300 elements): 6978.37 requests per second
LRANGE (first 450 elements): 4513.45 requests per second
LRANGE (first 600 elements): 3351.09 requests per secon

Is redis slow with a little old linux kernel?

CPU: Intel Xeon X5460 3.16GHz 
RAM: 32G 
Distro: CentOS 5.5 
Kernel: 2.6.18 
Redis: 2.0.3

# ./redis-benchmark -r 10000000 -q -n 100000
PING: 51526.02 requests per second
PING (multi bulk): 50715.01 requests per second
SET: 50254.77 requests per second
GET: 49659.39 requests per second
INCR: 50925.15 requests per second
LPUSH: 51576.59 requests per second
LPOP: 50311.87 requests per second
SADD: 50086.13 requests per second
SPOP: 50175.61 requests per second
LPUSH (again, in order to bench LRANGE): 51234.63 requests per second
LRANGE (first 100 elements): 16594.92 requests per second
LRANGE (first 300 elements): 5763.02 requests per second
LRANGE (first 450 elements): 3428.53 requests per second
LRANGE (first 600 elements): 2103.67 requests per second

much less than I expected.

Finally get the reason of slow, CPU affinity and Intel Xeon X54xx cpu platform.

test from other host, 9xxxx req/s

taskset -c 7 ./redis-server redis.conf

# taskset -c 3 /usr/local/src/redis-2.0.3/redis-benchmark -q -n 100000 
PING: 91006.38 requests per second 
PING (multi bulk): 92095.77 requests per second 
SET: 93471.02 requests per second 
GET: 92005.52 requests per second 
INCR: 94260.13 requests per second

# taskset -c 4 /usr/local/src/redis-2.0.3/redis-benchmark -q -n 100000 
PING: 52413.00 requests per second 
PING (multi bulk): 53450.56 requests per second 
SET: 54497.00 requests per second 
GET: 41513.91 requests per second 
INCR: 54978.56 requests per second

It seems cpu 3 and cpu 7 share the same L2 Cache.

os x 10.6.4 2.26ghz core2 duo

$ ./redis-benchmark -q -n 100000
PING: 33658.70 requests per second
PING (multi bulk): 33422.46 requests per second
MSET (10 keys, multi bulk): 27307.48 requests per second
SET: 31240.24 requests per second
GET: 32020.49 requests per second
INCR: 30940.59 requests per second
LPUSH: 31269.54 requests per second
LPOP: 31426.78 requests per second
SADD: 30873.73 requests per second
SPOP: 32351.99 requests per second
LPUSH (again, in order to bench LRANGE): 30220.61 requests per second
LRANGE (first 100 elements): 17924.36 requests per second
LRANGE (first 300 elements): 10598.83 requests per second
LRANGE (first 450 elements): 8320.16 requests per second
LRANGE (first 600 elements): 6665.78 requests per second

Servergrove VPS200

redis-benchmark -q -n 100000
PING: 56561.09 requests per second
PING (multi bulk): 57175.53 requests per second
MSET (10 keys, multi bulk): 64516.13 requests per second
SET: 59916.12 requests per second
GET: 60716.46 requests per second
INCR: 67476.38 requests per second
LPUSH: 63011.97 requests per second
LPOP: 58582.31 requests per second
SADD: 63131.31 requests per second
SPOP: 57870.37 requests per second
LPUSH (again, in order to bench LRANGE): 59844.41 requests per second
LRANGE (first 100 elements): 43365.13 requests per second
LRANGE (first 300 elements): 24437.93 requests per second
LRANGE (first 450 elements): 17863.52 requests per second
LRANGE (first 600 elements): 14448.78 requests per second

Amazon EC2

CentOS 5.4 64 bit (ami-0f42a966 )

redis-benchmark -q -n 100000

Amazon High-Memory Extra Large Instance

17.1 GB memory,

6.5 ECU (2 virtual cores with 3.25 EC2 Compute Units each),

420 GB of local instance storage,

64-bit platform

localhost client

PING: 65713.54 requests per second
PING (multi bulk): 67207.66 requests per second
SET: 67340.07 requests per second
GET: 66099.80 requests per second
INCR: 67083.16 requests per second
LPUSH: 66814.96 requests per second
LPOP: 66053.50 requests per second
SADD: 66674.66 requests per second
SPOP: 65834.76 requests per second
LPUSH (again, in order to bench LRANGE): 66328.25 requests per second
LRANGE (first 100 elements): 15410.70 requests per second
LRANGE (first 300 elements): 5478.91 requests per second
LRANGE (first 450 elements): 3611.67 requests per second
LRANGE (first 600 elements): 1914.21 requests per second

remote client

PING: 52615.51 requests per second
PING (multi bulk): 53109.40 requests per second
SET: 53481.29 requests per second
GET: 53769.89 requests per second
INCR: 54644.81 requests per second
LPUSH: 54235.90 requests per second
LPOP: 52806.23 requests per second
SADD: 54682.34 requests per second
SPOP: 54262.07 requests per second
LPUSH (again, in order to bench LRANGE): 52142.34 requests per second
LRANGE (first 100 elements): 12941.63 requests per second
LRANGE (first 300 elements): 4766.67 requests per second
LRANGE (first 450 elements): 3276.65 requests per second
LRANGE (first 600 elements): 2334.38 requests per second

Amazon Large Instance 7.5 GB of memory,

4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each),

850 GB of local instance storage,

64-bit platform

localhost client

PING: 40932.05 requests per second
PING (multi bulk): 41618.39 requests per second
SET: 43019.36 requests per second
GET: 32421.39 requests per second
INCR: 32327.73 requests per second
LPUSH: 32134.96 requests per second
LPOP: 31265.71 requests per second
SADD: 29451.12 requests per second
SPOP: 33151.48 requests per second
LPUSH (again, in order to bench LRANGE): 31095.46 requests per second
LRANGE (first 100 elements): 10233.32 requests per second
LRANGE (first 300 elements): 3343.14 requests per second
LRANGE (first 450 elements): 2086.72 requests per second
LRANGE (first 600 elements): 1332.68 requests per second

remote client

PING: 50819.61 requests per second
PING (multi bulk): 48546.12 requests per second
SET: 46234.39 requests per second
GET: 49098.68 requests per second
INCR: 50711.46 requests per second
LPUSH: 49656.90 requests per second
LPOP: 50583.21 requests per second
SADD: 48948.61 requests per second
SPOP: 50687.28 requests per second
LPUSH (again, in order to bench LRANGE): 50200.80 requests per second
LRANGE (first 100 elements): 12439.36 requests per second
LRANGE (first 300 elements): 3950.54 requests per second
LRANGE (first 450 elements): 2335.14 requests per second
LRANGE (first 600 elements): 1382.88 requests per second

CPU: Intel Xeon X5550, 2.67GHz
RAM: 48 GB
Redis: 2.0.1

PING: 64102.57 requests per second
PING (multi bulk): 69687.11 requests per second
SET: 77043.15 requests per second
GET: 73855.24 requests per second
INCR: 70677.74 requests per second
LPUSH: 60606.06 requests per second
LPOP: 71891.45 requests per second
SADD: 74571.22 requests per second
SPOP: 60753.95 requests per second
LPUSH (again, in order to bench LRANGE): 58287.88 requests per second
LRANGE (first 100 elements): 15174.51 requests per second
LRANGE (first 300 elements): 5836.35 requests per second
LRANGE (first 450 elements): 3869.82 requests per second
LRANGE (first 600 elements): 2895.45 requests per second

Ubuntu 10.10 64bits
Intel(R) Xeon(R) CPU           X3440  @ 2.53GHz
16GB memory


PING (inline): 126582.27 requests per second
PING: 125000.00 requests per second
MSET (10 keys): 19531.25 requests per second
SET: 70921.98 requests per second
GET: 78125.00 requests per second
INCR: 70422.53 requests per second
LPUSH: 97087.38 requests per second
LPOP: 96153.84 requests per second
SADD: 68493.15 requests per second
SPOP: 99009.90 requests per second
LPUSH (again, in order to bench LRANGE): 90090.09 requests per second
LRANGE (first 100 elements): 53763.44 requests per second
LRANGE (first 300 elements): 28985.51 requests per second
LRANGE (first 450 elements): 21929.82 requests per second
LRANGE (first 600 elements): 17543.86 requests per second

64gb ram, 2x4 core Intel Xeon - loads @ 0.00

MSET (10 keys)

1. 0000 requests completed in 1.61 seconds

50 parallel clients 3 bytes payload keep alive: 1

91.31% <= 1 milliseconds 100.00% <= 1 milliseconds 62034.74 requests per second

SET

1. 0000 requests completed in 1.12 seconds

50 parallel clients 3 bytes payload keep alive: 1

100.00% <= 0 milliseconds 89365.51 requests per second

GET

1. 0000 requests completed in 0.91 seconds

50 parallel clients 3 bytes payload keep alive: 1

100.00% <= 0 milliseconds 110011.00 requests per second

On Intel 2 Quad !8200 with 4Gb of RAM

root@servidor:/home/michel# redis-benchmark -q PING (inline): 70422.53 requests per second PING: 74074.07 requests per second MSET (10 keys): 46296.30 requests per second SET: 75757.58 requests per second GET: 75187.97 requests per second INCR: 75757.58 requests per second LPUSH: 76923.08 requests per second LPOP: 76335.88 requests per second SADD: 75187.97 requests per second SPOP: 75187.97 requests per second LPUSH (again, in order to bench LRANGE): 77519.38 requests per second LRANGE (first 100 elements): 57142.86 requests per second LRANGE (first 300 elements): 29411.76 requests per second LRANGE (first 450 elements): 21141.65 requests per second LRANGE (first 600 elements): 17211.71 requests per second