|
README
IntroductionRedis is a database. To be specific, Redis is a very simple database implementing a dictionary, where every key is associated with a value. For example I can set the key "surname_1992" to the string "Smith". What makes Redis different from many other key-value stores, is that every single value has a type. The following types are supported:
The type of a value determines what operations (called commands) are available for the value itself. For example you can append elements to a list stored at the key "mylist" using the LPUSH or RPUSH command in O(1). Later you'll be able to get a range of elements with LRANGE or trim the list with LTRIM. Sets are very flexible too, it is possible to add and remove elements from Sets (unsorted collections of strings), and then ask for server-side intersection, union, difference of Sets. Each command is performed through server-side atomic operations. Please refer to the Command Reference to see the full list of operations associated to these data types. In other words, you can look at Redis as a data structures server. A Redis user is virtually provided with an interface to Abstract Data Types, saving her from the responsibility to implement concrete data structures and algorithms. Indeed both algorithms and data structures in Redis are properly choosed in order to obtain the best performance. Redis loads and mantains the whole dataset into memory, but the dataset is persistent, since from time to time Redis writes a dump on disk asynchronously. The dataset is loaded from the dump every time the server is (re)started. Redis can be configured to save the dataset when a certain number of changes is reached and after a given number of seconds elapses. For example, you can configure Redis to save after 1000 changes and at most 60 seconds since the last save. You can specify any combination for these numbers. Because data is written asynchronously, when a system crash occurs, the last few queries can get lost (that is acceptable in many applications). Anyway it is possible to make this a non issue, since Redis supports master-slave replication from its early days, being effective even in the case where a few records lost are not acceptable. Beyond key-value databasesAll these features allow to use Redis as the sole DB for your scalable application without the need of any relational database. We wrote a simple Twitter clone in PHP + Redis to show a real world example, the link points to an article explaining the design and internals in very simple words. What are the differences between Redis and Memcached?In the following ways:
What are the differences between Redis and Tokyo Cabinet / Tyrant?Redis and Tokyo Cabinet can be used for the same applications, but actually they are very different beasts. If you read twitter messages of people involved in scalable things both products are reported to work well, but surely there are times where one or the other can be the best choice. Some differences are the followings (I may be biased, make sure to check yourself both the products).
Does Redis support locking?No, the idea is to provide atomic primitives in order to make the programmer able to use redis with locking free algorithms. For example imagine you have 10 computers and one Redis server. You want to count words in a very large text. This large text is split among the 10 computers, every computer will process its part and use Redis's INCR command to atomically increment a counter for every occurrence of the word found. INCR/DECR are not the only atomic primitives, there are others like PUSH/POP on lists, POP RANDOM KEY operations, UPDATE and so on. For example you can use Redis like a Tuple Space (http://en.wikipedia.org/wiki/Tuple_space) in order to implement distributed algorithms. (News: locking with key-granularity is now planned) Multiple databases supportAnother synchronization primitive is the support for multiple DBs. By default DB 0 is selected for every new connection, but using the SELECT command it is possible to select a different database. The MOVE operation can move an item from one DB to another atomically. This can be used as a base for locking free algorithms together with the 'RANDOMKEY' commands. Redis Data TypesRedis supports the following three data types as values:
Values can be Strings, Lists or Sets. Keys can be a subset of strings not containing newlines ("\n") and spaces (" "). Note that sometimes strings may hold numeric vaules that must be parsed by Redis. An example is the INCR command that atomically increments the number stored at the specified key. In this case Redis is able to handle integers that can be stored inside a 'long long' type, that is a 64-bit signed integer. Implementation DetailsStrings are implemented as dynamically allocated strings of characters. Lists are implemented as doubly linked lists with cached length. Sets are implemented using hash tables that use chaining to resolve collisions. Redis Tutorial(note, you can skip this section if you are only interested in "formal" doc.) Later in this document you can find detailed information about Redis commands, the protocol specification, and so on. This kind of documentation is useful but... if you are new to Redis it is also BORING! The Redis protocol is designed so that is both pretty efficient to be parsed by computers, but simple enough to be used by humans just poking around with the 'telnet' command, so this section will show to the reader how to play a bit with Redis to get an initial feeling about it, and how it works. To start just compile redis with 'make' and start it with './redis-server'. The server will start and log stuff on the standard output, if you want it to log more edit redis.conf, set the loglevel to debug, and restart it. You can specify a configuration file as unique parameter: ./redis-server /etc/redis.conf This is NOT required. The server will start even without a configuration file using a default built-in configuration. Now let's try to set a key to a given value: $ telnet localhost 6379 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. SET foo 3 bar +OK The first line we sent to the server is "set foo 3". This means "set the key foo with the following three bytes I'll send you". The following line is the "bar" string, that is, the three bytes. So the effect is to set the key "foo" to the value "bar". Very simple! (note that you can send commands in lowercase and it will work anyway, commands are not case sensitive) Note that after the first and the second line we sent to the server there is a newline at the end. The server expects commands terminated by "\r\n" and sequence of bytes terminated by "\r\n". This is a minimal overhead from the point of view of both the server and client but allows us to play with Redis with the telnet command easily. The last line of the chat between server and client is "+OK". This means our key was added without problems. Actually SET can never fail but the "+OK" sent lets us know that the server received everything and the command was actually executed. Let's try to get the key content now: GET foo $3 bar Ok that's very similar to 'set', just the other way around. We sent "get foo", the server replied with a first line that is just the $ character follwed by the number of bytes the value stored at key contained, followed by the actual bytes. Again "\r\n" are appended both to the bytes count and the actual data. In Redis slang this is called a bulk reply. What about requesting a non existing key? GET blabla $-1 When the key does not exist instead of the length, just the "$-1" string is sent. Since a -1 length of a bulk reply has no meaning it is used in order to specifiy a 'nil' value and distinguish it from a zero length value. Another way to check if a given key exists or not is indeed the EXISTS command: EXISTS nokey :0 EXISTS foo :1 As you can see the server replied ':0' the first time since 'nokey' does not exist, and ':1' for 'foo', a key that actually exists. Replies starting with the colon character are integer reply. Ok... now you know the basics, read the REDIS COMMAND REFERENCE section to learn all the commands supported by Redis and the PROTOCOL SPECIFICATION section for more details about the protocol used if you plan to implement one for a language missing a decent client implementation. LicenseRedis is released under the BSD license. See the COPYING file for more information. CreditsRedis is written and maintained by Salvatore Sanfilippo, Aka 'antirez'. Enjoy, antirez |
Sign in to add a comment
The documentation for the SET command is somewhat confusing. it implies that "SET key value" sets "key" to "value", but the bulk operations docs look like "value" there is really the length of the value, and then you have the actual value.
Hello Dane, yes I'm improving the docs, the SVN version is already a bit better (but note that the protocol it describes is not the one of beta-3 but the one of beta-4, there is some minor change to some command behavior).
Btw the commands section describe the semantic of the command, not the protocol. Every command that takes as last argument data it takes it in form of a bulk write. Only the last argument can be a bulk write argument btw.
Please if you want more details subscribe to the redis google group, I'll be very glad to describe every non clear point of the doc by mail, and then of course fix the doc too :) Thanks for your interest.
The description of LLEN says it returns -1 if the value is not a list; but the return-value doc says it returns -2. I'm guessing the second one is a typo. It might actually be a good idea to define a set of numeric error codes across the entire API. Operations that return errors as strings could return the numeric code instead or in addition to the string; this helps clients detect which error they got, and to localize error messages for the end user.
Thanks Jens, actually they are defined but there is a syntax error in the wiki:
ok documentation fixed.
I miss a delete command on a list by value. For example a user would like to delete a twitter message in your twitter clone. If you think I can detail the problem.
Hello, yes I've this in the TODO list
LSEARCH mylist <element>
returns the index of the first element, or -1 if it was not found.
LREM mylist <element>
remove all the occurrences of element in the list, returns 0 if the element was not found, 1 if the element was found and removed.
I don't know if it can be a good idea to add a command to remove just the first or the last occurrence found. Any hint?
Btw I need as much details as possible on your specific problem in order to understand better how generic it is. Thank you very much!
Yes, that LREM seems to be the solution.
So, let's say I would like to setup a twitter clone. As I see, I can store all the messages in a list, but if the user would like to delete a message, that's not possible with an atomic commmand, but I need two (getting the item number, deleting the item). As the list can change meanwhile, that seems to be a bad solution. So LREM seems to be a good idea for that if I'm including a timestamp or so in the messages, so they're unique.
Hello barthazi, there is a much simpler solution to implement comment deletion. Just remove the post<id> key and leave the ID on the list, but handle missing comments in pagination.
Anyway I think LREM o LDEL (I don't know how'll name it!) is an important addition. Also note that Redis is going to implement locking with key granularity so it will be possible to create new atomic operations from scratch.
(off: Andras is my first name, but we put it after the family name in Hungary just like Japanese)
You're right, but handling missing comments when I would like to get 10 items from the 3rd page can be quite difficult, and would cause additional requests (if a user delete a lot of comments, for example). And deleting the id from the list is the same case as I have wrote about. Anyway I agree with storing just the id in the list, this seems to be a better design solution.
Andras, I want to tell you that your country is fantastic, I was there the last summer and I had a very good time with wonderful food! In Italy it was also in use to put the name after the surname, now it is rarely used AFAIK.
About LREM, yes if there are a lot of deletes it's hard to handle, and actually deletion is used rarely enough so it's not a huge problem that LREM is O(N) in most of the cases. I'll absolutely add it, currently I'm just not sure if it should remove the first element, all the elements, or the two forms are required...
I think that to remove only the first occurrence is a good idea, since probably in many applications you have only this first occurrence, and you can stop scanning the list as soon as this element was found.
Thanks for the nice words. :) I think deleting only the first occurrence may be okay for most of the cases, but adding an optional parameter, or an other command to remove all the occurrences may be not a bad idea. Also implementing LOCK is a good idea.
Hello again Andras! I'm adding both the possibilities. Basically it is something like this:
LREM key count
LREM will start to remove elements from the List at key until count elements are removed. If count is 0 LREM will simply remove ALL the elements matching.
The command returns the number of elements removed. We should cover a lot of cases with this, just it is not able to remove elements in the reverse order, starting from tail...
Andras, we have LREM in the SVN. It's more powerful than the one I described since it supports a negative 'count' parameter to delete elements starting from the tail.
Cheers, antirez.
Hello. I am considering using redis (instead of memcache) as a cache server in front of multiple instances of MySQL DB instances. Do u know of anybody who has used redis + MySQL? Are there any guidelines or configuration info?
Regards, DT
Hello DT,
I use it myself exactly in this configuration :) We replaced Memcached with Redis and got a boost in latency, I don't understand if we did something from with Memcached or the Ruby lib we were using was slow, but it's much faster now using Redis.
Guidelines: well, EXPIRE is under development so for now you don't have the ability to set timeouts. Just make sure to disable the disk saving if for you it's just a cache, or maybe just save from time to time, like every 30 minutes.
This is how we work with Redis as cache:
when we want an object, we ask for this object in Redis. If we get 'nil' as reply the object is not on the cache, so we compute it and SET it back into the Redis server, serialized with Json, or simply as an HTML string in parts where we can just cache the HTML and don't care (not always possible).
When instead an object is modified, we just DEL-ete it. So the object will be created on the cache only if it will be requested again. The lazy-way.
That's all... please for more information make sure to subscribe to the Redis google group. Thank you very much.
Antirez, Thanks a lot for valuable feedback. I will subscribe, per your suggestion. DT
It might be helpful to mention somewhere that lists are implemented as a normal doubly-linked list (as opposed to an array). Once I realized this, it made a lot more sense why operations have the runtimes that they do.
For example, now it makes sense why LPUSH and LPOP are O(1), but LINDEX is O(N). And it makes sense that LINDEX(+/- small number) can be done quickly in practice, since you can just traverse from the head/tail (whichever is closer).
Note that TC can and will corrupt its database and applications may not notice. I'm thinking my time porting it to Win32 was wasted. TC directly mmaps the main hash bucket array and the start of the data, and as soon as you write to that mapped data the OS can push that into the file - which leads to a window for corruption from that first memory update to the completion of both the mflush and the flush of any data written using the file update APIs. There is no flag to indicate that the database was left in a dirty state, either. (Well, I should look at the latest code, but a change to fix this would be a big change)
A comparison with GigaBASE and/or FastDB is probably in order.
Hello, every time Redis saves it performs a full dump on disk, on a temp file, atomically renamed when the save finished. So it's always safe to copy the data while the server is running.
Hi, looks like this is the project I've been looking for.
One question though, is a C (not C++) library available?
Hi,
I am playing with Redis and its Tcl interface, congrats for the great job.
Two questions:
A) I am reading some utf-8 file via Tcl and want to store it in redis, but I suspect I am doing something wrong as I don't get my utf-8 back. Is there any particular point I should know when dealing with utf-8 and redis?
B) From the Tcl interface, if I ask for a key that does not exists I get, or so it seems, and empty string. Shouldn't I be getting a 'nil' code or anything like it?
Thanks a lot. Redis is great.
jima
There is outdated info on the page: 'Values can be Strings, Lists or Sets. Keys can be a subset of strings not containing newlines ("\n") and spaces (" ").' With new multi-bulk command protocol (from v1.1 and up) this limitation is no longer there.
@birukoff: in theory this limitation is no longer here but for latency concerns client libs should continue to use the old protocol when possible, so actually it's safe to consider keys not binary safe at least for now.