|
SimpleDbIntro
Introduction to SimpleDB in Boto
Featured IntroductionThis page will give you a quick overview of the current boto SimpleDB library. Before reading this it would probably be a good idea to check out these resources from Amazon: Let's Get StartedTo begin with SimpleDB, you need to create a connection to the SimpleDB service just like you do with all other boto interfaces. >>> import boto
>>> sdb = boto.connect_sdb('<your aws access key>', '<your aws secret key'>)Create a DomainOnce you have a connection, you can create a new domain. According to the AWS docs: All Amazon SimpleDB information is stored in domains. Domains are similar to tables that contain similar data. You can execute queries against a domain, but cannot execute joins between domains. To create a new domain: >>> domain = sdb.create_domain('my_domain')To list the all of the items in the domain, you could do this: >>> for item in domain: ... print item.name ... Of course, this is a brand new domain so there are no items contained within it yet. However, the above construct would cycle through ALL items within the domain. The paging of results is handled automatically by boto via a Python generator function. Adding Items to the DomainTo create a new item within our new domain: >>> item = domain.new_item('item1')Because this item has no attribute name/value pairs associated with it, it really doesn't exist yet in SimpleDB. It won't get persisted to SimpleDB until we call the .save() method on it. But first, let's add some key/value pairs to it: >>> item['key1'] = 'value1' >>> item['key2'] = 'value2' In addition to storing scalar string values, you can also store lists of values like this: >>> item['key3'] = ['one', 'two', 'three'] Or you can append values to existing attribute names using the add_value method like this: >>> item.add_value('key2', 'value2_1')With add_value, the value you add is to the attribute name ('key2') in addition to any existing values. So, it's like appending to a list. Time to save it to SimpleDB: >>> item.save() Getting Items from the DomainretrievedItem If we now go and ask the domain object to retrieve the Item called 'item1' from SimpleDB, we can see our values have, in fact, been stored: >>> retrievedItem = domain.get_item('item1')
>>> retrievedItem
{u'key2': [u'value2', u'value2_1'], u'key1': u'value1', 'u'key3': [u'one', u'two', u'three']}
>>> If we now go back and iterate over the items in the domain, we can see our new item, 'item1' show up. >>> for item in domain: ... print item.name ... item1 Note that item object returned by the query resultset iterator is a live Item object. When returned by the resultset object, it contains only the name of the item because SimpleDB does not allow attribute values to be returned as part of a query, only item names. However, if you access an attribute of the Item object, it will automatically retrieve all attributes for the item from SimpleDB. So, in our above example we could also do something like this: >>> for item in domain: ... print item.name, item['key1'] ... item1, value1 Querying SimpleDBWe can also query our domain, like this: >>> rs = domain.query("['key1' = 'value1']")
>>> for item in rs:
... print item.name
...
item1
>>> Here again, the paging of search results will be handled transparently by boto. If there are 10,000 items matching the query you will eventually pull all of those results over (unless SimpleDB cuts you off for taking up too much compute time!). If you already created your domain, and you want to reconnect to it: >>> retrievedDomain = sdb.get_domain('my_domain')
>>> for item in retrievedDomain:
... print item.name
...
item1
>>> More examples and good stuff to follow. Enjoy and don't be shy with those comments! Mitch |
As always, boto rocks!
Thanks!
I have actually made a change locally that I am considering incorporating into subversion. Basically, I changed the QueryResultSet? to return Item objects rather than item names. Then, if you actually want to pull the attribute values over you would just reference one of the attribute names and it would do it behind the scenes.
Sounds pretty cool.
Right now I am trying to learn the querying more. I am writing a complete user registration and user logon module using boto and Django.
Going very well!
I have updated the code in subversion to have the ResultSet? object return Item objects rather than item names. The examples on this page have been updated to reflect this change.
A new release of boto will be coming soon that incorporates these and other changes.
Cool...can't wait!
Hey, maybe this can be applied to a small tutorial??
http://codershangout.com/blogs/cbmeeks/2008/jan/how-select-top-x-domain-boto-and-simpledb
This tutorial is wrong here:
This should be
It depends, actually. This does accurately describe how the current version in SVN works. But I think the last official release works as you describe. I'm planning on a new release later this week and that will resolve the inconsistency.
item.name doesn't work! I need to do the form specified by zelo
Item.name works now, with the latest release....
I wonder why simpledb response seems to be incredibly slow for updating items. I only have about 30 items and it takes several seconds to update them all. Any ideas?
how do you get the next token?
I'm not sure what you mean by "token". If you mean the token sent from SimpleDB informing you that there are more results available, that's all handle by the generator function. Or am I misunderstanding your question?
trying to count the items in the database. So I was making a query to simpledb and if there is a next_token. So I make my first query("['value' > '1']",250) if the next_token existed then I would add 250 to my count and pass the next_token to the query again query("['value' > '1']",250,next_token). If it doesn't find it then it would go through and count the remaining items. Or is there an easier way to count the results? Thanks in advance, Dean
Just reread your post. Is there an easy way to count the results?
That is without going through the whole recordset.
Does boto support multi-valued queries? Perhaps my syntax is incorrect?
query_string = "['%s'='%s' and '%s'='%s']" % ('field_1', '0', 'field_2', 'abc') rs = domain.query(query_string)The boto.connect_sdb() step is going to fail for the vast majority of readers who have not already internalized the configuration step. Maybe for this it's best to use the boto.connect_sdb(key, secret) form of the call instead?
It might be worth mentioning in the example that create_domain() should only be used once, and then lookup() or get_domain(). While calling create_domain() repeatedly does return the expected results, it adds a good 5 seconds of delay each time it is called.