My favorites | Sign in
Project Home Wiki
Search
for
SimpleDbIntro  
Introduction to SimpleDB in Boto
Featured
Updated Oct 26, 2009 by Mitch.Ga...@gmail.com

Introduction

This page will give you a quick overview of the current boto SimpleDB library. Before reading this it would probably be a good idea to check out these resources from Amazon:

Let's Get Started

To begin with SimpleDB, you need to create a connection to the SimpleDB service just like you do with all other boto interfaces.

>>> import boto
>>> sdb = boto.connect_sdb('<your aws access key>', '<your aws secret key'>)

Create a Domain

Once you have a connection, you can create a new domain. According to the AWS docs:

All Amazon SimpleDB information is stored in domains. Domains are similar to tables that contain similar data. You can execute queries against a domain, but cannot execute joins between domains.

To create a new domain:

>>> domain = sdb.create_domain('my_domain')

To list the all of the items in the domain, you could do this:

>>> for item in domain:
...     print item.name
... 

Of course, this is a brand new domain so there are no items contained within it yet. However, the above construct would cycle through ALL items within the domain. The paging of results is handled automatically by boto via a Python generator function.

Adding Items to the Domain

To create a new item within our new domain:

>>> item = domain.new_item('item1')

Because this item has no attribute name/value pairs associated with it, it really doesn't exist yet in SimpleDB. It won't get persisted to SimpleDB until we call the .save() method on it. But first, let's add some key/value pairs to it:

>>> item['key1'] = 'value1'
>>> item['key2'] = 'value2'

In addition to storing scalar string values, you can also store lists of values like this:

>>> item['key3'] = ['one', 'two', 'three']

Or you can append values to existing attribute names using the add_value method like this:

>>> item.add_value('key2', 'value2_1')

With add_value, the value you add is to the attribute name ('key2') in addition to any existing values. So, it's like appending to a list.

Time to save it to SimpleDB:

>>> item.save()

Getting Items from the Domain

retrievedItem If we now go and ask the domain object to retrieve the Item called 'item1' from SimpleDB, we can see our values have, in fact, been stored:

>>> retrievedItem = domain.get_item('item1')
>>> retrievedItem
{u'key2': [u'value2', u'value2_1'], u'key1': u'value1', 'u'key3': [u'one', u'two', u'three']}
>>> 

If we now go back and iterate over the items in the domain, we can see our new item, 'item1' show up.

>>> for item in domain:
...     print item.name
... 
item1

Note that item object returned by the query resultset iterator is a live Item object. When returned by the resultset object, it contains only the name of the item because SimpleDB does not allow attribute values to be returned as part of a query, only item names. However, if you access an attribute of the Item object, it will automatically retrieve all attributes for the item from SimpleDB. So, in our above example we could also do something like this:

>>> for item in domain:
...     print item.name, item['key1']
...
item1, value1

Querying SimpleDB

We can also query our domain, like this:

>>> rs = domain.query("['key1' = 'value1']")
>>> for item in rs:
...     print item.name
... 
item1
>>> 

Here again, the paging of search results will be handled transparently by boto. If there are 10,000 items matching the query you will eventually pull all of those results over (unless SimpleDB cuts you off for taking up too much compute time!).

If you already created your domain, and you want to reconnect to it:

>>> retrievedDomain = sdb.get_domain('my_domain')
>>> for item in retrievedDomain:
...     print item.name
... 
item1
>>> 

More examples and good stuff to follow. Enjoy and don't be shy with those comments!

Mitch

Comment by cbme...@gmail.com, Dec 31, 2007

As always, boto rocks!

Comment by project member Mitch.Ga...@gmail.com, Jan 1, 2008

Thanks!

I have actually made a change locally that I am considering incorporating into subversion. Basically, I changed the QueryResultSet? to return Item objects rather than item names. Then, if you actually want to pull the attribute values over you would just reference one of the attribute names and it would do it behind the scenes.

Comment by cbme...@gmail.com, Jan 3, 2008

Sounds pretty cool.

Right now I am trying to learn the querying more. I am writing a complete user registration and user logon module using boto and Django.

Going very well!

Comment by project member Mitch.Ga...@gmail.com, Jan 4, 2008

I have updated the code in subversion to have the ResultSet? object return Item objects rather than item names. The examples on this page have been updated to reflect this change.

A new release of boto will be coming soon that incorporates these and other changes.

Comment by cbme...@gmail.com, Jan 4, 2008

Cool...can't wait!

Comment by cbme...@gmail.com, Jan 4, 2008
Comment by zelo.z...@gmail.com, Feb 18, 2008

This tutorial is wrong here:

>>> for item in domain:
...     print item.name
... 

This should be

>>> for item_name in domain:
...     item = domain.get_item(item_name)
...     print item
... 
Comment by project member Mitch.Ga...@gmail.com, Feb 19, 2008

It depends, actually. This does accurately describe how the current version in SVN works. But I think the last official release works as you describe. I'm planning on a new release later this week and that will resolve the inconsistency.

Comment by swir...@gmail.com, Mar 16, 2008

item.name doesn't work! I need to do the form specified by zelo

Comment by swir...@gmail.com, Mar 22, 2008

Item.name works now, with the latest release....

Comment by raja.dor...@gmail.com, Jul 1, 2008

I wonder why simpledb response seems to be incredibly slow for updating items. I only have about 30 items and it takes several seconds to update them all. Any ideas?

Comment by mrsixco...@gmail.com, Jul 31, 2008

how do you get the next token?

Comment by project member Mitch.Ga...@gmail.com, Jul 31, 2008

I'm not sure what you mean by "token". If you mean the token sent from SimpleDB informing you that there are more results available, that's all handle by the generator function. Or am I misunderstanding your question?

Comment by mrsixco...@gmail.com, Jul 31, 2008

trying to count the items in the database. So I was making a query to simpledb and if there is a next_token. So I make my first query("['value' > '1']",250) if the next_token existed then I would add 250 to my count and pass the next_token to the query again query("['value' > '1']",250,next_token). If it doesn't find it then it would go through and count the remaining items. Or is there an easier way to count the results? Thanks in advance, Dean

Comment by mrsixco...@gmail.com, Jul 31, 2008

Just reread your post. Is there an easy way to count the results?

Comment by mrsixco...@gmail.com, Jul 31, 2008

That is without going through the whole recordset.

Comment by jason.va...@gmail.com, Sep 29, 2008

Does boto support multi-valued queries? Perhaps my syntax is incorrect?

query_string = "['%s'='%s' and '%s'='%s']" % ('field_1', '0', 'field_2', 'abc')
rs = domain.query(query_string)
Comment by jeffreym...@gmail.com, Nov 30, 2008

The boto.connect_sdb() step is going to fail for the vast majority of readers who have not already internalized the configuration step. Maybe for this it's best to use the boto.connect_sdb(key, secret) form of the call instead?

Comment by telegram...@gmail.com, Dec 2, 2008

It might be worth mentioning in the example that create_domain() should only be used once, and then lookup() or get_domain(). While calling create_domain() repeatedly does return the expected results, it adds a good 5 seconds of delay each time it is called.


Sign in to add a comment
Powered by Google Project Hosting