Updated Dec 28, 2007 by Mitch.Garnaat
Labels:	Featured

SimpleDbIntro

Introduction to SimpleDB in Boto

Introduction

This page will give you a quick overview of the current boto SimpleDB library. Before reading this it would probably be a good idea to check out these resources from Amazon:

Let's Get Started

To begin with SimpleDB, you need to create a connection to the SimpleDB service just like you do with all other boto interfaces.

>>> import boto
>>> sdb = boto.connect_sdb()

Create a Domain

Once you have a connection, you can create a new domain. According to the AWS docs:

All Amazon SimpleDB information is stored in domains. Domains are similar to tables that contain similar data. You can execute queries against a domain, but cannot execute joins between domains.

To create a new domain:

>>> domain = sdb.create_domain('my_domain')

To list the all of the items in the domain, you could do this:

>>> for item in domain:
...     print item.name
...

Of course, this is a brand new domain so there are no items contained within it yet. However, the above construct would cycle through ALL items within the domain. The paging of results is handled automatically by boto via a Python generator function.

Adding Items to the Domain

To create a new item within our new domain:

>>> item = domain.new_item('item1')

Because this item has no attribute name/value pairs associated with it, it really doesn't exist yet in SimpleDB. However, by adding some key/value pairs to it, we automatically store those values in SimpleDB:

>>> item['key1'] = 'value1'
>>> item['key2'] = 'value2'

In addition to storing scalar string values, you can also store lists of values like this:

>>> item['key3'] = ['one', 'two', 'three']

Or you can append values to existing attribute names using the add_value method like this:

>>> item.add_value('key2', 'value2_1')

With add_value, the value you add is to the attribute name ('key2') in addition to any existing values. So, it's like appending to a list.

Getting Items from the Domain

If we now go and ask the domain object to retrieve the Item called 'item1' from SimpleDB, we can see our values have, in fact, been stored:

>>> item = domain.get_item('item1')
>>> item
{u'key2': [u'value2', u'value2_1'], u'key1': u'value1', 'u'key3': [u'one', u'two', u'three']}
>>>

If we now go back and iterate over the items in the domain, we can see our new item, 'item1' show up.

>>> for item in domain:
...     print item.name
... 
item1

Note that item object returned by the query resultset iterator is a live Item object. When returned by the resultset object, it contains only the name of the item because SimpleDB does not allow attribute values to be returned as part of a query, only item names. However, if you access an attribute of the Item object, it will automatically retrieve all attributes for the item from SimpleDB. So, in our above example we could also do something like this:

>>> for item in domain:
...     print item.name, item['key1']
...
item1, value1

Querying SimpleDB

We can also query our domain, like this:

>>> rs = domain.query("['key1' = 'value1']")
>>> for item in rs:
...     print item.name
... 
item1
>>>

Here again, the paging of search results will be handled transparently by boto. If there are 10,000 items matching the query you will eventually pull all of those results over (unless SimpleDB cuts you off for taking up too much compute time!).

More examples and good stuff to follow. Enjoy and don't be shy with those comments!

Mitch

Comment by cbmeeks, Dec 31, 2007

As always, boto rocks!

Comment by Mitch.Garnaat, Jan 01, 2008

Thanks!

I have actually made a change locally that I am considering incorporating into subversion. Basically, I changed the QueryResultSet? to return Item objects rather than item names. Then, if you actually want to pull the attribute values over you would just reference one of the attribute names and it would do it behind the scenes.

Comment by cbmeeks, Jan 03, 2008

Sounds pretty cool.

Right now I am trying to learn the querying more. I am writing a complete user registration and user logon module using boto and Django.

Going very well!

Comment by Mitch.Garnaat, Jan 04, 2008

I have updated the code in subversion to have the ResultSet? object return Item objects rather than item names. The examples on this page have been updated to reflect this change.

A new release of boto will be coming soon that incorporates these and other changes.

Comment by cbmeeks, Jan 04, 2008

Cool...can't wait!

Comment by cbmeeks, Jan 04, 2008

Hey, maybe this can be applied to a small tutorial??

http://codershangout.com/blogs/cbmeeks/2008/jan/how-select-top-x-domain-boto-and-simpledb

Comment by zelo.zejn, Feb 18, 2008

This tutorial is wrong here:

>>> for item in domain:
...     print item.name
...

This should be

>>> for item_name in domain:
...     item = domain.get_item(item_name)
...     print item
...

Comment by Mitch.Garnaat, Feb 19, 2008

It depends, actually. This does accurately describe how the current version in SVN works. But I think the last official release works as you describe. I'm planning on a new release later this week and that will resolve the inconsistency.

Comment by swirsky, Mar 16, 2008

item.name doesn't work! I need to do the form specified by zelo

Comment by swirsky, Mar 22, 2008

Item.name works now, with the latest release....

Comment by raja.doraisamy, Jul 01 (4 days ago)

I wonder why simpledb response seems to be incredibly slow for updating items. I only have about 30 items and it takes several seconds to update them all. Any ideas?