|
Project Information
Members
|
Pickled Object Database ("pod")Please note -- development of this project is currently paused. Based on some initial feedback, some large changes to the pod architecture are planned and I am working on a 0.6.0 release which fixes these problems. pod is a Object Relational Mapper written in Python and built upon the standard Python packages cPickle and the sqlite3 interface written by Gerhard Häring. The goal of pod is to provide an easy to setup and easy to use queryable database for arbitrary Python objects. To this end, pod provides a dynamic, schemaless object database API where you work with instances just as you would with normal Python instances in memory -- and no setup is required. You can create complex interwoven data structures without needing to define typed 'one-to-one' relationships for every object reference. Also, pod provides scalable list, set, and dict collections which can be used in place of one-to-many or many-to-many relationships. Since pod is built on sqlite it provides the ability to define 'typed' attributes within a class to increase database performance when and where it's needed (these are implemented using SQL columns as in other ORM systems). Further, changing attributes from dynamic to typed and vice versa can be performed at any time on the fly and does not require a schema migration tool or strategy. Whether you're using dynamic or typed attributes, pod provides the ability to perform 'ad-hoc' queries. While scalable collections are provided, the ability to query the database further enables the use of relational database structures where appropriate (using a 'typed' attribute is usually more efficient than using a general 'dynamic' scalable collection). Further, pod provides simple mechanisms for adding SQL indexes to both dynamic and typed attributes which greatly increases speed of queries at expense of slower insertions and a larger database. pod is built on sqlite and has the same scalability constraints. pod is ideally suited for people engaged in ground up projects who are willing to trade off scalability for a dynamic easy to use database API. Currently, pod is alpha stage, is still experimental, and is in active development. The API is fairly stable but still fluid. We are actively working towards a version 1.0 stable platform suitable for new development. So, of course, we're looking for any kind of help -- including just using pod and giving us some feedback. For more, look below or visit our project wiki. Overview
ExampleCreating Some Models and Objectsimport pod
import pod.set
class Person(pod.Object): # Any instances created by a class that descends from pod.Object
pass # will automatically persist.
class Tycoon(Person): # Now, add a class Tycoon that descends from Person . . .
def __init__(self, **kwargs):
Person.__init__(self, **kwargs)
self.mansions = [] # Also note, attributes can be of any type that can be pickled
self.villas = {} # including lists, sets, and dictionaries. Note, these are native
# Python collections and are not scalable but are fine for small collections.
self.workers = pod.set.Set() # If you want a scalable collection implemented using SQL, use the
self.yachts = pod.set.Set() # pod.list.List, pod.dict.Dict, or pod.set.Set collection objects.
class WorkerBee(Person):
def __init__(self, boss, **kwargs):
Person.__init__(self, **kwargs)
self.boss = boss # Pointer to pod object, much like foreign key.
self.boss.workers.add(self) # This pod.set.Set collection acts as one-to-many relation
def pre_delete(self): # In pod, nothing is done 'automatically' for you. You have to
self.boss.workers.remove(self) # take care of cleanup yourself
# Connect before making any instances . . .
db = pod.Db(file = 'mypod.sqlite3', dynamic_index = True) # By setting the dynamic_index to 'True' an SQL index is created for
# all untyped 'dynamic' attributes enabling fast queries
# (note, the default setting is 'index=False').
hsing = Person(name = 'Hsing', some_attr = 'foo') # Now, add some Person, Tycoon, and WorkerBee objects to
helen = Person(name = 'Helen', some_other_attr = ['bar', 'baz']) # the database . . .
don = Tycoon(name = 'Don', age = 63, catch_phrase = "You're fired!")
george = Tycoon(name = 'George', age = 61, catch_phrase = "I guarantee it")
bruce = WorkerBee(name = 'Bruce', age = 40, boss = don, random_attr = 10.23)
azhar = WorkerBee(name = 'Azhar', age = 40, boss = george, random_attr = {'key': ['value', (1, 2, 3)]})
db.commit() # or use db.rollback() to cancel this transaction.
Fetching Some ObjectsNow, in the same or some other script:
for peep in Person: # Note that any pod.Object class is itself an iterator . . .
print peep.name # Prints 'Hsing', 'Helen', 'Don', 'George', 'Bruce', 'Azhar'
for peep in Person: # Note, instances do not need to all have the same attributes.
try:
print peep.some_attr # Prints 'foo', then throws KeyError
except:
print getattr(peep, 'some_attr', None) # Prints None, None, None, None, None
for peep in [peep for peep in Person if peep.name[0] == 'H']: # You can 'query' the database using list comprehensions.
print peep.name # Prints 'Hsing', 'Helen'
# Just note, every object loaded from db and compared in Python.
for peep in Person.where.name[0] == 'H': # To query more efficently using SQL, use pod query syntax.
print peep.name # Prints 'Hsing', 'Helen'
for peep in Person.where.random_attr == {'key': ['value', (1, 2, 3)]}: # You can query with == and != on any type of object.
print peep.name # Print 'Azhar'
for peep in Person.where.age > 62: # '=', '!=', '<', '>', '<=', '>=' are very fast if you've defined an index
print peep.name, peep.age # Prints 'Don', 63
for peep in Person.where.some_other_attr: # This just gets all peeps that have an attribute 'some_other_attr'
print peep.some_other_attr # Prints ['bar', 'baz']
for peep in (Person.where.age > 62) & (Person.where.age < 64): # You can also chain together query conditionals
print peep.name, peep.age # Prints 'Don', 63
don = (Tycoon.where.name == 'Don').get_one() # Here, we show that pod maintains a single object reference.
bruce = (WorkerBee.where.name == 'Bruce').get_one() # In essence, pod recreates your original memory space.
print don is bruce.boss # Prints True.
for bee in don.workers: # You can iterate through pod collections just like native
bee.fired = True # Python collections
hsing = (Person.where.name == 'Hsing').get_one() # Also, any pod.Object instance implements the dictionary interface.
for key,value in hsing.iteritems(id = False): # If 'id = False', will not return the instance id item (default behavior).
print key,value # Prints 'name', 'Hsing' and 'some_attr', 'foo'
Defining Typed Attributes
class Yacht(pod.Object): # Let's add another class that descends directly from pod.Object.
name = pod.typed.String(index = False) # Here, we type the attributes in order to allow faster raw SQL queries.
age = pod.typed.Int(index = True) # You don't need to do this -- it just makes insertion/querying faster.
length = pod.typed.Float(index = True) # Add an SQL index to make it faster at expense of slower insert.
owner = pod.typed.Object(index = True) # You can even make an attribute of type typed.Object which can operate like a
# 'foreign id' column -- except it will accept any type of
# Python object that is pickelable including all pod.Object instances.
def __init__(self, owner, **kwargs):
pod.Object.__init__(self, **kwargs)
self.owner = owner
self.owner.yachts.add(self)
self.photos = [] # pod Objects can have a mix of dynamic and typed attributes.
Yacht(owner = george, name = 'The Trickle Downer', length = 40.5)
Yacht(owner = george, name = 'The TARP Runner', length = 42.1, some_random_attr = 'foo')
db.commit()
for yacht in Yacht.where.name == 'The Trickle Downer': # The query syntax using typed attributes is exactly the same.
print yacht.owner.name # Prints 'George'
for yacht in Yacht.name == 'The Trickle Downer': # However, with typed attributes you can drop the 'where' if you want.
print yacht.owner.name # Prints 'George'
for yacht in george.yachts: # You can iterate through this pod.set.Set . . .
print yacht.owner.name # Prints 'George', 'George'
for yacht in Yacht.owner == george: # Or you could have used a relational structure instead of a set . . . you choose
print yacht.owner.name # Prints 'George', 'George'
query = pod.Query(select = Yacht.name | Yacht.length, # Or, for full SQL control, use a query object.
where = (Yacht.length < 41) | (Yacht.length == 42.1), # Conditionals are chained together with '|' or '&'.
order_by = Yacht.length.desc(),
limit = 2)
for yacht in query: # Now iterate on the query . . .
print yacht.length # Prints 42.1, 40.5
if yacht.length < 41: # Just like regular Python objects, you can add attributes
yacht.another_random_attr = ['foo', 'bar', 'baz'] # on the fly . . .
The pod store AttributeAll pod db instances and pod.Object classes have a 'store' attribute which can be used like a Python shelve.
db.store['some_list'] = [10, 20, 30] # Note, Each 'store' is seperate from all others.
db.store.another_list = [1, 1, 2 ] #
Person.store['some_list'] = [40, 50, 60] # Also, you can use dictionary '[ ]' notation
Tycoon.store.main_tycoon = george # or object '.' notation.
db.commit() # Commit changes to the 'stores' to database.
for yacht in Tycoon.store.main_tycoon.yachts: # The store is useful for storing pointers to objects you
print yacht.owner.name # want access to later on -- saving you a SQL query.
# Using the store in this way to access the database is
# similar to how you access the database in other ODBMS systems
# like ZODB or Durus.
Design Features
For documentation and more discussion on pod's inner workings, please visit the project wiki. Design Goals
Requirements
|
|
|
