The App Engine datastore supports transactions. A transaction is an operation or set of operations that either succeeds completely, or fails completely. An application can perform multiple operations and calculations in a single transaction.
A transaction is a datastore operation or a set of datastore operations that either succeed completely, or fail completely. If the transaction succeeds, then all of its intended effects are applied to the datastore. If the transaction fails, then none of the effects are applied.
Every datastore write operation is atomic. An attempt to create, update or delete an entity either happens, or it doesn't. An operation may fail due to a high rate of contention, with too many users trying to modify an entity at the same time. Or an operation may fail due to the application reaching a quota limit. Or there may be an internal error with the datastore. In all cases, the operation's effects are not applied, and the datastore API raises an exception.
An application can execute a set of statements and datastore operations in a single transaction, such that if any statement or operation raises an exception, none of the datastore operations in the set are applied. The application defines the actions to perform in the transaction using a Python function, then calls db.run_in_transaction() with the function as an argument:
|
from google.appengine.ext import db
class Accumulator(db.Model):
counter = db.IntegerProperty()
def increment_counter(key, amount):
obj = db.get(key)
obj.counter += amount
obj.put()
q = db.GqlQuery("SELECT * FROM Accumulator")
acc = q.get()
db.run_in_transaction(increment_counter, acc.key(), 5)
db.run_in_transaction() takes the function object, and positional and keyword arguments to pass to the function. If the function returns a value, db.run_in_transaction() will return the value.
If the function returns, the transaction is committed, and all effects of datastore operations are applied. If the function raises an exception, the transaction is "rolled back," and the effects are not applied.
If the function raises the Rollback exception, db.run_in_transaction() returns None. For any other exception, db.run_in_transaction() re-raises the exception.
The datastore imposes several restrictions on what can be done inside a single transaction.
All datastore operations in a transaction must operate on entities in the same entity group. This includes querying for entities by ancestor, retrieving entities by key, updating entities, and deleting entities. Notice that each root entity belongs to a separate entity group, so a single transaction cannot create or operate on more than one root entity. For an explanation of entity groups, see Keys and Entity Groups.
An app can perform a query during a transaction, but only if it includes an ancestor filter. An app can also get datastore entities by key during a transaction. You can prepare keys prior to the transaction, or you can build keys inside the transaction with key names or IDs.
All other Python code is allowed inside a transaction function. The transaction function should not have side effects other than the datastore operations. The transaction function may be called multiple times if a datastore operation fails due to another user updating entities in the entity group at the same time. When this happens, the datastore API retries the transaction a fixed number of times. If they all fail, db.run_in_transaction() raises a TransactionFailedError. You can adjust the number of times the transaction is retried using db.run_in_transaction_custom_retries() instead of db.run_in_transaction().
Similarly, the transaction function should not have side effects that depend on the success of the transaction, unless the code that calls the transaction function knows to undo those effects. For example, if the transaction stores a new datastore entity, saves the created entity's ID for later use, then the transaction fails, the saved ID does not refer to the intended entity because the entity's creation was rolled back. The calling code would have to be careful not to use the saved ID in this case.
The datastore's isolation level outside of transactions is closest to READ_COMMITTED. Inside transactions, on the other hand, the isolation level is SERIALIZABLE, specifically a form of snapshot isolation. See the Transaction Isolation article for more information on isolation levels.
Queries and gets inside a transaction are guaranteed to see a single, consistent snapshot of the datastore as of the beginning of the transaction. In particular, entities and index rows in the transaction's entity group are fully updated so that queries will return the complete, correct set of result entities, without the false positives or false negatives described in Transaction Isolation that can occur in queries outside transactions.
This consistent snapshot view also extends to reads after writes inside transactions. Unlike with most databases, queries and gets inside a datastore transaction do not see the results of previous writes inside that transaction. Specifically, if an entity is modified or deleted within a transaction, a query or get will return the original version of the entity as of the beginning of the transaction, or nothing if the entity did not exist then.
This example demonstrates one use of transactions: updating an entity with a new property value relative to its current value.
def increment_counter(key, amount):
obj = db.get(key)
obj.counter += amount
obj.put()
This requires a transaction because the value may be updated by another user after this code fetches the object, but before it saves the modified object. Without a transaction, the user's request will use the value of counter prior to the other user's update, and the save will overwrite the new value. With a transaction, the application is told about the other user's update. If the entity is updated during the transaction, then the transaction is retried until all steps are completed without interruption.
Another common use for transactions is to update an entity with a named key, or create it if it doesn't yet exist:
class SalesAccount(db.Model):
address = db.PostalAddressProperty()
phone_number = db.PhoneNumberProperty()
def create_or_update(parent_obj, account_id, address, phone_number):
obj = db.get(Key.from_path("SalesAccount", account_id, parent=parent_obj))
if not obj:
obj = SalesAccount(key_name=account_id,
parent=parent_obj,
address=address,
phone_number=phone_number)
else:
obj.address = address
obj.phone_number = phone_number
obj.put()
As before, a transaction is necessary to handle the case where another user is attempting to create or update an entity with the same string ID. Without a transaction, if the entity does not exist and two users attempt to create it, the second will overwrite the first without knowing that it happened. With a transaction, the second attempt will retry, notice that the entity now exists, and update the entity instead.
Create-or-update is so useful that there is a built-in method for it: Model.get_or_insert() takes a key name, an optional parent, and arguments to pass to the model constructor if an entity of that name and path does not exist. The get attempt and the create happen in one transaction, so (if the transaction is successful) the method always returns a model instance that represents an actual entity.
Tip: A transaction should happen as quickly as possible to reduce the likelihood that the entities used by the transaction will change, requiring the transaction be retried. As much as possible, prepare data outside of the transaction, then execute the transaction to perform datastore operations that depend on a consistent state. The application should prepare keys for objects used inside the transaction, then fetch the entities inside the transaction.
Finally, a transaction can be used to read a consistent snapshot of the datastore. This can be useful when multiple reads gets are needed to render a page or export data that must be consistent. These kinds of transactions are often called read-only transactions, since they perform no writes. Committing and rolling back a read-only transaction are both no-ops.
class Customer(db.Model):
user = db.UserProperty()
class Account(db.Model):
"""An Account has a Customer as its parent."""
address = db.PostalAddressProperty()
balance = db.FloatProperty()
def get_all_accounts():
"""Returns a consistent view of the current user's accounts."""
accounts = []
for customer in Customer.all().filter('user =', users.get_current_user()):
accounts.extend(Account.all().ancestor(customer))
return accounts