Introduction

Google AppEngine applies a 30 seconds limit to any request, which implies that any long-running processes should be divided by hand. There are many different use cases for this:

• Tasks that send a periodical e-mail to all users.
• Database schema upgrades.
• Delete all expired user sessions from the database.
• Anything that processes an unknown number of persistent entities.

SimpleDS introduces the concept of Background Tasks to transparently break long tasks into smaller ones, using cursors to resume work at the last known point.

NOTE: there is a feature in the AppEngine product roadmap for these tasks. Once GAE starts providing with "Background servers", this feature will probably be deprecated or adapted to work with the new feature, but it will not be dropped overnight.

Configuration

To use Background Tasks you should first configure the TaskServlet in web.xml:

<servlet>
	<servlet-name>tasks</servlet-name>
	<servlet-class>org.simpleds.bg.TaskServlet</servlet-class>
</servlet>
<servlet-mapping>
	<servlet-name>tasks</servlet-name>
	<url-pattern>/tasks</url-pattern>
</servlet-mapping>

<security-constraint>
	<web-resource-collection>
		<url-pattern>/tasks</url-pattern>
	</web-resource-collection>
	<auth-constraint>
		<role-name>admin</role-name>
	</auth-constraint>
</security-constraint>

Adding tasks

Tasks must be configured before being executed. Configuration is done just once, typically at deployment time. For example:

/**
 * Register this class in web.xml
 */
public class TaskInitializer implements javax.servlet.ServletContextListener {

	public void contextInitialized(ServletContextEvent event) {
		TaskServlet.add(
			new DeleteSessionsTask(),
			new MailerTask()
		);
		
	}
	
}

In this example, DeleteSessionsTask is a task included with SimpleDS that will delete all expired user sessions. To invoke it:

TaskOptions to = TaskOptions.Builder.url("/tasks")
		.param(TaskRequest.TASK_PARAM, "delete-sessions");
QueueFactory.getDefaultQueue().add(to);

Tasks can be invoked by any POST request (including queues) or cron trigger.

Task example

In the previous snippet we were adding a MailerTask. This is an example of a task implemented by the application developer:

public class MailerTask extends IterableTask<User> {

	public MailerTask() {
		super("mailer");
	}

	@Override
	protected SimpleQuery createQuery(TaskRequest request) {
		return entityManager.createQuery(User.class);
	}

	@Override
	protected void process(User user, TaskRequest request) {
		// send an email to the provided user
	}


}

Tasks do not have to worry about cursors or limiting the query results. They will execute in configurable batches set by default to 150 results. Any results past that point will be deferred using queues.

Task classes must be:

• Idempotent: It is impossible to know in advance how many times a task will be invoked, so it must be capable of resuming work. Specifically, a previous execution may have failed at any point.
• Thread-safe and immutable: Tasks are stored in a static structure, which means that several threads may be executing tasks at the same time. A task should not be modified by the current request contents.

Checking task stats

Since tasks are mostly a "fire-and-forget" feature, it can be useful to check the task stats to know how did the last execution go. For this, invoke the tasks servlet with any GET request (just point the browser to the mapped URL).

It can also be integrated with the admin console by adding the following snippet to appengine-web.xml:

<admin-console>
	<page name="Taskstats" url="/tasks" />
</admin-console>

After this, redeploy the application and search for a "Taskstats" option in your admin console.