English | Site Directory

Google Apps APIs

Google Apps Email Migration API Developer's Guide: Java

The Google Apps Email Migration API allows administrators and users of Google Apps to migrate mail from legacy email systems into their domain's hosted Gmail accounts. Your client application can upload email messages into these accounts using standard Google data API feeds.

Note: This API is only available to Google Apps Premier, Education, and Partner Edition domains, and cannot be used for migration into Google Apps Standard Edition email or Gmail accounts.

Contents

  1. Audience
  2. Getting Started
  3. Authenticating to the Email Migration service
    1. ClientLogin username/password authentication
  4. Migrating email messages
    1. Sample RFC 822 email message
  5. Usage notes and troubleshooting

Audience

This document is intended for programmers who want to write client applications to allow domain administrators or end users to migrate email into Google Apps mailboxes. It provides a series of examples of basic data API interactions using the Java client library, with explanations. After reading this document, you may wish to learn more about the underlying protocol (that is, the actual XML messages sent and the HTTP requests used to transmit them) by reading the protocol section of this developer's guide.

This document assumes that you understand the general ideas behind the Google data APIs.

For Apps Email Migration API reference information, see the reference guide.

Getting Started

You may want to create a Google Apps account for testing purposes. Email Migration uses Google Apps Accounts, so if you already have a test Google Apps account, you're all set. You can create new test accounts with the Google Apps control panel.

To use the Java client library, you must be running Java 1.5. After downloading the client library, you'll find the classes you need to get started in the java/lib/gdata-appsforyourdomain-1.0.jar and java/lib/gdataclient-1.0.jar jar files.

A full working copy of this sample is available in the distribution, under the directory gdata/java/sample/appsforyourdomain/migration. Build and execution instructions are included in the same directory, in the file README.txt. To run the sample, you'll need to modify the following values in gdata/java/build.properties:

  • sample.appsforyourdomain.migration.username is the username you use to log in.
  • sample.appsforyourdomain.migration.password is the password you use to log in.
  • sample.appsforyourdomain.migration.domain is your domain name.

Note: To migrate email to be owned by a different user, you must be an administrator; you can set the owner with the --destination_user command-line option.

To compile the examples in this document into your own code, you'll need to use the following import statements:

import com.google.gdata.client.appsforyourdomain.migration.*;
import com.google.gdata.data.appsforyourdomain.migration.*;
import com.google.gdata.data.batch.*;
import com.google.gdata.util.*;

For more information, see Getting Started with the Java Client Library.

Authenticating to the Email Migration service

To migrate email messages, you must first authenticate using the ClientLogin authentication system. To use this system, you submit the email address and password of the domain user or administrator making the request. The authentication service returns an authentication token that your client can then send to the Email Migration API service along with every subsequent request on behalf of that user.

ClientLogin username/password authentication

To authenticate to the Email Migration service using ClientLogin, create a new MailItemService object, and invoke its setUserCredentials method, as follows:

MailItemService mailItemService = new MailItemService("exampleCo-exampleApp-1");
mailItemService.setUserCredentials(userEmail, password);

Replace userEmail with the email address of the user to authenticate (usually an administrator) and password with that user's password.

For more information about the ClientLogin authentication system, see the Google Account Authentication for Installed Applications documentation.

Migrating email messages

To migrate mail messages into a hosted Gmail account, start by creating a MailItemEntry object for each message you're migrating, as demonstrated in the following example:

MailItemEntry entry = new MailItemEntry();

entry.setRfc822Msg(new Rfc822Msg(rfcTextOfMessage));

entry.addMailItemProperty(MailItemProperty.STARRED);
entry.addMailItemProperty(MailItemProperty.UNREAD);

entry.addLabel(new Label("Friends"));
entry.addLabel(new Label("Event Invitations"));

Set or add the following properties:

  • An Rfc822Msg object, containing the RFC 822 content of the email message. (See below for an example.)
  • Optionally, one or more MailItemProperty objects, corresponding to properties that the server should apply to the message when the message is inserted into Gmail. Each MailItemProperty must be one of the following:
    MailItemProperty.DRAFT
    The message should be marked as a draft when inserted.
    MailItemProperty.INBOX
    The message should appear in the Inbox, regardless of its labels. (By default, a migrated mail message will appear in the Inbox only if it has no labels.)
    MailItemProperty.SENT
    The message should be marked as "Sent Mail" when inserted.
    MailItemProperty.STARRED
    The message should be starred when inserted.
    MailItemProperty.TRASH
    The message should be marked as "Trash" when inserted.
    MailItemProperty.UNREAD
    The message should be marked as unread when inserted. Without this property, a migrated mail message is marked as read.
  • Optionally, one or more Label objects, corresponding to Gmail labels that should be applied to the message when it is inserted. In the example above, we're specifying that the message should have the "Friends" and "Event Invitations" labels in Gmail.

It's possible to post a single mail item entry at a time, but in most cases when you're migrating mail you'll be migrating multiple entries at once. We therefore recommend that you take advantage of the batch insertion feature of the Email Migration API, which greatly reduces the number of HTTP requests you have to make and thereby lowers your client's bandwidth.

To insert your mail items in a batch, construct an array of MailItemEntry objects (named entries in the following example) and initialize each one as shown above. Then construct a feed out of the entries, and "tag" each entry with a batch ID so you can track the status of each one:

MailItemFeed feed = new MailItemFeed();
for (int i = 0; i < entries.length; i++) {
  BatchUtils.setBatchId(entries[i], Integer.toString(i));
  feed.getEntries().add(entries[i]);
}

Then post the feed as follows:

MailItemFeed feed = mailItemService.batch(domain, username, feed);

There are two different groups who can migrate mail:

  • A domain administrator can migrate mail to any mailbox, by specifying a username to be used in the above code.
  • An end user can migrate mail only to their own mailbox, and only if an administrator has enabled end-user access to migration. The username in the above code must be the same as the currently authenticated username.

If your request is successful, the server returns a feed containing the inserted mail item entries. You can check the batch status of each entry in the feed that's returned to determine if the insertion succeeded, as follows:

for (MailItemEntry returnedEntry : feed.getEntries()) {
  BatchStatus status = BatchUtils.getBatchStatus(returnedEntry);
  System.out.println("Entry " + BatchUtils.getBatchId(returnedEntry) + ": " +
      status.getCode() + " " + status.getReason());
}

For more information about batched insertion requests, see the reference documentation on batch processing with Google data APIs.

Sample RFC 822 email message

Below is a sample RFC 822 email message that might appear in an <apps:rfc822Msg> element. For more information about this format, including an explanation of how each field is interpreted, refer to the official RFC 822 standard definition.

Received: by 140.23.6.190 with HTTP; Mon, 16 Jul 2007 10:12:26 -0700 (PDT)
Message-ID: <c8acb6980707161012i5d395392p5a6d8d14a8582613@mail.gmail.com>
Date: Mon, 16 Jul 2007 10:12:26 -0700
From: "Elizabeth Bennet" <bennet@example.com>
To: "Fitzwilliam Darcy" <darcy@example.com>
Subject: Lunch on Monday
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Delivered-To: darcy@example.com

This is the body of the email message.  Would you like to have lunch this week?

The client library ensures that characters in the message aren't interpreted as XML, so if you use the Java client library, you don't need to worry about things like escaping the angle brackets. However, if your message is encoded in a format other than UTF-8, we recommend using Base64 encoding to ensure proper transmission. To do this, supply the Base64-encoded form of the message in the Rfc822Msg constructor as follows:

new Rfc822Msg(base64EncodedMessage, Rfc822Msg.Encoding.BASE64)

There are a variety of online tools and sample programs that demonstrate the process of encoding text in Base64.

Note: Each line in an RFC 822 message should be terminated by a CR+LF (that is, "\r\n") style newline.

Usage notes and troubleshooting

Here are some notes about limits and error situations:

  • You may send a maximum of 32MB of data in each HTTP POST request.
  • The Gmail server may reject malformed messages. In particular, make sure that each message has at least the From:, To:, and Date: header fields required by the RFC822 specification. If a message is rejected as malformed, you'll receive a 400 Bad Request batch status code.
  • If your client exceeds the maximum allowed rate of message uploads per second, the entries that failed contain the batch status code 503 Service Unavailable. If you receive that status code, then record which entries failed (using their batch IDs) and retry the upload. We recommend using an exponential backoff strategy for this process. For example, you might wait thirty seconds and retry the upload; then if your request still returns 503s, wait 60 seconds before trying again, and so on.