Introduction

DMigrate helps you migrate your database by creating a script which creates a new database in the required structure, and copies all the data from your existing database to your new database. The automatically created script only copies data. To make any further modifications, you edit the script. You then run the script, and if everything is fine, you say goodbye to your old database!

Since Django's ORM allows only one database connection at a time, SQLAlchemy is used to access the data in the existing database, and Django's ORM is used to create the tables in the new database and to populate them.

Installation

Installation is very simple. To install, just:

1. Install SQLAlchemy.
2. Download dmigrate.py and put it somewhere in your path.

Usage

In one line, DMigrate usage looks like this:

dmigrate.py [--rename RENAME_LIST] [SOURCE_DIR] DEST_DIR MIGRATION_SCRIPT

SOURCE_DIR is the directory of the source project, which defines the models of the existing database.

DEST_DIR is the directory of the destination project, which defines the models of the new database.

MIGRATION_SCRIPT is the name of the script that will be created. It's usually migrate.py.

RENAME_LIST should be used if you rename models. See example 4 below.

Evolving a Database Structure

To evolve a database structure, you need two copies of your Django project: One which works with the current database (the source project), and one which should work with the new database (the destination project). The destination project has modified model.py files. It is suggested that the destination project's settings will be changed to use the newly created database. In this way you can immediately check if the migration results are satisfactory.

This means that in order to migrate your database you should do the following steps:

1. Copy the current, working project, to another directory. Note that if the other directory has a different name than that of the current directory, your Django project might not work from there. It should not disturb DMigrate, though.
2. Edit settings.py to change the database settings to those of the new database. You may need to create an empty database using your database administration tool.
3. Edit models.py to make the changes in database structure you wish.
4. Run dmigrate.py <source project dir> <destination project dir> migrate.py to create the migrate.py script.
5. Edit migrate.py and change it as you like. Look for TODO comments which mark added or removed fields that need your attention. See below for examples.
6. Run ./migrate.py.
7. If you are not satisfied with the results, return to step 3 if you want to change your models, or to step 5 if you don't.

Example 1: Renaming a Field

Say you have a model like this:

class User(models.Model):
    firstname = models.CharField(maxlength=30)
    lastname = models.CharField(maxlength=30)

and you want to change it to a model like this:

class User(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)

Edit the migrate.py file, and search for TODO comments. You'll find a comment like this:

        # TODO:
        # Removed fields: firstname, lastname
        # Added fields: first_name, last_name

Replace it with this code:

        d.first_name = s.firstname
        d.last_name = s.lastname

Now run migrate.py, and you are done!

Example 2: A slightly more complex transformation

Say you have a model like this:

class User(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)

and you want to change it to a model like this:

class User(models.Model):
    full_name = models.CharField(maxlength=60)

Edit the migrate.py file, and search for TODO comments. You'll find a comment like this:

        # TODO:
        # Removed fields: first_name, last_name
        # Added fields: full_name

Replace it with this code:

        d.full_name = s.first_name + ' ' + s.last_name

Now run migrate.py, and you are done!

Example 3: Creating a New Model

Say you have a model like this:

class User(models.Model):
    full_name = models.CharField(maxlength=60)

You now want each user to have a city. So your new models.py looks like this:

class City(models.Model):
    name = models.CharField(maxlength=30)

class User(models.Model):
    full_name = models.CharField(maxlength=60)
    city = models.ForeignKey(City)

For this example, we'll assume that all current users live in Jerusalem.

In the migrate.py file you'll find two TODO comments. One will look like this:

    # TODO:
    # Added models: dst.myapp.City

Replace it with this code:

    jerusalem = dst.myapp.City()
    jerusalem.name = 'Jerusalem'
    jerusalem.save()

The second TODO comment will look like this:

        # TODO:
        # Added fields: city

Replace it with this code:

        d.city = jerusalem

Run migrate.py, and you're finished!

Example 4: Renaming a model

Say you want to change the model name User to Person. If you'll just run DMigrate, it will report about a removed model named User, and about an added model named Person, and you'll have to copy all the fields by yourself. Instead, you should use the --rename option. Just run

dmigrate.py --rename myapp.User-myapp.Person <source dir> <dest dir>

and everything should be Ok. If all you did was change the model name, you don't even have to edit migrate.py!

Migrating from One Database Engine to Another

This is very easy. Run

dmigrate.py <project dir> migrate.py

This will create a file called migrate.py. Edit it, and change the destination database settings. Run ./migrate.py, and a new database will be created, with all the data copied from the old database.

Conclusion

Have fun! If you encounter any problems, or just want to say that you find DMigrate useful, please drop me a message!

Noam Raphael

noamraph at gmail dot com