My favorites | Sign in
Project Home Wiki Issues
New issue   Search
  Advanced search   Search tips   Subscriptions
Issue 60: Rolling out Django Evolution into an existing project requires rigorous process
2 people starred this issue and may be notified of changes. Back to list
Status:  Accepted

Sign in to add a comment
Project Member Reported by, Jun 19, 2008
Original report text from Sedrik:
Let's assume I'm developing a new feature for my site.

1.  I create a new model in my file.
2.  During development, I realize I need to change the model in one
way or the other.
   a.  I make the changes to the file
   b.  I make evolutions for said changes so that other developers
can get my changes as well.
3.  My feature is ready to go live.
   a.  I deploy my new code to production.
   b.  I run syncdb, which creates the new tables in the DB with the
most recent checked in code.

At this points, Evolution picks up that I have evolutions that haven't
yet been run on the production baseline.  However, all the changes in
these evolutions have already been applied implicitly by syncdb
creating the model with the latest code.

Jun 19, 2008
Project Member #1

There is the non-solution - always include django-evolution in the
startup list of applications. This avoids the problem, but isn't
really a solution.

There is a procedural workaround - that is, enforce at a project
procedure level that if django-evolution is added to a project, the
'add django-evolution' svn update must be applied atomically. This
also works, but relies upon developers having (and adhering to) strong
process, which isn't a reliable solution.

As for a 'real' solution - I'm open to suggestions.

One thought: the current behaviour of syncdb is correct for any
developer with a fresh checkout. However, it's not correct if
django-evolution is added to a project mid-lifecycle. There's no
canonical way to establish if a project is mid-lifecycle, but we could
check to see if one of the common Django contrib projects (say,
contrib.sessions) is in the sync list. If sessions is in the sync
list, then this the first sync, and the existing behavior should be
implemented; if it isn't, the baseline evolutions should be ignored
for later application using evolve.

Jun 19, 2008
How would Evolution get notification of the post_syncdb signal?  Currently, syncdb
and evolve are two separate processes.

What might work however is having evolve --execute be a complete replacement for
syncdb.  It could explicitly call syncdb at the beginning of the process, and when
the post_syncdb signal gets emitted, note all the models that were just installed. 
From there, Evolution is free to ignore any mutations that target these newly
installed models.

One trick here is figuring out how to ignore SQL mutations, as they don't identify
what model they're operating on.

This may be too different from the normal Django process, in that running a common
Django command (syncdb) could severely screw you the baseline.  It may be possible to
hook in to syncdb somehow if Evolution is installed and raise an error if syncdb is
being run from the command line.  Although that means syncdb would have to be
modified to emit a pre_syncdb signal that Evolution can listen for, and promptly
raise at.
Jun 19, 2008
Project Member #3
The idea would be that at the end of syncdb, it is possible that your system could be in a 'evolution required' 
state. We can drop a message to that effect to warn the user. Yes, it would be nice to chain the syncdb into 
calling the actual evolutions, but I could certainly live with a two step process for this case given that this is a 
once-off effort for the developer.

Also - unless I'm mistaken, there isn't a need to identify SQL mutations. Either you're starting a clean 
checkout, or you are updating a project that has added dajngo-Evolution to the app list and added some 
evolutions that need to be run. In the former case, all evolutions are ignored. In the latter case, all evolutions 
need to be executed. 

Jun 27, 2008
It's not just adding evolution to a new project.  It's also when a developer adds a
new model AND evolutions on that that new model before another user runs a syncdb.

Currently, if Dev1 does what's described above, then Dev2 updates and runs a syncdb,
syncdb will create the new table with the most recent columns.  However, evolution
still sees unapplied evolutions.  Running these evolutions is obviously not desired,
because the table is already at the correct state.

For Python mutations, we could hack around this problem because we know specifically
what model the mutation is pointing to, and could ignore any mutation on the newly
created model after getting a post_syncdb signal (provided it could be listened for
within the same process).

However, for SQL mutations, we don't have enough meta data to know what what table
the mutation is on.  The SQL could be parsed/hacked, but that's pretty lame.

Perhaps this could be solved with a similar solution that INSTALLED_APPS is going
for; Each item in the app's evolution.SEQUENCE could be an Evolution object rather
than a string.  Each Evolution object would have enough meta-data, such as Model
name, that would allow the evolution system to identify which evolutions were safe to

This could be used for other interesting behavior eventually, to.
Jul 4, 2008
Why don't you just develop a Mutation AddModel or AddTable, that creates a fresh db
table. It could be a container for several AddField-Mutations, which would be a bit
more code, but who cares - --hint creates it for us. 
That way we can create a new table in exactly the specified condition and all further
mutations will still work, even if a developer starts with a fresh checkout. That way
creating a table is more explicit, too. 

Besides, I really think that syncdb should not be used together with evolutions. They
represent completely different concepts and any attempts to bring them together must
be hackish. Better make django-evolution so comfortable, that you won't need syncdb. 

Sign in to add a comment

Powered by Google Project Hosting