Using South for schema and data migrations in Django

Check out the new site at https://rkblog.dev.

18 June 2012 Comments

South is a schema and data migrations for Django. In an easy way you may update your database tables after you change your models. It also keeps all migrations in the codebase allowing to migrate backward if needed. In this article I'll show some standard South usage cases.

Installing and configuring South

The quickest way to install south is to use pipL

pip install South

Next we have to add south to INSTALLED_APPS in your Django project. Then do syncdb and south will be ready. syncdb will create south table in which it will store information about current migration status of each app.

Basic usage

Without south you would use syncdb to create table of your new models. When using south you create migrations to do all the database work, including the initial migration creating tables:

python manage.py schemamigration APPLICATION_NAME --initial

In the application folder a "migrations" subfolder will be created. In it you will find the initial migration - 0001_initial.py. The migration was created, but not executed. To execute migrations you have to use:

python manage.py migrate APPLICATION_NAME

If we would have such simple model:

from django.db import models

class MyModel(models.Model):
    name = models.CharField(max_length=100)

Then the initial migration would create the table. Now if we add a new field we can create a new migration (south will notice the change):

class MyModel(models.Model):
    name = models.CharField(max_length=100)
    description = models.TextField(blank=True)

Instead of --initial we use --auto for for all new migrations (after the first one for given application):

python manage.py schemamigration APPLICATION_NAME --auto

In this example south will do everything on its own and we can just execute the migration.

Now lets try a field that can't be empty:

class MyModel(models.Model):
    name = models.CharField(max_length=100)
    description = models.TextField(blank=True)
    number = models.IntegerField()

When you try creating a migration south will ask:

 ? The field 'MyModel.number' does not have a default specified, yet is NOT NULL.
 ? Since you are adding this field, you MUST specify a default
 ? value to use for existing rows. Would you like to:
 ?  1. Quit now, and add a default to the field in models.py
 ?  2. Specify a one-off value to use for existing columns now

You can choose 1 and quit (and then for example add default to the field) or use 2 and provide a default value for the migration (value used only for entries existing at the time of migration).

Data migrations

What if we would want to add a unique field to a table with existing entries?:

slug = models.SlugField(unique=True)

You can't make a unique column with bunch of existing entries having a Null value. At start we have to make a field that allows null values (they won't count for the unique constraint):

slug = models.SlugField(unique=True, null=True)

The plan is simple - we create such column (migration), then we do a data migration (we make unique values for the existing entries) and in the end we remove the null with another schema migration.

To make an empty data migration use this:

python manage.py datamigration APPLICATION_NAME MIGRATION_NAME

You will get a new file in the "migrations" folder. The core part will look like so:

# -*- coding: utf-8 -*-
import datetime
from south.db import db
from south.v2 import DataMigration
from django.db import models

class Migration(DataMigration):

    def forwards(self, orm):
        # Note: Remember to use orm['appname.ModelName'] rather than "from appname.models..."

    def backwards(self, orm):
        "Write your backwards methods here."

The forwards method is used to make the migration. backwards is used to do a backward migration (if it's something complex/non reversible you can disable it). The key element is to write the migration.

Here is a simple example providing unique values for the slug column (we could use Django slugify helper and some unique value generators, but this is just an example):

    def forwards(self, orm):
        # Note: Remember to use orm['appname.ModelName'] rather than "from appname.models..."
        for i in orm['first_app.MyModel'].objects.all():
            i.slug = i.id
            i.save()

You execute data migrations just like schema migrations.

Backward migrations

South allows backward migrations. You can migrate to an older migration by specifying its number:

python manage.py migrate APPLICATION_NAME NUMER

like:

python manage.py migrate first_app 0005

To go back before any migration use this:

python manage.py migrate APPLICATION_NAME zero

Dependencies for migrations

In some cases when you have migrations covering model/database relations there may be a problem with the order in which migrations are executed. The migration with the relation may try to execute before the related table will be created by another migration. To apply correct order we can use "depends_on" in the migration class:

depends_on = (
        ("APPLICATION_NAME", "NAZWA_MIGRACJI"),
    )

Such problems may show up during tests or build processes where everything is created from scratch.

You will find more in the south documentation.

RkBlog

Django web framework tutorials, 18 June 2012

Check out the new site at https://rkblog.dev.