Removing a Django Field with SeparateDatabaseAndState

The legacy_code field on Order has been unused for months, so you deleted it from the model, ran makemigrations, and deployed. The migration dropped the column in forty milliseconds. For the next five minutes, every request served by an old pod failed with column orders_order.legacy_code does not exist, because Django builds its SELECT statements from the model’s full field list — the old code did not use the field, but it still selected it on every query. Removing a field safely takes two releases: first stop the code from referencing the column, then drop the column. Django’s SeparateDatabaseAndState operation lets you express exactly that split, so model state and database schema change at different times. This guide walks through it as part of Django Migrations Without Downtime.

Symptom / Error Signatures

A premature field removal produces errors from the old version of the code during the rollout:

django.db.utils.ProgrammingError: column orders_order.legacy_code does not exist
LINE 1: SELECT "orders_order"."id", ..., "orders_order"."legacy_code" FROM "orders_order" ...

On MySQL the error is ERROR 1054 (42S22): Unknown column 'orders_order.legacy_code' in 'field list'. The same failure can come from other consumers — a reporting service with its own copy of the model, a Celery worker on an older image, an admin export — that the deploy did not replace at the same time.

Root Cause Analysis

Django’s ORM selects every concrete field of a model unless told otherwise (only()/defer()), and it names each column explicitly. So a column that no line of application code reads is still referenced by every query on the model. When a single migration removes the field from Django’s state and drops the column, the database changes the moment the migration commits, while pods built from the previous release keep generating queries that name the column until they are replaced.

SeparateDatabaseAndState resolves this by letting one migration contain two lists of operations: state_operations, which change Django’s in-memory model state (what makemigrations compares against), and database_operations, which run SQL. By putting RemoveField in state_operations with no database operations in release one, you tell Django “the field is gone” without touching the table. In release two, a migration with RunSQL (or a RemoveField in database_operations with an empty state list) drops the column.

Immediate Mitigation

If old pods are failing now because the column was dropped early:

1. Re-add the column as nullable. This is instant on PostgreSQL and MySQL 8.0, and immediately satisfies the old pods’ SELECT lists. The data is gone, but the errors stop.

-- PostgreSQL · migration role · metadata-only ADD COLUMN
-- WARNING: restores the column shape only; values dropped earlier are not recovered.
SET lock_timeout = '2s';
ALTER TABLE orders_order ADD COLUMN IF NOT EXISTS legacy_code varchar(20) NULL;
-- ROLLBACK PATH: ALTER TABLE orders_order DROP COLUMN IF EXISTS legacy_code;  (after rollout completes)

2. Finish the rollout, then remove the column again properly. Once only new pods remain, drop the column in a later release with the procedure below. If the dropped data mattered, recover it from backups as described in recovering data after an irreversible migration.

Permanent Fix / Long-Term Pattern

1. Release N+1: remove the field from the model and from Django’s state only. Delete the field from models.py, run makemigrations, and then edit the generated migration to wrap RemoveField in SeparateDatabaseAndState. If the column is NOT NULL without a database default, make it nullable in the same migration’s database operations, because code in N+1 no longer writes it and inserts would otherwise fail.

# Python · orders/migrations/0050_remove_order_legacy_code_state.py · release N+1
# WARNING: the column remains in the database; do not drop it until release N+2.
from django.db import migrations

class Migration(migrations.Migration):
    dependencies = [("orders", "0049_previous")]
    operations = [
        migrations.SeparateDatabaseAndState(
            state_operations=[migrations.RemoveField("order", "legacy_code")],
            database_operations=[
                migrations.RunSQL(
                    sql='ALTER TABLE "orders_order" ALTER COLUMN "legacy_code" DROP NOT NULL',
                    reverse_sql=migrations.RunSQL.noop,
                ),
            ],
        ),
    ]
# ROLLBACK PATH: reversing restores the field in Django state; the column was never removed.

2. Verify no code references the column. Search the codebase, other services and raw SQL for legacy_code, and confirm on production that queries no longer mention it. pg_stat_statements is a practical check:

-- PostgreSQL · read-only · requires the pg_stat_statements extension
SELECT calls, left(query, 120) AS query
FROM pg_stat_statements
WHERE query ILIKE '%legacy_code%'
ORDER BY calls DESC;

3. Release N+2: drop the column. Use RunSQL so Django’s state (already without the field) is unaffected, and bound the lock.

# Python · orders/migrations/0051_drop_order_legacy_code_column.py · release N+2
# WARNING: irreversible for data; archive the column first if it may be needed.
from django.db import migrations

class Migration(migrations.Migration):
    dependencies = [("orders", "0050_remove_order_legacy_code_state")]
    operations = [
        migrations.RunSQL("SET LOCAL lock_timeout = '2s'", reverse_sql=migrations.RunSQL.noop),
        migrations.RunSQL(
            'ALTER TABLE "orders_order" DROP COLUMN IF EXISTS "legacy_code"',
            reverse_sql='ALTER TABLE "orders_order" ADD COLUMN IF NOT EXISTS "legacy_code" varchar(20) NULL',
        ),
    ]

The same two-release shape applies to deleting a model (DeleteModel in state first, DROP TABLE later) and to renames, which become add-new, dual-write, switch, remove-old — see renaming a column with expand and contract. The general contract-phase rules are in safely removing a NOT NULL column with expand-contract.

Verification Checklist

• The release N+1 migration contains RemoveField only inside state_operations, and sqlmigrate shows no DROP COLUMN.
• The column is nullable (or has a database default) before release N+1 ships, so new inserts succeed.
• pg_stat_statements or query logs show no queries referencing the column after the N+1 rollout.
• The release N+2 migration drops the column with a lock_timeout and IF EXISTS.
• python manage.py makemigrations --check passes after both releases, confirming state and models agree.

Frequently Asked Questions

Why do old pods fail if they never use the field? Because Django selects every concrete field of a model by default and names each column explicitly. A query for Order.objects.get(pk=1) includes legacy_code in its SELECT list, so it fails once the column is gone.

Can I use defer() instead of two releases? Deferring the field in every query path in release N+1 would work, but it is easy to miss one — the admin, serializers, values() calls. Removing the field from the model is simpler and complete, and SeparateDatabaseAndState keeps the column around until nothing selects it.

Does SeparateDatabaseAndState work on MySQL? Yes. It is database-agnostic; it only controls which operations change Django’s state and which run SQL. The column drop in release N+2 uses the MySQL syntax your backend requires.

What if another service reads the same table? Treat it as another “old pod”: the column cannot be dropped until that service has also stopped selecting it. Query statistics on the database are the reliable way to confirm, since you may not control that service’s code.