Why Alembic Autogenerate Misses Column and Index Changes

You changed a SQLAlchemy model — widened a String(32) to String(128), dropped a server_default, added an index — ran alembic revision --autogenerate, and the new file came back with an empty upgrade() body containing nothing but pass. Or worse, it emitted some of the change and silently ignored the rest, so the revision applies cleanly, the deploy goes green, and production still carries the old column type. Autogenerate did not error; it compared what it was configured to compare and honestly reported no difference. The gap is almost always one of three things: target_metadata is not wired to your real models, the type and default comparisons are switched off by default, or the object you changed lives outside SQLAlchemy’s metadata entirely. This page is the diagnostic for all three, and it assumes the trustworthy-autogenerate discipline set out in the parent Alembic & SQLAlchemy Migrations guide.

Symptom / Error Signatures

There is no exception here — that is what makes the failure dangerous. The tell is a revision file that does not match the change you made. The purest signature is the empty body:

# alembic/versions/9f2a_add_status.py — the whole upgrade() Alembic generated for a real change.
def upgrade() -> None:
    pass


def downgrade() -> None:
    pass

When target_metadata is missing entirely, Alembic prints an explicit hint to stderr instead:

Context impl PostgreSQLImpl.
Generating /app/alembic/versions/9f2a_add_status.py ...  done
FAILED: Can't proceed with --autogenerate option; environment script
env.py does not provide a MetaData object or sequence of objects to the context.

The most destructive signature is the inverted one: you renamed or re-typed a column and autogenerate emitted a drop plus an add, which passes review at a glance but discards data on apply:

# A type change mis-emitted as destructive DDL when compare_type is off in some edge cases,
# or a rename always emitted as drop+add because Alembic cannot see intent.
op.drop_column("users", "status")
op.add_column("users", sa.Column("status", sa.String(128), nullable=True))

If your upgrade() body is empty, partial, or destructive when you expected a clean in-place alter_column, you are on the right page.

Root Cause Analysis

Autogenerate builds two MetaData snapshots and diffs them. One is your metadata, handed to the migration context as target_metadata in env.py; the other is reflected live from the database catalog. Every miss is a case where one snapshot cannot see something the other has, or the comparator is told not to look. The single most common cause is that target_metadata points at an empty or partial MetaData — you set target_metadata = Base.metadata, but the module that actually defines your models was never imported, so Base.metadata is empty when env.py runs and every table looks like it should be dropped, or nothing is detected at all.

The second cause is comparator scope. By default Alembic detects added and removed tables and columns, but it does not compare column types or server defaults unless you set compare_type=True and compare_server_default=True in the context.configure() call. A String(32) to String(128) widening, a server_default="0" change, an Integer to BigInteger promotion — all invisible until those flags are on. The third cause is that the object simply is not in your metadata: a hand-written CHECK constraint, a partial or expression index, an ENUM value, a trigger, or a table Alembic has been told to ignore. If SQLAlchemy does not model it, autogenerate cannot diff it, and the change must be hand-written as an explicit op call.

A fourth, subtler cause is over-filtering. If your project uses an include_object or include_name hook to exclude noise (a spatial index, an audit table managed elsewhere), an over-broad predicate silently swallows legitimate changes too. The categories differ in how you fix them:

Immediate Mitigation

The goal is to get autogenerate to see the change now, then regenerate. Work in a throwaway revision against a scratch database — never edit env.py blind against production.

1. Confirm target_metadata is actually populated. Print the table names Alembic can see before diffing. An empty list means your models were never imported.

# alembic/env.py — add every model module import so Base.metadata is fully populated.
# Context: local/CI authoring only. Importing the models has no side effect on the database.
from app.db import Base
import app.models.user      # noqa: F401 — imported for side effect: registers tables on Base
import app.models.order     # noqa: F401
import app.models.billing   # noqa: F401

target_metadata = Base.metadata
print("autogenerate sees:", sorted(target_metadata.tables))  # temporary sanity check

2. Turn on the type and default comparators in the run_migrations_online() block. These two flags are the difference between a detected and an undetected column change.

# alembic/env.py — inside run_migrations_online(), the context.configure() call.
# Context: authoring config; affects what --autogenerate compares, not how upgrades apply.
context.configure(
    connection=connection,
    target_metadata=target_metadata,
    compare_type=True,             # detect String(32) -> String(128), Integer -> BigInteger
    compare_server_default=True,   # detect added/changed/dropped server_default
)

3. If you filter objects, make the predicate additive, not subtractive. An include_object that returns False too eagerly hides real changes. Exclude by explicit name, and never blanket-skip a whole type.

# alembic/env.py — a safe include_object that skips ONLY a named externally-managed table.
# Context: authoring only. Returning False hides an object from BOTH snapshots — be conservative.
def include_object(obj, name, type_, reflected, compare_to):
    if type_ == "table" and name == "audit_log_external":
        return False  # managed outside this migration history
    return True

context.configure(
    connection=connection,
    target_metadata=target_metadata,
    compare_type=True,
    compare_server_default=True,
    include_object=include_object,
)

4. Regenerate and read the result. Produce the revision against a scratch database and confirm the body is now non-empty and non-destructive.

# Context: local workstation or CI, scratch/dev database only — applies nothing to production.
# Requires the env.py edits above committed first.
alembic revision --autogenerate -m "widen users.status to 128"
grep -E "alter_column|add_column|create_index" alembic/versions/*status*.py \
  || echo "STILL EMPTY: change is outside metadata — hand-write the op call"

5. Hand-write anything metadata cannot model. A CHECK constraint, partial index, or enum value will never appear no matter how the comparators are set — author the op directive yourself.

# alembic/versions/xxxx_add_partial_index.py — hand-written; autogenerate cannot emit this.
# Context: PostgreSQL. CREATE INDEX CONCURRENTLY must run OUTSIDE a transaction.
def upgrade() -> None:
    with op.get_context().autocommit_block():
        op.create_index(
            "idx_users_active_email", "users", ["email"],
            unique=True, postgresql_where=sa.text("status = 'active'"),
            postgresql_concurrently=True,
        )

Permanent Fix / Long-Term Pattern

The durable fix is to treat autogenerate as a checked assistant, not an oracle, and to make its blind spots impossible to ship. Wire compare_type=True and compare_server_default=True permanently in env.py and keep a single module that imports every model so target_metadata is never partial — the Alembic & SQLAlchemy Migrations guide treats this import completeness as a first prerequisite before any revision is generated. Then close the loop with a drift probe in CI: run autogenerate against a clone of production and fail the build if the generated upgrade() is not empty. An empty diff proves the model and the database agree; a non-empty one proves a change was missed or applied out of band, which is the same parity discipline the environment parity strategies page enforces across environments.

# Context: CI gate against a fresh clone of production. A non-empty body means drift — fail the job.
alembic revision --autogenerate -m "drift-probe"
if ! grep -q "^    pass$" alembic/versions/*drift-probe*.py; then
  echo "DRIFT: model and database disagree — a change was missed"; exit 1
fi

For the changes autogenerate detects but cannot sequence safely — a type change that rewrites the table, a NOT NULL addition — detection is only half the job. Once the revision is correct it still has to be split across deploys following the running Alembic migrations with zero downtime sequence, because a correctly-generated ALTER COLUMN ... TYPE on a large table takes an ACCESS EXCLUSIVE lock under the additive expand-and-contract methodology. Detection and safe application are separate problems; this page fixes the first, and a partial revision that also forks the graph is compounded by the multiple-heads recovery that a bad merge triggers.

Verification Checklist

• target_metadata prints every expected table name when env.py runs (temporary print confirms the import).
• compare_type=True and compare_server_default=True are set in the context.configure() call.
• Regenerating the revision now produces a non-empty, non-destructive upgrade() matching the model change.
• Any include_object / include_name filter excludes only named objects, never a whole type.
• Objects outside metadata (CHECK, partial index, enum value, trigger) are hand-written as explicit op calls.
• A CI drift-probe autogenerate against a production clone yields an empty upgrade().

Up one level: Alembic & SQLAlchemy Migrations.

Related

• Fixing Alembic Multiple Heads After a Merge
• Running Alembic Migrations with Zero Downtime
• Resolving Drizzle Schema Drift Detection Errors

Frequently Asked Questions

Why does alembic revision --autogenerate produce an empty migration when I changed a column type? Because Alembic does not compare column types by default. The type comparison is gated behind compare_type=True in the context.configure() call inside env.py; likewise server_default changes are gated behind compare_server_default=True. With both off, widening a String, promoting an Integer to BigInteger, or changing a default all diff as “no change” and the revision comes back empty. Enable both flags, regenerate, and the alter_column appears.

I set the compare flags and it is still empty — what now? The change is almost certainly not represented in your SQLAlchemy metadata, or target_metadata is not wired to your real models. First confirm the models are imported before the diff runs — print sorted(target_metadata.tables) in env.py; an empty or short list means a model module was never imported and its tables are invisible. If the object is a hand-written CHECK constraint, a partial or expression index, an enum value, or a trigger, autogenerate can never see it regardless of the flags, and you must author the op call by hand.

Can include_object cause autogenerate to miss real changes? Yes. An include_object or include_name predicate that returns False too broadly hides the object from the comparison entirely, so a genuine change to it is dropped silently. Keep the predicate conservative: exclude only specific named objects that are managed outside the migration history, and never blanket-skip an entire object type such as all indexes or all constraints. When a change goes missing right after someone added a filter, the filter is the first suspect.