Choosing a Migration Tool for Go Services

A Go service is about to get its first real schema, or has outgrown a hand-rolled schema.sql executed at startup, and the team has to pick a migration tool. The three names that come up are golang-migrate, goose and Atlas. All three can apply versioned SQL files to PostgreSQL and MySQL; the differences show up in the situations that matter for zero-downtime deploys — what happens when a migration fails halfway, how a CREATE INDEX CONCURRENTLY is run outside a transaction, whether migrations run from the binary at startup or from a separate CI step, and how much the tool helps you catch unsafe DDL. This guide compares them on exactly those points and recommends a setup. It is part of Migration Tool Comparison.

Symptom / Error Signatures

These are the problems that usually prompt the choice, or a switch:

• golang-migrate: error: Dirty database version 17. Fix and force version. — a migration failed partway and the tool refuses to continue until someone runs migrate force.
• ERROR: CREATE INDEX CONCURRENTLY cannot run inside a transaction block because the tool wrapped the file in a transaction, or the opposite — a multi-statement file ran without a transaction and left partial changes.
• Several replicas of the service start at once, each tries to run migrations on startup, and deploys race or time out waiting on the tool’s advisory lock.
• A DROP COLUMN shipped while the previous release was still reading the column, because nothing in the toolchain flagged it.

Root Cause Analysis

golang-migrate is the minimal option: numbered up and down SQL files, a schema_migrations table with a version and a dirty boolean, and drivers for many databases. It sets dirty = true before running a migration and clears it after; if the process fails in between, the flag stays set and every subsequent run refuses to proceed. That is deliberately conservative — the tool cannot know how far the failed file got — but it means every failure needs a human to inspect the schema, finish or undo the partial change, and force the version. Transaction handling depends on the driver and the file contents; for PostgreSQL, statements run as written, so you control transactions with explicit BEGIN/COMMIT.

goose uses annotated SQL files (-- +goose Up, -- +goose Down) or Go functions, and records applied versions in goose_db_version. Each SQL migration runs in a transaction by default, and a -- +goose NO TRANSACTION annotation opts a single file out — exactly what concurrent index builds need. Because a version is only recorded after success, a failed transactional migration simply rolls back and can be rerun. Go-function migrations are useful for data migrations that need application logic, though they deserve the same batching care as any backfill.

Atlas offers versioned migrations with an atlas.sum integrity file, per-file transaction directives, and — its differentiator — atlas migrate lint, which analyses new migrations for destructive changes, backward-incompatible changes and blocking operations. It also supports generating migrations from a desired state, covered in combining declarative diffs with versioned migration files.

Immediate Mitigation

1. If golang-migrate reports a dirty version, inspect before forcing. Determine whether the failed migration’s statements applied; finish or revert them manually; then force the version that matches reality.

# Shell · operator workstation · migration role credentials
# WARNING: force only records a version; it runs no SQL. Forcing the wrong version hides a broken schema.
migrate -path ./migrations -database "$DATABASE_URL" version    # e.g. "17 (dirty)"
psql "$DATABASE_URL" -c '\d orders'                               # check what 17 actually changed
migrate -path ./migrations -database "$DATABASE_URL" force 16     # if 17 fully rolled back
migrate -path ./migrations -database "$DATABASE_URL" up

2. Move migrations out of service startup. Run them as a single CI/CD step before the new version rolls out, so replicas never race and a slow migration never blocks health checks. The ordering rules are in Migration Pipeline Gating.

3. Isolate non-transactional statements. In goose, one annotation per file does it:

-- PostgreSQL · goose migration 20260918120000_idx_orders_region.sql
-- +goose NO TRANSACTION
-- WARNING: runs outside a transaction; keep this statement alone in the file.
-- +goose Up
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_orders_region ON orders (region);
-- +goose Down
DROP INDEX CONCURRENTLY IF EXISTS idx_orders_region;

4. Make every file idempotent. Guards such as IF NOT EXISTS turn most failure recoveries into a simple rerun, whichever tool you use; see how to write idempotent SQL scripts for safe deploys.

Permanent Fix / Long-Term Pattern

A good default for a new Go service is goose or Atlas with SQL files embedded in the binary via embed.FS, executed by a dedicated migrate subcommand that CI runs once per deploy — never on every replica’s startup. Choose goose when you want the smallest dependency and occasionally need Go-code migrations; choose Atlas when you want linting of new migrations in pull requests or a desired-state workflow. golang-migrate remains a fine choice for teams that already use it, provided every file is idempotent and the dirty-state runbook above is written down.

// Go · cmd/app/migrate.go · runs as `app migrate` in the deploy pipeline, not at server startup
// WARNING: set lock_timeout for the migration session; goose applies files in version order.
package main

import (
	"database/sql"
	"embed"

	"github.com/pressly/goose/v3"
)

//go:embed migrations/*.sql
var migrations embed.FS

func runMigrations(db *sql.DB) error {
	if _, err := db.Exec("SET lock_timeout = '2s'"); err != nil {
		return err
	}
	goose.SetBaseFS(migrations)
	if err := goose.SetDialect("postgres"); err != nil {
		return err
	}
	return goose.Up(db, "migrations")
}
// ROLLBACK PATH: goose.Down(db, "migrations") reverts the newest migration using its Down section.

Note that SET lock_timeout on a *sql.DB applies only to whichever pooled connection runs it; for a guarantee, set it on the migration role (ALTER ROLE migrator SET lock_timeout = '2s') or in the connection string’s options parameter. Add linting regardless of the tool: Atlas’s analyzers, or a generic linter as described in Migration Linting & Static Analysis.

Verification Checklist

• Migrations run from one pipeline step before rollout; no replica runs them at startup.
• Every migration containing CONCURRENTLY is marked non-transactional for the chosen tool and contains nothing else.
• The migration role has a default lock_timeout, so the setting does not depend on which pooled connection runs a file.
• A written runbook covers failed-migration recovery for the chosen tool, including golang-migrate’s force.
• New migrations are linted in pull requests for destructive and blocking operations.

Frequently Asked Questions

Why does golang-migrate mark the database dirty? It sets the flag before running a migration and clears it after success. If the run fails, the tool cannot know which statements applied, so it refuses to continue until an operator inspects the schema and uses force to record the correct version.

Can goose run Go code as a migration? Yes. goose supports migrations written as Go functions registered with the library, which is useful for data transformations that need application logic. Treat them like any backfill: batch the work and keep each transaction short.

Should Go services run migrations on startup? Not in production. With several replicas starting at once, migrations race or serialise behind the tool’s lock, slow migrations delay health checks, and a failure crash-loops every replica. Run migrations once as a separate deploy step before the rollout.

Does Atlas require a desired-state schema file? No. Atlas works as a plain versioned migration tool with SQL files and an integrity sum; the desired-state features are optional. Many teams adopt it for migrate lint alone.