Removing Columns Safely with ignored_columns

The legacy_code column on orders was dead weight, so the pull request removed every reference and added remove_column :orders, :legacy_code, :string. Tests passed, the migration ran in a few milliseconds, and then the error tracker lit up: PG::UndefinedColumn: ERROR: column orders.legacy_code does not exist, thousands of times, from web workers and Sidekiq processes that had not been restarted yet. None of that code referred to legacy_code either. Active Record did, implicitly — each process loaded the table’s column list when it booted and used it for every INSERT and for queries that select all columns. The fix is a two-release removal with ignored_columns, which teaches Active Record to forget the column before the database does. It is one of the core practices in Rails Active Record Migrations.

Symptom / Error Signatures

A column removed too early produces errors only from processes started before the migration:

ActiveRecord::StatementInvalid: PG::UndefinedColumn: ERROR:  column "legacy_code" of relation "orders" does not exist
LINE 1: INSERT INTO "orders" ("customer_id", "total", "legacy_code", "created_at", ...

On MySQL the equivalent is Mysql2::Error: Unknown column 'legacy_code' in 'field list'. The errors stop once every web and job process has restarted — which, for long-running background workers or processes that deploy on a different schedule, can take much longer than the web rollout.

Root Cause Analysis

Active Record builds model attributes from the database: the first time a model class needs its columns, it queries the schema and caches the result for the life of the process. That cached list drives which columns appear in INSERT statements (Rails inserts every attribute, including defaults and nil values, unless partial inserts are enabled) and in queries that load whole records. A column that no application code mentions is still part of the model, so a process with the old cache will reference it until it restarts.

self.ignored_columns removes named columns from the model’s attribute set even though they still exist in the table. After a release that adds a column to ignored_columns, every freshly started process omits it from inserts and from SELECT lists. Once no process with the old cache remains, the column can be dropped without anyone noticing.

Immediate Mitigation

If the column is already gone and old processes are failing:

1. Re-add the column as nullable. This is instant and makes the old processes’ statements valid again; values that were in the column are gone, but errors stop immediately.

-- PostgreSQL · migration role · metadata-only
-- WARNING: restores the column shape only; dropped data is not recovered.
SET lock_timeout = '2s';
ALTER TABLE orders ADD COLUMN IF NOT EXISTS legacy_code varchar NULL;
-- ROLLBACK PATH: ALTER TABLE orders DROP COLUMN IF EXISTS legacy_code;  (in release N+1)

2. Restart or finish rolling out every process. Include background workers and scheduled jobs, which often run older code for longer than web processes.

3. Recover the data if it mattered. Restore the column’s values from a backup or point-in-time recovery into a scratch database and copy them back, as described in recovering data after an irreversible migration.

Permanent Fix / Long-Term Pattern

Release N: ignore the column. Remove every reference in code and add the column to ignored_columns. If the column is NOT NULL without a default, make it nullable in the same release, because inserts from the new code will omit it.

# Ruby · app/models/order.rb · release N
# WARNING: use += so columns ignored by concerns or parent classes are preserved.
class Order < ApplicationRecord
  self.ignored_columns += ["legacy_code"]
end

# Ruby · db/migrate/20260918110000_make_orders_legacy_code_nullable.rb · release N · PostgreSQL
class MakeOrdersLegacyCodeNullable < ActiveRecord::Migration[7.1]
  def change
    change_column_null :orders, :legacy_code, true   # dropping NOT NULL does not scan the table
  end
end
# ROLLBACK PATH: change_column_null :orders, :legacy_code, false (only if no NULLs were written)

Verify nothing reads it. Before release N+1, confirm on production that no query mentions the column:

-- PostgreSQL · read-only · requires pg_stat_statements
SELECT calls, left(query, 100) FROM pg_stat_statements
WHERE query ILIKE '%legacy_code%' ORDER BY calls DESC;
-- also check dependent views:
SELECT DISTINCT dependent_view.relname
FROM pg_depend d
JOIN pg_rewrite r ON r.oid = d.objid
JOIN pg_class dependent_view ON dependent_view.oid = r.ev_class
JOIN pg_attribute a ON a.attrelid = d.refobjid AND a.attnum = d.refobjsubid
WHERE d.refobjid = 'orders'::regclass AND a.attname = 'legacy_code';

Release N+1: drop it. Use remove_column with the type so the migration stays reversible, and let strong_migrations apply its lock timeout. Remove the ignored_columns entry in the same release.

# Ruby · db/migrate/20260925090000_remove_legacy_code_from_orders.rb · release N+1
# WARNING: irreversible for data; archive values first if they may be needed.
class RemoveLegacyCodeFromOrders < ActiveRecord::Migration[7.1]
  def change
    safety_assured { remove_column :orders, :legacy_code, :string }  # ignored since release N
  end
end

The safety_assured here is legitimate and self-documenting: the comment records that the column has been ignored for a full release. The same two-release structure applies to renames, which become add, dual-write, switch reads, ignore and drop — the expand-and-contract rename — and to dropping whole tables, where the model is removed first and the table dropped later.

Verification Checklist

• The model lists the column in ignored_columns (using +=), and release N is fully rolled out, including background workers.
• The column is nullable or has a database default before release N ships.
• pg_stat_statements (or query logs) show no queries referencing the column after release N.
• No database views, triggers or other services depend on the column.
• Release N+1 drops the column and removes the ignored_columns entry together.

Frequently Asked Questions

Why does Rails reference a column my code never uses? Active Record caches the table’s columns when a model loads and treats each as an attribute. Inserts include all attributes, and loading records selects all columns, so an unused column is still part of every statement until the process restarts with a model that ignores it.

Is ignored_columns enough if other services read the table? No. It only affects processes running your Rails code. Views, reporting queries and other services must be checked separately — pg_stat_statements and pg_depend are the practical tools.

Can I remove the column and add ignored_columns in the same release? No. The processes that fail are the ones started before the release, and they do not have the ignored_columns change. The column must outlive every such process, which means dropping it in a later release.

Does this apply to MySQL too? Yes. Active Record’s column caching is the same on MySQL, and so is the failure. On MySQL 8.0, dropping a column can often use ALGORITHM=INSTANT (8.0.29+), but the two-release sequence is still required for application safety.