The application had been running Rails 4.2.7.1 since before Rails 4.2 went end-of-life in August 2017. That is roughly eight years without a security patch, on Ruby 2.2.10. Nobody had touched the upgrade because the first estimate anyone produced was measured in quarters.
The estimate was 2.5 to 3.5 months for one engineer. The climb from Rails 5.0 to Rails 7.2, including Ruby 2.6 to 3.1, landed in a single day across 27 commits. That gap is not a story about going fast. It is a story about the fact that essentially every number used to size this work was wrong, always in the same direction, and the things that actually consumed the time were never on the risk register.
The Problem
The application is a multi-tenant JSON API with a single-page frontend and a set of background workers. The domain matters very little for what follows. The scale does:
Ruby in app/ and lib/ |
46,211 lines |
| Models | 165 |
| Controllers / actions | 119 / 1,251 |
| Tables | 162 |
| ActiveRecord callbacks | 172, of which 27 veto their operation |
| Migrations | 814, all of them unversioned |
| Sidekiq workers | 16 |
Rails 4.2 to Rails 8 crosses eight major version gates. Each gate has its own removals. Some of them are mechanical, some of them change behaviour silently, and the difference between those two categories turns out to be the only thing worth planning around.
Three things made this tractable, and it is worth being honest that without them the quarter-scale estimate would have been correct:
- The frontend was already extracted. It had its own build toolchain rather than living in a Sprockets tree. This removes most of the Rails 7 asset pipeline gate before you start.
- There was a containerised dev and test environment, including a dedicated
testservice that live-mounts the app and runs RSpec. - There was a green test suite: 1,605 examples, including 67 request specs and JSON-shape characterization of 278 GET routes.
The first two were inherited. The third was not: it was built for this, on a separate branch, before any version number moved. That sequencing is the load-bearing decision in the whole project, so it gets its own section below. An upgrade without a test suite is not an upgrade, it is a rewrite with extra steps.
Prerequisites
- Working knowledge of Rails and ActiveRecord internals, particularly the callback chain
- Familiarity with Bundler dependency resolution and why a gem can be held back by something unrelated to itself
- Docker Compose, enough to run an app and its database as separate services
- An existing test suite you trust, or the willingness to write one first
That last one is not optional. Roughly a third of the work described here was writing characterization coverage before touching a single version number.
Technical Decisions
Incremental in-place, not a rewrite and not a strangler fig
165 models carrying real domain logic and 814 migrations of history are too much to re-derive. A rewrite means reproducing behaviour nobody has written down, in a system where the specification is the code. A strangler-fig rebuild has the same problem plus a coexistence period where two stacks write to the same tables.
Incremental in-place upgrading has one property the others do not: the app is deployable at every single step. Every phase can merge on its own. That matters more than it sounds, because the app is under active development, with new migrations landing most months. A long-lived upgrade branch bleeds merge conflicts against ongoing feature work. Small, independently mergeable PRs are not a style preference here, they are the only way the branch survives.
Build the safety net before writing any upgrade code
An upgrade is a large, diffuse behavioural change. You cannot review it. You can only observe it, which means the observing apparatus has to exist first.
That work is substantial enough that it gets its own section below. The decision worth stating here is the sequencing: no version number moved until the suite could answer the question "did anything change?" The upgrade branch was blocked on the test branch, deliberately, for weeks.
Squash 814 migrations, and pick the dump format by experiment
Rails 5 rejects migrations that inherit from bare ActiveRecord::Migration at class-definition time, not at run time. 814 files, every one of them affected. Versioning them individually is a codemod nobody wants to review.
Squashing into db/structure.sql replaces all 814 with one baseline. The risk is obvious: if the squashed schema does not match production, you have silently baked in drift. So it was verified by full replay. All 814 migrations were run onto an empty scratch database and the result diffed against the working database:
| Check | Result |
|---|---|
| Migrations failing during replay | 0 of 814, single pass |
| Tables | 162 vs 162, identical sets |
| Columns (name, type, nullability) | 1,911 vs 1,911, identical |
The structure.sql versus schema.rb choice was also decided by round-trip rather than by preference: each dump was loaded into an empty database and diffed against the original. schema.rb loses things Rails' DSL cannot express. structure.sql did not.
Move Ruby first, then Rails
This is the counter-intuitive one, and it is the single decision that most changed the shape of the work.
The instinct is to bump Rails as far as it will go on the current Ruby, then move Ruby. Doing it that way required four gem pins to keep Rails 4.2 booting on Ruby 2.2: mime-types, nokogiri, loofah, rails-html-sanitizer. Every one of those pins was an artefact of Ruby 2.2's floor, not of Rails. The logger gem's use of the safe navigation operator broke the boot outright.
Moving Ruby first made all four pins unnecessary. Bundler simply resolved better versions: nokogiri went 1.6.8.1 to 1.13.10, loofah 2.0.3 to 2.25.2, devise 4.2.0 to 4.9.4, rspec-rails 3.5.2 to 4.1.2. What was planned as five sequential Ruby hops plus a separate Rails patch bump landed as one change, with zero version pins.
The general form of this: a gem that looks stuck is often not stuck on itself. It is held back by a floor somewhere else in the graph, and raising the floor releases it for free.
Do not adopt config.load_defaults
This app has never called config.load_defaults. Every framework default introduced since 4.2 is off unless something set it individually. That is real debt, and it is tempting to clear it as part of the upgrade.
It was deliberately left alone, for a reason worth stating plainly: flipping thirty flags together gives you a red suite with no attributable cause. Two of the flags are known-hostile in this codebase. belongs_to_required_by_default touches 218 associations. to_time_preserves_timezone produced 171 order-dependent failures. Bundling those into a version bump means you cannot tell which change broke what.
The pattern that worked instead: one flag at a time, in its own commit, via config/initializers/new_framework_defaults_*.rb. Two of them landed that way during this work and each took an afternoon.
Fortifying the App Before Touching It
The suite went from 1,605 examples to 2,206 across this work, in 153 spec files. That undersells it, because roughly a third of the total effort went here and because the test work found more defects than the upgrade did. None of what follows is Rails-version-specific. It is what you build when you need to be able to tell whether an eight-version climb changed anything.
Seven layers, each answering a different question:
| Layer | Question it answers |
|---|---|
| Foundation | Do factories, DB cleanup, auth helpers and external stubs work at all? |
| Model specs (42) | Do validations, associations, cascades and callbacks behave? |
| Request specs (83) | Does the HTTP surface return the right thing to the right actor? |
| Policy specs (5) | What does each authorization scope actually resolve to, per role? |
| Service / lib / worker specs (17) | Do the helper trees and background jobs hold? |
| Characterization (4 files, 278 routes) | Did any response shape change, anywhere? |
| Cross-cutting sweeps (6) | Does one property hold across every resource? |
The rule that was learned expensively: assert content, not shape
A list endpoint returned an empty collection to every administrator, for as long as anybody could tell. It had a dedicated spec file, a characterization baseline, and an entry in the authenticated smoke sweep.
All three passed. All three asserted form: the status, the JSON keys, the envelope, that the session was accepted. The bug produced perfectly well-formed output containing the wrong data. The action passed the current user's 32-character hex id where a numeric tenant id was expected, and ActiveRecord cast that hex to an integer, which matched nobody about 99% of the time and raised RangeError the other 1%. The 1% is how it was eventually noticed, years late.
What came out of that:
- Seed rows that are distinguishable, then assert identity. That a specific seeded name or id came back, not merely that the response is an array of the right shape. A bare
expect(body['users']).not_to be_emptywould have caught this on the first run instead of never. - Seed a second tenant. It makes "returns nothing" and "returns everything" both fail, which is what converts a silent wrong answer into a loud one.
- Treat an empty collection as suspicious. From the outside, a correct empty result and a broken query are identical.
And the corollary that matters most for anyone building a worklist from a coverage report: line coverage counted that endpoint as covered the whole time, because the line executed. Execution is not verification. A worklist driven by uncovered-line counts will systematically skip exactly the code that runs but is never asserted on.
Assertions must be falsifiable
This pattern was banned outright:
expect([200, 401, 403, 422, 500]).to include(response.status) # NO
It spans success, auth failure, validation failure and crash. It can only fail if the route is missing or Rails itself dies. Seventy-eight of these were removed and replaced with expect(response.status).to eq(N) against the status the endpoint actually returns, plus an assertion on the error key where the body carries one. Every one was verified stable across two independent random seeds before being pinned.
The rule that goes with it: if you cannot predict the status, the spec is not ready. Mark it pending rather than writing something that cannot fail.
Table-driven sweeps, written out rather than derived
Most specs are organised per resource. Six are organised per property, asserting one invariant across every resource at once: no token yields exactly 401; a list returns my rows and only mine; a refused write changes nothing; a rejected create writes nothing; which roles may destroy what; what each authorization scope resolves to.
Adding a resource means adding a row to six tables, not writing six new files.
The important design choice is that those tables are written out by hand rather than derived from the route list or from the policy classes. A derived expectation only restates what the code already says, so it cannot disagree with the code. Three of the divergences found this way were precisely where the derived answer and the real one differed. Where an endpoint genuinely cannot satisfy a property, the row stays in the table carrying a pending: string that names the mechanism, so it fails the moment somebody fixes it.
That last detail generalises: a pending example that names its cause is a finding, not filler. The suite finished with 15 pendings and every one is a documented defect waiting for an owner.
Golden-master characterization
Every GET route in the inventory is requested against a seeded tenant, and its status and JSON shape are compared against a committed baseline. A shape is the body with scalars replaced by type names, arrays collapsed to the merged shape of their elements, and id-like keys collapsed:
{"sites": [{"id": "Number", "name": "String", "hub": "Null"}],
"total_count": "Number"}
So ids, timestamps and row counts are free to differ between runs while structure is pinned. Nullable fields become unions ("Null|String"), and on compare the observed set has to be a subset of the recorded one.
Three decisions made this useful rather than noisy:
- Baselines record what the app does, not what it should. Twenty-two routes are recorded as returning 500 and thirty-three as raising an uncaught exception, with the class and normalised message pinned. A fix and a further regression both show up as a diff, which is the point. A recorded baseline is not a claim that the endpoint works.
- Actor probing. Recording tries each role in turn and keeps the first response that is not 401 or 403, so the baseline captures a real payload wherever one is reachable rather than a wall of authorization failures. Compare mode replays only the recorded actor.
- Re-recording is a review event, not a chore. The diff on the baselines is the API change list for that commit, and a recorded change is only acceptable with a stated reason.
A per-file coverage ratchet, not a global threshold
Each file's worst observed coverage is committed to a JSON file, and a coverage run fails if any file drops more than a point below its floor.
Per-file, deliberately: a single global threshold lets a new shallow spec mask a real regression somewhere else, which is exactly how the number got inflated in the first place. Floors merge by taking the minimum against the existing value rather than overwriting, because a handful of memoised files swing three to five points depending on which spec loads them first, so one snapshot is not a safe floor.
One trap worth naming. SimpleCov's result merging had to be turned off. It silently folded in whatever was measured last, including single-file runs, so a full-suite run could report a previous run's numbers and trip the ratchet with a wall of phantom regressions.
Writing specs that pass on two Rails versions at once
For a period the test branch was on Rails 4.2 and the upgrade branch was on Rails 5. Specs were required to pass on both, which is a genuine differential check that the upgrade preserved behaviour, and which constrains what a spec may assert: observable behaviour only, never mechanism.
The concrete rule that fell out: do not assert framework-generated messages. Assert status codes, error keys, and persistence effects, and where a message must be matched, match one the application produces rather than one from Rails, Pundit or ActiveModel. Framework wording moves between majors.
The instructive case was an error class that moved from ActiveRecord::Type::Integer to ActiveModel::Type::Integer. Because the new error was a subclass of the old one, the class half of raise_error(RangeError, /…/) kept passing and only the message moved. A spec asserting the class alone would have stayed green and told you nothing; a spec asserting the full message would have failed for a reason unrelated to the behaviour under test. Assert the class loosely enough to admit a subclass, and the message loosely enough to survive a rename.
The same principle covers callback halts, which signal with return false on 4.2 and throw :abort on 5.0. Assert the outcome (the row is not written, the error is recorded), never the mechanism, and never read a callback's return value. One existing spec did exactly that, and converting the callback turned a green assertion into an UncaughtThrowError.
What the fortification actually found
This is the part that justifies the effort, and none of it was upgrade damage. These were all live, pre-existing, and invisible to the suite as it stood:
- Cross-tenant writes answering 200. Controllers that skipped their authorization verification and looked rows up by bare integer id with no tenant scope, so any authenticated user could rename or delete another tenant's records. The classic Rails insecure-direct-object-reference shape, found by the tenant-isolation write sweep asserting that a refused write changes nothing.
- A destroy action with no authorization call at all, sitting next to a structurally identical action that had one. Found by the role matrix, not by reading code.
- A bulk-write controller that disabled six model callbacks and saved with
validate: falseprocess-wide while it ran, so anything written concurrently bypassed every model guard the rest of the app relies on. - Uniqueness validations that were not tenant-scoped, so the first tenant to take a value denied it to everyone else and could probe which values were taken.
- Validation branches that were dead code. One compared a
Stringcolumn against anIPAddrobject built two lines earlier;String#eql?is false for any non-String, so the branch could never fire. - A mailer rendering a template that does not exist, which raised only when the message body was materialised, by which point the surrounding
rescuehad already returned. - Guards that raised instead of failing validation, taking the caller down on a missing value rather than returning false.
Nearly all of these are the same underlying shape: code that never runs, or runs and is never checked. Neither is visible in a coverage percentage, and neither is visible to a spec that asserts form.
Implementation
Phase 0: measure everything, believe nothing
Every blocker was audited by running code, not by grepping. The distinction matters more than it sounds.
Take the return false callback halt, which Rails 5 replaces with throw :abort. Grep finds 311 raw return false statements in app/models. That is the number that produces a quarter-long estimate.
Two passes, and the first was wrong. Instrumenting halted_callback_hook during a full suite run found only 4, because runtime instrumentation sees only callbacks that halt on a path the suite actually exercises. A static filter down to methods actually registered as before_save, before_create, before_update, before_destroy or before_validation found 29. Of those, 6 were unreachable because the return sits after a raise. The real number was 27.
311 to 27. Neither grep nor runtime instrumentation alone got there.
The belongs_to audit went the same way, done through ActiveRecord reflection so that :foreign_key and :class_name options resolve correctly:
| Category | Count | Action |
|---|---|---|
Total belongs_to |
218 | grep undercounted this at 208 |
| FK column nullable | 159 | mechanical, add optional: true |
| FK column NOT NULL | 20 | safe to leave required |
| Polymorphic | 10 | explicit handling |
| FK column does not exist | 29 | dead code |
That last row was the find. Twenty-nine associations pointed at foreign key columns that do not exist. Classified against the migration history: 5 had their column dropped by a later migration that left the association behind, 16 never had a column created at all, and 0 were genuine schema gaps.
These were a live bug, not upgrade work. Each returned nil silently rather than raising, because the reader reads the missing FK attribute, gets nil, and short-circuits before ever resolving the class. They would have become hard failures at Rails 5. They were removed ahead of the upgrade as a standalone fix, which is the right way to handle anything you find that is broken today rather than broken by the upgrade.
Phase 1: the Rails 5 gate, where silent failure lives
The 27 callback conversions are mechanical to write and dangerous to get wrong. The shape that matters:
# Twenty of the twenty-seven look like this.
def validate_range
if range_malformed?
errors.add(:range, "is malformed")
return false # Rails 4.2: vetoes the save
end # Rails 5.0: this is just a return value
end
With the flag flipped and the conversion incomplete, that write succeeds with errors populated. The application believes it refused. The database disagrees. Nothing raises, nothing logs, and no test that checks record.errors catches it.
There is also a trap in the conversion itself. throw :abort is only safe for callbacks with no other call site, because outside a catch(:abort) block it raises UncaughtThrowError. One method in this codebase was called directly from a spec, so converting it turned a green assertion into an exception.
An abstract base class for form objects with no backing table broke on Rails 5 schema loading. A Pundit name collision accounted for 7 failures on its own: a controller had defined an action called policies, which shadows the method Pundit calls internally to build its policy cache. Every policy_scope call in that controller rendered "The API has not been configured properly" from inside a before_action.
Total for the Rails 5.0 bump: 18 failures, from four causes, none of which were the framework's headline changes.
Phase 1.5: the deprecation that was easy and the one nobody listed
Rails 5.1 emitted 1,431 deprecation warnings for positional arguments in integration tests. That was the headline blocker on every plan.
It was the easy one. 1,431 warnings resolved to 579 source call sites across 83 files, converted mechanically by a parser that inserts params: and headers: at the right offsets. Zero failures.
The real 5.1 work was HasManyThroughOrderError, which appeared on no blocker list. Rails 5.1 enforces that a has_many :through be declared after the association it goes through, and this codebase writes those chains in reverse:
class Account < ActiveRecord::Base
has_many :items, through: :groups # declared first
has_many :groups # ...goes through this
end
9 violations. They accounted for 10 of the 12 first-run failures. They were found by scanning all 165 models programmatically, not by running the suite and fixing what broke, which would have surfaced them one at a time across many runs.
Phase 1.5 continued: a security patch with no deprecation cycle
Rails 5.2.8.1 is a patch release. It reads YAML-serialized columns through Psych's safe loader instead of the unrestricted one.
public_activity stores symbol keys in a table that gets written on nearly every save. 1,352 examples failed on that one cause.
No deprecation warning precedes this. It cannot, because it arrived through the security-release channel rather than the deprecation cycle. The lesson generalises further than it looks:
Clearing every deprecation warning on version N tells you nothing about whether version N+1 will boot.
This is the single most expensive assumption in any upgrade plan, because it is the one that makes people think a bump is inert and skip the full suite run.
Phase 1.5 continued: when the deprecation message is wrong for your codebase
Rails deprecated delete_all with conditions. The message says:
DEPRECATION WARNING: Passing conditions to delete_all is deprecated.
To achieve the same use where(conditions).delete_all.
In this codebase that is not the same, and following the advice would have been a silent production incident.
A caching concern wraps the class-level delete_all and destroy_all to stamp a Redis key that signals downstream consumers that the data changed. where(...) returns an ActiveRecord::Relation, whose delete_all has no such wrapper. The rewrite leaves the database perfectly correct while every downstream consumer keeps serving stale data. Nothing fails anywhere.
Two deprecation warnings. Both from one shared helper on the write path of every cached model. There is now a spec pinning exactly this asymmetry, asserting that the relation form does not mark the cache dirty, so the difference is documented as behaviour rather than as a comment somebody will delete.
Phase 2: Zeitwerk was small, the asset pipeline was not
The Zeitwerk audit found 390 files whose constant matched their path, 1 mismatch (a file spelling an acronym in full caps, module HTTPHelper in http_helper.rb, where Zeitwerk's default inflector expects HttpHelper), and 1 file defining no constant at all, which was 0 bytes and got deleted. zeitwerk:check then reported "All is good!" across every root.
The one real conflict was invisible to static analysis. A model file defined class Entry, while a file under lib/ reopened the same name as module Entry. The names agreed, so a path-versus-constant pre-scan passed it clean. Only loading it surfaces the class-versus-module mismatch. The lib/ side had zero callers and was removed.
What actually gated Rails 6.0 was the asset pipeline, two versions earlier than any plan had it. sass-rails ~> 5.0 depends on railties < 6, so the Rails 7 cleanup became mandatory at Rails 6. Removing nine Sprockets gems made Sprockets 4 demand an app/assets/config/manifest.js for a pipeline with nothing in it, so require 'rails/all' became explicit requires minus sprockets/railtie.
That exposed a production-only boot failure. A file in lib/ subclasses Uglifier, and Zeitwerk eager-loads lib/ in production but not in development:
# lib/custom_minifier.rb
class CustomMinifier < Uglifier # NameError at boot, in production only
Removing the uglifier gem would have raised NameError at boot in production and nowhere else. Development would have been green. CI would have been green.
Two more that no audit predicted. Classic autoloading resolved unknown constants by requiring their underscored name, so CSV worked without a require 'csv' anywhere. Zeitwerk does not do this. The resulting NameError fired inside a controller action that had disabled model callbacks class-wide and had not yet restored them, so 19 specs failed on one missing require. And ActiveSupport 6.0 does not boot against concurrent-ruby 1.3.5 or later, which dropped an implicit require 'logger'.
Phase 3: 26,427 warnings that were one obsolete gem
Ruby 2.7 emitted 26,427 keyword-argument separation warnings.
Every single one came from json-1.8.6, pulled in transitively by sdoc, a gem retained for rake doc:rails, a task Rails 6.0 had already deleted. Removing sdoc freed json to 2.x and cleared all 26,427. As a bonus, json 1.8.x does not compile on Ruby 3 at all, so the gem generating all the noise was independently the next blocker.
Then the part that matters: Ruby 2.7 warned about none of the four sites that Ruby 3.0 actually made fatal.
| Shape | Direction |
|---|---|
| A session store initializer | hash meant as keywords became positional |
| A test helper with 81 callers | hash meant as positional became keywords |
A save / save! wrapper using (*) |
could not forward keywords to AR's **options |
A mock expectation written .with(k: v) |
states a keyword expectation |
Ruby 2.7's separation warnings are not an audit of what 3.0 makes fatal. Only running on 3.0 establishes that.
The audit question is also subtler than it first appears. It is not "which wrappers use (*)", it is "which use (*) and delegate to something that has keyword parameters". Ruby 3 still converts keywords back to a positional hash when the callee takes none, which is exactly why create and update in the same file were fine while save was not.
Phase 4: 180 failures from one lambda
Rails 7.0 was the only bump in the entire climb that needed no fixes at all. The asset gate had been forced at 6.0, and most 7.0 changes sit behind load_defaults, which this app never adopted. Debt occasionally pays a dividend.
Rails 7.2 produced 180 failures from one cause. A shared transaction-wrapper helper passes the caller's lambda straight through as the transaction block, and Rails 7.2 yields the transaction object to that block. Lambdas enforce arity. Procs do not.
It surfaced as undefined method 'errors' for nil:NilClass across ten different controllers, because they assign inside the lambda and read after it. The real ArgumentError was invisible, swallowed by a bare rescue => e that renders 422 {"message":"failure"}. That pattern appears throughout this codebase, and it is the reason a one-line framework change presented as ten unrelated controller bugs.
The final leg
Rails 7.2 to 8.0.5.1 was a short hop. Ruby moved to 3.3.9, Bundler to 2.4.22, the base image from ubuntu:20.04 plus a third-party PPA to ruby:${RUBY_VERSION}-bullseye, with the Ruby version as a Dockerfile build arg so the next hop is one flag:
$ docker compose build --build-arg RUBY_VERSION=3.4 test
Final suite: 2,206 examples, 0 failures, 15 pending. The 15 pending are generator placeholders that predate all of this.
Verifying it in a browser, which nothing had done
Everything above measures the Rails side. At no point had anything driven the actual UI against the upgraded backend, and a green request-spec suite proves the JSON is right, not that the app works.
A Playwright harness now captures a screenshot of 24 screens as an authenticated admin, in about 57 seconds, recording the console errors and non-2xx responses each screen produced alongside the PNG.
The recording is the part that earns its keep here. Given controllers that wrap whole actions in rescue => e and render a 422, a screen can look entirely correct while its requests are failing behind a modal that has already closed. Screenshots alone cannot see that. The request log can.
Three findings from the first real run were worth the exercise on their own:
- The
webcompose service has no source bind mount, unliketest, so it runs the image'sCOPY . .snapshot. The existing image crash-looped every Puma worker on a routing validation this branch had already fixed. Anyone running that service from a pre-built image was testing the wrong tree entirely. - A database bootstrap script baked into the container image called
rake db:structure:load, which was deprecated at Rails 6.1 and removed at 7.0. Withset -euo pipefail, the script aborted before its remaining steps. Nothing caught it because every environment that already had a populated database volume skipped that code path. It only fails on a genuinely empty volume, which is to say, on a new developer's first day. - Two pre-existing application defects, including an unguarded
nilin a controller that the characterization suite had already recorded as raisingNoMethodErrorand nobody had triaged.
How It All Fits Together
The whole climb, in the order it happened:
Ruby 2.2.10 ─→ 2.6.6 ────────────→ 2.7.5 → 3.0.7 → 3.1.7 → 3.2.9 → 3.3.9
│ │
Rails 4.2.7.1 ─→ 5.0.7.2 → 5.1.7 → 5.2.8.1 → 6.0.6.1 → 6.1.7.10
│
7.0.8.7 → 7.1.5.2 → 7.2.3 → 8.0.5.1
^ Ruby moves FIRST, which dissolves four gem pins
^ 814 migrations already squashed to structure.sql
^ 1,352 failures: YAML safe-load, no warning
^ asset pipeline gates HERE,
not at 7.0 as planned
^ 180 failures,
one lambda
The working loop at each gate was the same four steps, and the order is what makes it work:
- Audit by running code, through reflection or callback chains, never by grepping.
- Fix the blockers while still on version N, where the changes are behaviour-neutral, and verify green. A green run before the bump proves the changes stand on their own.
- Bump, then run the full suite regardless of what the deprecation count said.
- Attribute every remaining warning to a known source. The count dropping is a weak check.
That last step is not pedantry. At the Rails 5.2 gate the warning count fell as expected while hiding three silent-failure sites, and the only thing that surfaced them was noticing that the warnings had changed type rather than just gone down.
Lessons Learned
Count sites, never warnings. Five separate times, a large number collapsed under inspection: 311 became 27, 293 became a single line, 26,427 became one obsolete gem, 1,431 became one codemod, 16,818 became a single change. Warning volume measures how hot a code path is, not how much work there is. A warning emitted inside a loop that runs ten thousand times in your suite is one fix.
The test work found more than the upgrade did, and it was not close. The version climb surfaced framework incompatibilities. The suite built beforehand surfaced cross-tenant write paths, an authorization check missing from a destroy action, validation branches that could never fire, and a mailer that had been raising on a missing template for years. If you are being asked to justify the time spent writing specs before an upgrade, the honest pitch is not "it de-risks the upgrade." It is that an upgrade is the first time in years anybody looks at all the code at once, and that is worth instrumenting for its own sake.
The dangerous changes are the silent ones, and they are never the headline. Everything that showed up in a release-notes summary was mechanical. What actually cost time was a partial callback conversion where writes succeed with errors populated, and a Rails 5.2 dirty-tracking change where attribute_was inside an after_save starts returning the just-written value. That second one has real consequences: scheduled key regeneration silently stops, cached session purging silently stops, and a change-detection helper answers false for every save. Sixteen call sites, nothing raising at any of them.
A run that reports no number has told you nothing. This happened twice, and both times the output read like success. An rspec-support snippet guard captured a mock proxy and killed the summary, so 180 genuine failures printed as no output at all. Separately, a load_defaults probe returned "0 examples, 3 errors outside examples", which looks clean at a glance and means the app did not boot.
Lazy loading hides class-body failures, so probe with eager_load!. Adding config.load_defaults 8.0 boots perfectly fine under rails runner in development, because lazy loading means the serialize calls in model class bodies never execute. Anyone probing that without forcing an eager load concludes, wrongly, that the defaults are safe to adopt.
Estimates were wrong in a consistent direction, and that is diagnostic. All five of the dominant risks on the original register came in smaller than estimated. The keyword-argument sweep was budgeted at 3 to 4 weeks and was four sites. Meanwhile every genuine time sink (the asset pipeline gating two versions early, a security patch with no deprecation cycle, stale environment assumptions, test infrastructure failing in ways that look like application failure) appeared on no list at all. Risk registers are good at enumerating known mechanisms and bad at anticipating interactions, and the fix is not better estimation, it is short feedback loops that surface the unknowns early.
Half of a "fresh environment" bug is not a framework bug. On the first full run against a genuinely empty database, eight failures came from a schema the migration squash had dropped, and two from an IPAddr error-message comparison that Ruby 2.5 had changed. Neither was a Rails problem. Both had been masked for months by everyone reusing an already-populated MySQL volume.
What I would do differently: run the browser harness at the start, not at the end. It took an afternoon to build and immediately surfaced a removed rake task in a migration script, a compose service running a stale image, and two pre-existing defects. Having that from Phase 0 would have caught the db:structure:load removal at the Rails 7 gate rather than three versions later.
What's Next
The remaining gap between this app and a stock Rails 8 one is config.load_defaults, which is still not adopted. Its first blocker is concrete and known: Rails 7.1 sets active_record.default_column_serializer = nil, so a bare serialize :col raises ArgumentError: missing keyword: :coder at class-load time. Seven sites across five models.
The codebase already contains the answer. Of 14 serialize calls, seven already pass coder: JSON, all of them in newer models. So the fix is not a design question, it is applying a convention the newer code already follows. The only real decision is per column, whether JSON is safe for existing rows.
Rails 8.1 is not assumed cheap. The previously documented blocker for it is resolved, but the lesson from Rails 5.2 applies exactly here: clearing the known blocker on version N tells you nothing about N+1 until you actually run it.
References
- Rails Upgrade Guide
- Zeitwerk and the
zeitwerk:checkrake task - Ruby 3.0 keyword argument separation
- Working Effectively with Legacy Code, for the characterization testing approach
- Playwright