The outage was caused by a decision leadership made against an engineer's explicit warning. Writing that postmortem honestly tested our blameless culture more than any other incident had.

An engineer flagged, in writing, three weeks before a migration, that the rollback plan hadn't been tested. Leadership — under deadline pressure — approved the migration anyway. It failed, and the rollback took six hours instead of the planned twenty minutes.

Why this postmortem was different

Every retro up to this point had involved peers investigating peers. This one required an engineer's manager's manager to sit in a room and have the timeline show, in black and white, that a documented warning had been overridden by a leadership decision. Nobody wanted to write the section that made that explicit. The draft's first version quietly left it out.

"It would have been so easy to write 'the rollback plan was insufficiently tested' and leave it at the level of the system. That's true. It's also not the whole truth, and I'd already decided the whole truth mattered more than my own comfort."

What we decided, deliberately

We kept the timeline factual and specific, including the leadership decision, without editorializing or assigning personal blame — the same standard we hold every other postmortem to. The hard part wasn't the writing; it was leadership agreeing, out loud, in the doc, that pressure to hit a deadline had overridden a documented risk.

## Timeline excerpt (as published, unedited)
2026-04-18 14:02 — Eng flags in #migration-planning: rollback plan
  untested in staging; recommends 1-week delay to validate.
2026-04-18 16:40 — Leadership approves proceeding as scheduled,
  citing Q2 commitment date; rollback validation deferred to
  "if needed, live."
2026-04-25 09:14 — Migration begins.
2026-04-25 09:51 — Migration fails; rollback attempted; rollback
  plan fails at step 3 due to untested assumption about schema lock.
2026-04-25 15:58 — Service restored via manual intervention.

What changed because we published it honestly

We added a rule: any risk flagged in writing by an engineer and knowingly overridden by a decision-maker must be named as such in any resulting postmortem, with the decision-maker's role, not name, attached. It's uncomfortable by design. Since adding it, we've seen a measurable rise in decision-makers asking for the untested risk to be resolved before proceeding, rather than accepting it — because now they know it will be visible if it goes wrong.

  • Blameless does not mean erasing an accurate account of a leadership decision that overrode a documented warning.
  • Making overridden risks explicitly visible in postmortems changes future leadership behavior more than any policy memo.
  • The hardest postmortems to write honestly are usually the most valuable ones to get right.