A runbook nobody can find during an incident might as well not exist. Link it from the alert itself, and structure it so it is useful at 3am.

A wiki full of runbooks written months ago, in a folder nobody on the current on-call rotation remembers exists, provides close to zero value during an actual page. The fix is not writing better runbooks, it is making the existing ones reachable in one click from the alert that would need them.

1. Attach the runbook URL to the alert rule directly

Every Helix alert rule has a runbook_url field, included in the page notification itself. An engineer paged at 3am should never have to search for the right document, it should already be one link away in the notification.

ALERT checkout_api_p95_latency
FROM (...)
WHEN p95(duration) > 900ms FOR 3 consecutive windows
NOTIFY pagerduty:checkout-oncall
RUNBOOK https://runbooks.acme.dev/checkout-api-latency

2. Structure it for a stressed reader, not a calm one

The first thing a runbook should answer is not background context, it is what to check first. Lead with the diagnostic query, not a paragraph explaining what the service does.

## First check

from traces
| where service.name == "checkout-api"
| where duration > 900ms
| summarize count() by route, downstream.service
| sort by count() desc

## If downstream.service == "payments-api", see payments-api-timeout runbook
## If no clear downstream pattern, check deploy history next

3. Link runbooks to each other, not just to the alert

Incidents rarely stay within the boundary of a single alert. A runbook that ends with a pointer to the two or three most likely follow-on runbooks saves the second lookup that would otherwise happen mid-incident under pressure.

4. Treat a stale runbook as a bug, not as documentation debt

A runbook that references a dashboard which no longer exists, or a query field that got renamed in a schema change, is actively harmful, it wastes the exact minutes an on-call engineer cannot spare. Review runbooks whenever the alert or dashboard they reference changes, not on a separate documentation cadence that nobody prioritizes under deadline pressure.

  • Attach the runbook URL directly to the alert rule so it rides along in the page itself.
  • Lead with the first diagnostic query, not background explanation, for a reader under pressure.
  • Cross-link related runbooks so a second lookup is not needed mid-incident.
  • Update a runbook the moment the alert or dashboard it references changes.