Zero pages, seven days. I still learned more about our system that week than in most of my incidents combined, just by looking on purpose.
Zero pages, seven days. I still learned more about our system that week than in most of my incidents combined, just by looking on purpose.
Quiet Weeks Are Rare Enough to Waste
Most of my on-call learning happens under duress, which is a terrible way to build understanding — you're motivated, but you're also stressed and time-boxed to whatever gets the fire out fastest. A genuinely quiet week gave me the chance to do something I never do during an incident: read the system for curiosity instead of urgency.
What I Went Looking For
I picked one service I was nominally responsible for but had never really understood end to end, and just traced a normal request through it, slowly:
from traces
| where service == "recommendation-engine"
| where trace_id in (sample(1000))
| summarize avg(duration), p95(duration) by span.name
| sort by avg(duration) desc
Nothing here was broken. But I found that one span — a feature-vector lookup — accounted for 60% of average request time on a completely healthy path, which I'd never have noticed without deliberately looking, because 60% of a fast request is still fast in absolute terms. It wasn't an incident. It was capacity I now knew existed, banked for the day traffic doubles and that span becomes the bottleneck that pages someone.
Turning Curiosity Into Institutional Knowledge
I wrote up what I found as a short internal note, not a runbook, just "here's what normal looks like for this service and where the slack is." Three weeks later, a genuine incident hit that exact service, and the engineer on call that week told me my note saved her from having to build a mental model of the request path from scratch while also fighting a live fire. Quiet weeks are the cheapest time to build that kind of knowledge, and I'd been treating them purely as rest instead of also as an opportunity.
You learn a system's failure modes during incidents. You learn its actual shape during the quiet weeks, if you bother to look.
- Use quiet on-call weeks to trace normal requests through services you're responsible for but don't deeply understand.
- Document "what normal looks like" separately from runbooks — it's context, not a procedure.
- Slow spans on a healthy path today are the bottleneck of tomorrow's incident; find them before they page you.
- Treat quiet weeks as an investment opportunity, not just downtime to be grateful for.