Every feature request now gets asked one question first: does this make 3am better or worse? It's reshaped what we build and, just as often, what we don't.

A year ago our roadmap prioritization was mostly "what do the biggest deals need." That produced a product that was impressive in a sales demo and mediocre at 3am, which is the only time it actually matters. We changed the filter, and it changed the roadmap.

The question we started asking in every planning review

Before a feature gets scoped, someone has to answer: does this reduce the time between "something's wrong" and "I know what to do about it" for a person who was asleep ten minutes ago? Features that only helped a dashboard-building admin during business hours got deprioritized, even when they were the ones a sales team wanted for a demo.

What this filter actually killed

A planned "executive summary dashboard" with fancy annual-trend charts got cut entirely — nobody being paged cares about a year-over-year chart. In its place we shipped the alert-to-context pivot: clicking an alert now opens directly to the relevant trace, log, and profile pre-filtered to the alert's time window, instead of a generic dashboard the on-call engineer has to re-navigate from scratch.

from alerts
| where id == "alrt_8f3c"
| expand context(traces, logs, profiles, window=10m)

That single change cut median time-to-first-relevant-signal from 4 minutes to 50 seconds across the design partners who adopted it, based on their own instrumentation of the on-call workflow.

Where the filter has limits

Not everything can be justified by 3am usefulness — billing, SSO, and admin tooling matter for very real reasons, they just get judged by a different bar. The filter is for the signal and incident-response surface specifically, and we're explicit about that so it doesn't become an excuse to starve necessary but boring work.

How we keep the filter honest

Every quarter, product leads present a short review where each shipped incident-response feature is checked against instrumentation we added specifically to measure whether it moved time-to-context, not just whether it shipped on schedule. A couple of features have failed that review after launch — a proposed "AI incident summary" prototype measured no improvement in time-to-first-relevant-signal over the alert-to-context pivot alone, and we shelved further investment in it rather than keep polishing a feature the data said wasn't earning its complexity.

  • Every incident-response feature scoped against "does this help at 3am"
  • Alert-to-context pivot: one click from alert to trace, log, and profile
  • Median time-to-first-relevant-signal cut from 4 minutes to 50 seconds
  • Business-hours-only features judged on a separate, explicit bar