Our first SLI was 100% green while users were filing complaints. The gap taught us to stop measuring what was easy and start measuring what mattered.

We had a beautiful 99.97% availability SLI and a support queue full of angry users during the exact window it claimed was healthy. The SLI was measuring the wrong thing.

Server success is not the same as user success

Our original SLI counted a request as "good" if the server returned a 2xx status code. It didn't catch a client-side bug where the checkout button silently failed to submit after a successful API call — technically a server success, functionally a broken checkout. We rewrote the SLI to include the full user-facing outcome, not just the backend response.

from traces
| where service == "checkout-frontend" and span.name == "checkout_submit"
| extend good = status_code < 300 and duration < 800ms and outcome == "order_confirmed"
| summarize good_ratio = avg(good) by bin(_time, 5m)

Validate against a channel users actually use

We cross-checked our new SLI against support ticket volume for the same period, rather than trusting the new definition just because it felt more thorough. When the SLI dipped, ticket volume should rise within the hour; when it didn't correlate, that told us the SLI still had a gap, and we iterated twice more before the two lined up consistently.

Accept that some SLIs need synthetic checks, not just real traffic

For low-traffic but high-importance paths — password reset, for instance — real traffic volume is too sparse to compute a meaningful rolling SLI. We layered in synthetic checks that exercise the flow every few minutes specifically to keep the SLI statistically meaningful even during quiet hours.

Let support flag SLI gaps, not just confirm them

Beyond validating the SLI after the fact, we now give the support team a lightweight way to tag a ticket as "the dashboards said we were fine" directly, which routes straight to the SLI owner instead of getting lost in a general feedback backlog. That direct channel caught a second gap — a subtly broken retry path — faster than any scheduled review would have.

  • Define "good" as the full user-facing outcome, not just the backend response code.
  • Validate a new SLI against an independent signal like support ticket volume.
  • Iterate the SLI definition until it actually correlates with user pain, not once.
  • Add synthetic checks for low-traffic, high-importance paths.
  • Give support a direct channel to flag when the SLI disagrees with what users felt.