An SLO is a query, a target, and a burn rate, nothing more exotic than that. Build the first one on the metric your users would actually notice.
The hardest part of a first SLO is rarely the tooling, it is picking the right indicator. Pick something a user would notice breaking, not something convenient to measure, and the rest of the setup is close to mechanical.
1. Choose an indicator users would feel
For a request-driven service, the standard choice is the ratio of fast-enough, successful requests to total requests. Resist the temptation to start with something internal like queue depth, that measures the system, not the experience.
from traces
| where service.name == "checkout-api"
| summarize
good = countif(duration < 300ms and status_code < 500),
total = count()
| project sli = good * 1.0 / total2. Set the target from history, not from ambition
Query the last ninety days of the indicator above before choosing a number. A target of 99.9 percent means very little if the service has never sustained better than 99.5 percent, it just guarantees the SLO starts life already burned.
from traces
| where service.name == "checkout-api"
| summarize p50 = countif(duration < 300ms and status_code < 500) * 1.0 / count() by bin(timestamp, 1d)
| summarize percentile(p50, 10)3. Define the error budget and the burn-rate alert
A 99.9 percent monthly target leaves roughly 43 minutes of budget. The useful alert is not "the SLO breached", by the time that fires it is too late to matter, it is a fast-burn alert that catches the budget being consumed at a rate that would exhaust it early.
ALERT checkout_slo_fast_burn
WHEN burn_rate(checkout_api_availability, window=1h) > 14.4
NOTIFY pagerduty:checkout-oncall4. Put the budget on a dashboard, not just in an alert
A team that only hears about the SLO when it pages will treat every burn-rate alert as an emergency. A visible budget-remaining panel, checked in a weekly review, turns the same signal into a planning input instead of a fire drill.
- Base the indicator on something a user would notice, not an internal system metric.
- Set the target from ninety days of real history, not from an aspirational number.
- Alert on burn rate, not on breach, so there is time to act before the budget is gone.
- Review the remaining budget weekly as a planning input, not only during incidents.