Raw 15-second resolution can't survive 13 months of retention economically. Here's how rollups are built, and how a query picks which one to read.

Storing every 15-second metric point at full resolution for 13 months would be prohibitively expensive for most workloads. Helix builds progressively coarser rollups — 1 minute, 1 hour, 1 day — and the planner decides which one a query actually needs.

How rollups are built

A background compaction job aggregates raw chunks into 1-minute rollups (count, sum, min, max, and a handful of quantile estimates via t-digest) six hours after ingest, once the tail-based sampling and late-arrival windows have closed. 1-hour rollups are built from 1-minute rollups a day later, and 1-day rollups from those a week later — each pass roughly 15-20x smaller than its source.

Resolution selection at query time

The planner picks a resolution based on the requested time range and the visualization's effective pixel width, aiming for roughly 2-4 raw points per rendered pixel — enough to preserve the shape without transferring resolution the query can't display.

from metrics
| where name == 'http.server.duration' and range == 90d
| summarize p95(value) by bin(timestamp, 1h)
# planner selects the 1-hour rollup directly, skips raw chunks entirely

Had that same query asked for `bin(timestamp, 15s)` over 90 days, the planner would reject it outright above a configurable row-count ceiling (default 2 million points) rather than silently scanning raw data for three months — a query that would otherwise touch on the order of 518 million raw points.

The quantile problem

Rolling up p95/p99 correctly is not as simple as averaging averages — you can't merge two p95 values into a valid combined p95. Rollups store t-digest sketches instead of pre-computed percentiles, letting later aggregation stay mathematically sound at the cost of roughly 3x more storage per rollup point than a plain scalar would need. We measured error against ground truth: t-digest-derived p99s stay within 2-4% of the true value even after two levels of re-aggregation (raw → 1min → 1hour).

  • Rollups build progressively: raw → 1min (6hr delay) → 1hour (1 day delay) → 1day (1 week delay).
  • The planner auto-selects resolution to match roughly 2-4 raw points per rendered pixel.
  • Quantiles are rolled up as t-digest sketches, not scalars, keeping re-aggregated p99 within 2-4% of ground truth.