HelixQL doesn't guess at filter selectivity — it keeps rolling histograms per label so the planner can cost a query before running it.

Before HelixQL executes a query, the planner needs to know roughly how many rows each filter will match, so it can order operations and pick join strategies. Guessing wrong means scanning far more data than necessary. Helix keeps lightweight statistics specifically to avoid guessing.

What gets tracked

Each shard maintains a rolling histogram (64 buckets, refreshed every 5 minutes) for common label values and a separate cardinality estimate via HyperLogLog for high-cardinality fields. These aren't perfectly accurate — HLL sketches carry roughly 2% standard error at our configured precision — but they're cheap to maintain and good enough to separate a filter that matches 50 rows from one that matches 5 million.

How estimates drive plan choice

Given a query with multiple `where` clauses, the planner orders them most-selective-first using these estimates, so expensive predicates run against the smallest possible row set. It's the same principle behind the hash-join-vs-merge-join decision, just applied to filter ordering instead of join strategy.

from traces
| where service == 'checkout-api'     # est. 40,000 rows/min
| where duration > 2s                # est. 1,200 rows/min
| where status == 'error'             # est. 90 rows/min
| summarize count() by route

The planner reorders this internally to filter on `status == 'error'` first if its selectivity estimate is lower, even though it's written last, cutting the working set before the more expensive duration comparison runs.

When estimates are wrong

Stale histograms are the main failure mode — a traffic shift can make a 5-minute-old estimate wrong by an order of magnitude during a spike. We mitigate this with a fallback: if actual rows scanned during execution diverge from the estimate by more than 10x, the query re-plans mid-execution rather than continuing down a bad path. This costs a small re-planning overhead (typically 2-4ms) but has saved several queries from turning a 200ms plan into a 20-second one during traffic anomalies.

What this means for query authors

Filter ordering in HelixQL is a readability choice, not a performance one — write filters in whatever order tells the story best, and let the planner reorder for cost. The one thing that still matters: giving the planner a service or env filter early in a query, even redundantly, helps it narrow the shard set it needs to touch at all, before histogram-based row estimation even comes into play.

  • Row estimates come from rolling histograms (64 buckets, 5-min refresh) and HyperLogLog cardinality sketches.
  • The planner reorders filters by estimated selectivity regardless of how the query was written.
  • A greater than 10x estimate-vs-actual divergence mid-query triggers a re-plan, costing 2-4ms.