Not all data needs to sit on NVMe. Helix moves aging segments through storage tiers automatically, and the planner has to know which tier it's scanning.

Keeping 13 months of full-resolution data on NVMe would be economically absurd for most workloads. Helix ages data through three storage tiers — hot local SSD, warm object storage with a local cache, and cold object storage — and the query planner has to account for wildly different access latencies across them.

The tiers, by age

Data under 3 hours old lives on local NVMe (hot). From 3 hours to 7 days, it moves to object storage but stays in a local read-through cache seeded on access (warm). Past 7 days, it's object storage only, uncached until requested (cold). Movement between tiers happens as part of the compaction passes described elsewhere, so tiering and compaction share a background worker pool rather than running as separate jobs.

from metrics
| where name == 'cpu.utilization' and range == 45d
| summarize avg(value) by bin(timestamp, 1h)
# planner flags this as a cold-tier scan: expect object storage latency, not NVMe

Why the planner needs to know the tier

A chunk fetch from hot NVMe averages 80 microseconds. The same fetch from warm cache, on a cache hit, averages 400 microseconds; on a cache miss, it's an object storage round trip averaging 12-18ms. Cold tier fetches are always that 12-18ms round trip. If the planner assumed uniform latency, it would badly underestimate query cost for anything touching data older than a week, and could choose a join or scan order that makes sense for NVMe but is disastrous for a fan-out of object storage requests.

Batching cold-tier fetches

To avoid paying that 12-18ms round trip per chunk, cold-tier scans batch fetch requests — up to 64 chunks per object storage GET where the storage layout allows it — instead of issuing one request per chunk. This turns what would be an 18-second scan (1,000 chunks × 18ms sequential) into roughly 280ms of batched, parallelized fetches on the same query.

What this costs customers

In our published cost model, a query touching only hot-tier data costs roughly 1/150th what an equivalent cold-tier query costs in compute-seconds, which is why dashboards defaulting to a 24-hour view stay cheap by construction, and why we surface tier information in query plan output so teams can see when a "simple" historical query is quietly expensive.

  • Three tiers age data by time: hot NVMe (0-3h), warm cached object storage (3h-7d), cold object storage (7d+).
  • Cold-tier chunk fetches cost 12-18ms each; the planner batches up to 64 per request to avoid serial round trips.
  • Hot-tier queries cost roughly 1/150th of equivalent cold-tier queries in compute-seconds.