Helix's metric engine writes fixed-size chunks, not a growable array. Here is why 128 points won our benchmarks over 64 and 256.

Every time series in Helix is a sequence of 128-point chunks: 128 delta-encoded timestamps, 128 XOR-encoded float values, and a small header. That number was not a guess — we benchmarked 32, 64, 128, 256, and 512 across three workloads before settling.

Why chunk size matters at all

A chunk is the unit of compression, the unit of disk I/O, and the unit of cache residency. Too small and header overhead dominates — at 32 points, our chunk header (14 bytes of metadata: series ID, min/max timestamp, encoding flags) ate 8.7% of the compressed payload. Too large and a single late-arriving point forces us to rewrite a chunk that may already be flushed to object storage.

The 128-point sweet spot

At a 15s scrape interval, 128 points covers 32 minutes of data — long enough to amortize the header, short enough that an active chunk sits comfortably in the 64KB write buffer per shard. Compressed, a 128-point chunk of a well-behaved counter averages 180-260 bytes, versus roughly 340 bytes for two 64-point chunks covering the same window. That's a 25-30% storage win purely from fewer headers and better delta runs.

What it costs on the read path

The tradeoff shows up in point queries. Asking for a single sample forces us to decode an entire chunk average 210 bytes — not free, but cheap enough that our p50 chunk-decode time sits at 4.2 microseconds. We tested 512-point chunks hoping for even better compression (it was, roughly 6% smaller) but query fan-out queries that touch thousands of series paid a decode tax that pushed p99 query latency up 22%, which killed it for dashboard workloads.

from metrics
| where name == 'cpu.utilization' and pod =~ 'checkout-.*'
| where value > 0.85
| summarize count() by bin(timestamp, 5m)

That query touches roughly 40,000 active chunks in a mid-size cluster. At 4.2us median decode, the scan alone costs about 168ms before any aggregation runs — which is why chunk size is a planner input, not just a storage detail.

  • 128 points/chunk balances header overhead against rewrite cost for late data.
  • Compressed chunks average 180-260 bytes for typical counters and gauges.
  • Larger chunks compress marginally better but hurt fan-out query latency.
  • Chunk size is exposed to the query planner as a cost input, not hidden as an implementation detail.