A naive inverted index per label value falls over past a few million series. Helix's postings-list design keeps lookups fast without that blowup.

A classic time-series inverted index maps each label value to a postings list of series IDs. It's simple and it's fast — until cardinality climbs and postings lists for common labels like `service` balloon into gigabytes that get rewritten constantly. Helix's index avoids that by splitting structure from content differently.

The problem with naive postings lists

If `region=us-east-1` matches 400,000 series, its postings list is 400,000 series IDs, sorted, delta-encoded, still a meaningful chunk of memory, and every new series matching that label means an insert into an already-large sorted structure. Multiply across dozens of common labels and index write amplification becomes the dominant cost, not query performance.

Helix's two-tier approach

We split labels into "structural" (low cardinality, high match rate: service, env, region — typically under 500 distinct values) and "identifying" (high cardinality, near-unique: pod, instance_id). Structural labels get traditional postings lists, since they're small and stable. Identifying labels get a different structure: a sorted, block-compressed term dictionary with binary search, avoiding a postings list per unique value entirely.

from traces
| where service == 'checkout-api'          # structural: postings-list lookup
| where pod == 'checkout-7f9d-x92k1'        # identifying: dictionary binary search
| summarize count() by bin(timestamp, 1m)

Query-time cost

A structural-label filter resolves in roughly 200 nanoseconds via postings-list intersection. An identifying-label filter costs more — around 1.8 microseconds for the binary search plus lookup — but since identifying labels are usually applied after structural ones narrow the candidate set, the absolute row count being searched is already small by the time it matters.

Rebuild cost, measured

The real win shows up in index maintenance. On a cluster with 2.8 million active series, rebuilding postings lists for structural labels after a compaction pass takes about 340ms. The old naive approach, before this split shipped, took 4.1 seconds for the same workload because every identifying-label postings list had to be rewritten too. That gap widens further as retention grows: at 90 days of retained series metadata on the same cluster, the naive rebuild stretched past 30 seconds, long enough to visibly delay downstream compaction, while the two-tier rebuild stayed under 1.2 seconds because the term dictionary for identifying labels never needs a full postings-list rewrite in the first place, only an append.

  • Structural labels (low cardinality) get classic postings lists; identifying labels (high cardinality) get a term dictionary instead.
  • Structural lookups: ~200ns. Identifying lookups: ~1.8us, applied after narrowing.
  • Index rebuild after compaction: 340ms vs 4.1s under the old single-postings-list design, and the gap widens further at longer retention.