Label sets repeat constantly across a fleet. Encoding them once and referencing them by ID, instead of storing them, is where most of our storage savings come from.

A label set like {service="checkout-api", region="us-east-1", pod="checkout-7f9d"} might repeat across thousands of chunks. Storing that string blob every time is wasteful, so Helix runs a shared dictionary per shard and stores a 4-byte reference instead of the label set itself.

How the dictionary works

Every unique label-set string, sorted and canonicalized so key order never matters, gets interned into a shard-local dictionary the first time it's seen, and every chunk that shares it just stores the resulting integer ID. The dictionary itself is stored once, compressed with zstd at level 9 since it's write-once, read-many.

# before dictionary encoding, per chunk header
labels: 'service="checkout-api",region="us-east-1",pod="checkout-7f9d"'  # ~68 bytes

# after dictionary encoding
label_id: 4821  # 4 bytes, dictionary entry stored once per shard

Measured savings

Across a representative 24-hour segment from a mid-size customer (around 340,000 active series, 2.1 billion chunk headers written that day), raw label strings would have cost roughly 142GB. Dictionary encoding plus zstd compression on the dictionary itself brought that down to 55GB — a 61% reduction, and that's before we count the downstream win of smaller headers meaning more chunks per I/O page.

The cost of dictionary lookups

The tradeoff is an extra indirection on every label read. A dictionary lookup is a hash-map access, averaging around 40 nanoseconds warm, but a cold shard (just loaded from object storage) pays a one-time cost to rehydrate the full dictionary into memory — typically 80-200ms for a shard with a few hundred thousand distinct label sets. We prefetch dictionaries alongside chunk indexes specifically to hide this behind I/O that was happening anyway.

Why this doesn't fight the cardinality limiter

Dictionary encoding and the write-path cardinality budget solve different problems: the budget stops the dictionary from growing unbounded in the first place, and encoding shrinks the cost of the label sets that are legitimately allowed to exist. Without the budget, a runaway metric would still blow up dictionary size even at 4 bytes per reference, just more slowly.

  • Label sets are interned once per shard and referenced by a 4-byte ID instead of stored inline.
  • Measured storage reduction: 61% on a 340,000-series representative segment.
  • Cold shard dictionary rehydration costs 80-200ms, hidden behind existing chunk-index I/O.