The cheapest cardinality is the series you never index. Here's how Helix enforces per-tenant budgets at ingest, not at query time.

By the time a runaway label set shows up in a query timeout, the damage is already in the index. Helix enforces cardinality budgets at the write path instead, rejecting or downsampling series before they ever get a postings-list entry.

Where cardinality actually comes from

In practice, cardinality blowups are rarely malicious — they're a customer_id, a request_id, or a raw URL path accidentally attached as a label instead of a value. One incident we studied had a single deploy add a `user_agent` label to a request-duration histogram, taking that metric from 4,200 active series to 1.9 million in under six hours.

The write-path budget

Each tenant gets a configurable series budget per metric name, default 50,000. Every write carries its full label set through a fast fingerprint hash before it touches the index. If the resulting series is new and the tenant is within budget, it's admitted. If the tenant is over budget, new series are dropped and counted, and existing series continue to be written.

ingest:
  cardinality_limits:
    default_series_per_metric: 50000
    enforcement: reject_new   # options: reject_new, sample, alert_only
    overflow_metric: 'helix.ingest.cardinality_dropped'

Why we don't just downsample silently

We considered silently merging overflow series into an <other> bucket, which is what several competitors do. We rejected it as a default because it hides the real problem — a mislabeled metric — behind a graph that still looks fine. Instead, `helix.ingest.cardinality_dropped` becomes its own low-cardinality counter, alertable and visible, so the team that shipped the bad label finds out within the first scrape interval rather than during a billing review.

The index-side payoff

Enforcing budgets at ingest means our inverted index never has to deal with the ugly tail of one-off series. Across our fleet, this keeps index memory per node predictable within about a 12% band week to week, versus the 3-4x swings we saw before the limiter shipped, driven entirely by occasional bad deploys elsewhere in a customer's stack.

  • Cardinality limits are enforced per tenant, per metric, at write time — not after the index has already grown.
  • Overflow is rejected and counted, not silently merged, so the bad label gets found fast.
  • Default budget is 50,000 series per metric name, configurable per tenant.