Before dashboards-as-code shipped to customers, we ran it internally against our own 200-plus dashboards for a full quarter. It changed the format twice.

We dogfood everything, but dashboards-as-code got a longer internal-only period than most features — a full quarter running against every dashboard our own engineering, sales, and support teams actually use day to day, before a single external customer saw the YAML format.

The first format was too clever

Version one supported inheritance — a dashboard could extend a base template and override specific panels, which looked elegant in the design doc. In practice, engineers on our own team couldn't tell what a dashboard actually rendered without mentally resolving two or three levels of inheritance, which defeated the entire point of a plain-text, reviewable format. We cut inheritance before anyone outside the company saw it.

The second problem: merge conflicts on panel order

# Before: panels as an ordered YAML list -> every reorder is a conflict
panels:
  - name: p95 latency
  - name: error rate

# After: panels keyed by stable id, order is a separate, rarely-touched field
panels:
  p95_latency: { name: p95 latency, order: 1 }
  error_rate: { name: error rate, order: 2 }

With panels as an ordered list, two engineers adding different panels to the same dashboard in parallel branches produced a git conflict almost every time, on a part of the file neither of them actually cared about resolving carefully. Keying panels by a stable ID and separating position into its own field cut that conflict rate to nearly zero in our internal usage.

Why the delay was worth it

Both of those problems would have been much more expensive to fix after customers had built up months of dashboards in the old format — a breaking migration instead of an internal iteration. Dogfooding at 200-plus real dashboards surfaced friction that a handful of beta customers with a dozen dashboards each likely wouldn't have hit in the same timeframe.

What we'd still change

Even after the quarter of internal use, one thing only became obvious once customers with genuinely different repo conventions than ours started adopting it: our default file-per-dashboard layout assumes one repo per team, and organizations with a single monorepo housing every team's dashboards wanted a directory convention we hadn't designed for. We shipped a configurable path template in response, but it's the kind of gap that only shows up once you leave the shape of your own internal usage behind.

  • Inheritance removed from the format before external release, for reviewability
  • Panels keyed by stable ID instead of list order, to avoid merge conflicts
  • A full quarter of internal-only dogfooding against 200+ real dashboards
  • Format changes made before they'd require a breaking customer migration