Co-located Grafana dashboards

A modular feature’s dashboards are part of the service, not an artifact managed somewhere else. The observability configuration for a service ships in the same repository, the same merge request, and the same deploy as the code it observes. This page sets out that strategy: Grafana dashboards as jsonnet in the service repo, compiled in CI, packaged into the service’s Helm chart, and applied automatically wherever the service lands.

The problem with centrally managed dashboards

GitLab’s dashboards today are managed centrally — jsonnet in a shared repository, deployed by a dedicated pipeline to a shared Grafana. The model works when there is exactly one Grafana to deploy to. Cells break that assumption: a modular feature deploys into many cells, each with its own observability stack, and a central pipeline pointed at a single Grafana cannot reach them.

The central model has costs even before cells force the issue:

• Split ownership. The dashboard lives in a different repository from the service it describes. The merge request that changes a metric does not change the panel that graphs it, so the two drift apart quietly.
• Coordination overhead. Shipping a service means also shipping a change to a central repo, on a separate pipeline, on someone else’s review queue.
• Coupled delivery. Dashboards arrive when the central pipeline runs, not when the service deploys. A new service can be live in a cell for hours or days with no dashboards waiting for it.

Desired outcomes

• Dashboards versioned with the code — a change to a metric and a change to its panel land in the same merge request, reviewed together.
• Every cell gets its dashboards automatically — deploying the service to a new cell requires zero extra wiring or coordination.
• No central chokepoint — teams ship observability changes at their own pace, in their own repo.
• Lifecycle tied to the service — uninstalling the service removes its dashboards; there is no orphaned configuration to garbage-collect.

The strategy

1. Dashboards live in the service repository

Dashboard definitions live as jsonnet under a dashboards/ directory in the service repo, alongside the code they observe. Jsonnet keeps dashboards reviewable and composable — shared panels, templated variables, and consistent layouts come from libraries rather than copy-paste — and a dashboard change is an ordinary merge request against the service.

2. CI compiles jsonnet into the Helm chart

Helm cannot render jsonnet, so compilation happens where the rest of the service is built: CI compiles dashboards/*.jsonnet to JSON as part of the build, and the compiled dashboards are packaged into the service’s Helm chart. The chart is the single deployable artifact — code, configuration, and dashboards travel together, at the same version.

3. The chart ships dashboards as labelled ConfigMaps

The chart templates each compiled dashboard into a ConfigMap carrying the well-known discovery label:

metadata:
  labels:
    grafana_dashboard: "1"

Grafana’s dashboard sidecar — standard in the Grafana and kube-prometheus-stack Helm charts — watches for ConfigMaps with that label and loads them into the local Grafana instance. The service pushes nothing; the cell’s Grafana discovers what has been deployed into it:

    flowchart LR
    subgraph cell [Cell]
        subgraph release [Service Helm release]
            SVC[service]
            CM1[ConfigMap<br/>grafana_dashboard: 1]
            CM2[ConfigMap<br/>grafana_dashboard: 1]
        end
        subgraph grafana [Grafana]
            SC[dashboard sidecar]
            GF[Grafana]
        end
        SC -->|watches labelled ConfigMaps| CM1
        SC -->|watches labelled ConfigMaps| CM2
        SC --> GF
    end
  

4. Inert by default

If no Grafana exists in the cluster, the ConfigMaps simply sit there. helm install never fails, branches, or asks whether observability infrastructure is present — the chart makes no assumptions about its surroundings. The moment a Grafana with the sidecar appears in the cell, every already-deployed service’s dashboards load without a single redeploy.

Why this works

• It is cells-native. The mechanism is discovery, not push. A service deployed to one cell or fifty carries its dashboards with it, and each cell’s Grafana picks up exactly the dashboards of the services running there. Scaling to N cells adds zero wiring.
• Dashboard review happens in the feature merge request. The reviewer who approves the new metric approves the panel that graphs it. Drift between code and dashboard stops being a failure mode.
• No CRD dependency. ConfigMaps are core Kubernetes. The chart installs into any cluster without requiring an operator or custom resource definitions to exist first — which is what makes inert-by-default possible.
• Lifecycle is the Helm release. Dashboards appear on install, update on upgrade, and vanish on uninstall. There is no separate system whose state can disagree with what is deployed.