Skip to content
Metropolis: Monitoring as Part of the Service

Metropolis: Monitoring as Part of the Service

The co-located dashboards strategy sets the principle: a service’s observability configuration belongs in the service’s repository, reviewed in the same merge request as the code it observes, shipped by the same deploy. Metropolis — the GitLab Metrics Operator — is the mechanism this strategy proposes for delivering on it. This page sets out how the two fit together and what changes from the original co-location design.

Metropolis ships in the standard Caproni rig, so services can develop against it today. Production adoption is staged across Dedicated and Cells, Runway, and Self-Managed in the Metropolis Adoption epic. The Observability group owns the operator, per the call-to-ownership issue, and the Theseus-side adoption questions continue in the decision record.

What Metropolis is

Metropolis is a Kubernetes operator plus a CLI. A service declares its monitoring as two custom resources, shipped in its Helm chart like any other template:

  • GitLabServiceMetrics — the service’s SLIs, SLO targets, and saturation points. On deploy, the operator turns the declaration into Prometheus recording rules, multi-window burn-rate alerts, and a generated overview dashboard — built with the same metrics-catalog library that has monitored GitLab.com through six years of incident-driven iteration. Additionally, the same monitoring stack has been used for GitLab Dedicated since Limited Access.
  • GitLabGrafanaDashboard — a Grafonnet entrypoint for dashboards beyond the generated overview. The operator evaluates it against its vendored library bundle and delivers the result to whatever Grafana lives in the environment. No jsonnet toolchain in the service repo.

The metropolis CLI renders exactly what the operator would, for local inspection and debugging; end-to-end verification happens against a running stack in the Caproni rig before a change is committed. The declaration is the reviewed surface — rendered output is a build artifact and is not checked in.

What changes from the original co-location design

The co-located dashboards page described compiling jsonnet in the service’s CI and shipping the compiled JSON as labelled ConfigMaps. Metropolis keeps every desired outcome of that design — dashboards versioned with the code, reviewed in the feature MR, discovered per cell, lifecycle tied to the Helm release — and moves the compile step from each service’s CI into one shared operator:

CI-compiled ConfigMapsMetropolis CRDs
Service repo carriesjsonnet + toolchain + compile stepa declaration (two CRs)
Dashboardshand-built per servicegenerated overview + optional custom
Rules and alertsnot coveredrecording rules + burn-rate alerts
Size ceiling1 MiB per ConfigMaplifted (gzipped delivery)
Generator upgradesevery repo re-vendorsupgrade the operator once

The consistency argument from the observability story applies here too: one battle-tested generator producing every service’s rules and dashboards keeps them uniform in a way per-repo toolchains never stay.

How it flows

    flowchart LR
    subgraph repo ["Service repository"]
        code["Service code"]
        gsm["GitLabServiceMetrics<br/>(SLIs, SLOs, saturation)"]
        ggd["GitLabGrafanaDashboard<br/>(Grafonnet entrypoint)"]
    end
    repo -->|"chart ships CRs"| release["Helm release"]
    release --> metro["Metropolis operator<br/>(one per cluster)"]
    metro --> rules["PrometheusRule<br/>rules + alerts"]
    metro --> dash["Dashboards<br/>(sidecar ConfigMap or<br/>grafana-operator resource)"]
    rules --> prom["Prometheus"]
    dash --> graf["Grafana in the cell"]
  

The cells posture

The design matches the platform’s rules for cell-based deployment:

  • Install once per cluster. The operator watches every namespace; a cell that runs it picks up every service’s monitoring declarations automatically, exactly as the sidecar-discovery model intended.
  • Quiet when the operator is absent. With the CRDs installed but no operator running, the CRs sit unreconciled: missing dashboards, never broken deploys. One difference from the ConfigMap model is worth knowing — a custom resource cannot be applied where its CRD is absent, so charts gate emission behind a value rather than relying on raw inertness. The outcome is still design for absence applied to monitoring; the mechanism is a flag instead of a no-op.
  • No cell-awareness. The service declares what to measure; where the rules and dashboards land is the environment’s concern.

Related reading

Last updated on