Feature Flag Strategy in Theseus

Feature flags are a superb tool when it comes to de-risking the rollout of potentially scary changes to production environments. A change ships dark, gets enabled deliberately, and can be turned off in seconds if it misbehaves — without a revert, a rebuild, or a redeploy. This page sets out how feature flags work for modular features on Theseus: what exists today, where the boundary between flags and configuration sits, and how the model extends to Cells and self-managed customers.

What’s Available Today

A single feature-flags service runs in production. It is backed by Flipt and serves flag evaluation over HTTP with an OpenFeature-compatible API. You do not need to know anything about Flipt to use it: flags are plain YAML files in the feature-flags repository, and the running service picks up a merged change within about 30 seconds — no deploy of the flag service, and no deploy of yours.

The service has been used in production by gitlab-shell as part of its validation, and is available for non-monolith development. Services consume it through the LabKit v2 featureflag client, which the go-service-template exemplar wires up end to end. The how-to guide walks through defining, evaluating, toggling, and removing a flag.

The service is available for production use by all of GitLab engineering. For questions or support, reach out to the team via the feature-flags repository.

Design for absence

Do not assume the feature-flags service is reachable — or even deployed — in every environment your code runs in. Self-managed customers will, for the vast majority of installations, not have it, and even on GitLab.com the service can be degraded or unreachable. Your application must work correctly regardless.

The rule is fail closed: when evaluation fails for any reason, fall back to the existing behaviour, never the new one. The LabKit client makes this the path of least resistance — pass false as the default value and an unreachable flag service has zero impact on your users. A feature that requires the flag service to be present in order to function is a design error; the flag service gates rollout, it does not gate the product.

Configuration vs Feature Flags

Feature flags and configuration answer different questions. A flag answers “is this change safe to be on yet?” — it exists to de-risk a rollout, and it should disappear once the rollout is complete. Configuration answers “how should this service behave in this environment?” — it is a permanent, documented part of the service’s contract.

In practice:

Feature flags should be short-lived. A flag is scaffolding around a rollout. Once the change is fully enabled and stable, remove the flag and the old code path.
Feature flags are not configuration. If a value needs to differ between environments indefinitely — endpoints, tuning knobs, per-tenant behaviour — it belongs in configuration, not behind a flag.
Feature flags are not for long-term rollouts. A flag that has been half-enabled for months is configuration wearing a disguise, with none of configuration’s discoverability or review guarantees.
For cell-based work, longer-lived items go through Theseus configuration. Anything that needs to vary per cell and persist belongs in the application configuration mechanism, where the tenant model materialises per-cell values deliberately and reviewably.

Cell-based feature flag strategy

Today there is a single instance of the feature-flags service, and it serves GitLab.com only. Cells change the shape of the problem: a modular feature deploys into many cells, and flag state needs to be evaluable wherever the service runs.

The building block is that the feature-flags service is an ordinary Helm release. One instance deploys into a Kubernetes cluster with whatever bespoke setup that environment needs — its own values, its own sizing, its own network posture — and everything stateful lives outside the cluster in the flags repository, which the service polls:

    flowchart LR
    repo["feature-flags repo<br/>flags/&lt;namespace&gt;/features.yaml"]
    chart["feature-flags<br/>Helm chart"]
    subgraph cluster ["Kubernetes cluster (bespoke setup)"]
        subgraph release ["Helm release: feature-flags"]
            flipt["Flipt<br/>OpenFeature evaluation API"]
        end
        svc["Consuming services<br/>LabKit featureflag client"]
        svc -->|HTTP evaluate| flipt
    end
    chart -->|"helm install<br/>(environment-specific values)"| release
    repo -->|"git poll (~30s)"| flipt

Because the release carries no state of its own, the same chart stamps out an instance per cell. Each cell that wants flag evaluation deploys its own feature-flags release with cell-scoped values, and every instance syncs from the same flags repository — one source of truth, evaluated locally in each cell. A cell that does not deploy one loses nothing: its services fail closed to their defaults, exactly as the design-for-absence rule above requires.

    flowchart TB
    repo["feature-flags repo<br/>(single source of truth)"]
    subgraph cell1 ["Cell 1"]
        ff1["feature-flags release<br/>(cell-scoped values)"]
        s1["services"]
        s1 -->|HTTP evaluate| ff1
    end
    subgraph cell2 ["Cell 2"]
        ff2["feature-flags release<br/>(cell-scoped values)"]
        s2["services"]
        s2 -->|HTTP evaluate| ff2
    end
    subgraph cell3 ["Cell 3 — no flag service"]
        s3["services<br/>fail closed to defaults"]
    end
    repo -->|git poll| ff1
    repo -->|git poll| ff2

Work is needed to get there: enabling an instance of the feature-flags service per deployment context, and allowing per-cell configuration of flag state. Until that lands, treat the service as a dotcom-only rollout tool — which is consistent with the design-for-absence rule above: a service that behaves correctly when the flag service is missing needs no changes to run in a cell that does not yet have one.

Self-Managed Customer Support

Modular features can configure the feature-flag client endpoint via Helm values, and services pick that up like any other configuration — so a self-managed customer technically can point a service at their own flag backend, or at nothing at all.

Encourage customers to do this only in exceptional cases. The moment we ask customer X to enable a feature flag to turn off a feature, that flag becomes a contract we then have to negotiate: it must keep existing, keep meaning the same thing, and keep being supported, which is precisely what short-lived flags are not. Application configuration has similar drawbacks — a documented setting is also a commitment — but it is far better suited to long-lived, customer-visible switches: it is reviewed as part of the service’s contract, documented alongside it, and not scheduled for deletion.

The default posture for self-managed remains: no feature-flags service deployed, every service falling back to its defaults, and everything working.