Onboard to Release Framework
This document walks you through joining Release Framework. The goal is to
get you from “I want to put my project on Release Framework” to a merged
MR that provisions everything you need, in one repository (infra-mgmt),
self-service.
If you are looking for the moving-parts story, what each piece does and why it’s wired the way it is, read Architecture and Internals instead.
What Release Framework gives you
Joining Release Framework provisions:
- A Canonical project on
gitlab.com(public by default) and a Security mirror (always private), with a push mirror from canonical to security. - A per-project service-account bot with its own subgroup. The bot is the credential for repository mirroring and the identity that the security mirror’s pipelines run as.
- Branch and tag protections following Release Framework’s locked-down defaults.
- Configurable approval rules, plus a mandatory PSIRT approval rule on the security mirror.
- common-ci-tasks integration (autolabels, danger, renovate, semantic-release on by default; goreleaser, terraform opt-in).
- Vault integration on both projects.
- A smart sync path from the security mirror back to the canonical project’s container and package registries (deploy token + CI/CD job-token allowlist), so builds and deploys originating from the security mirror can publish to Canonical’s registries.
- Optional sharing into a runway deployment project so that pipelines on the security mirror can trigger downstream deploys.
- Optional customer-facing publishing: releasing your Fairway-generated Helm chart to charts.gitlab.io and copying your released container image into the CNG registry. See Publish to charts.gitlab.io and the CNG registry.
There is no separate Build mirror. Builds and deploys for Release Framework projects run from the Security mirror.
If you need a configuration option that doesn’t exist yet, whether a new YAML field, a new module variable, or a new escape hatch, open a discussion on the Delivery tracker. We’d rather extend the platform deliberately than have you patch around a missing knob.
Release Framework is opinionated
Some defaults are non-negotiable on purpose (canonical public, security private, PSIRT approval mandatory on the security mirror, Vault always on, per-project service-account bot, locked-down protected-branch pushes). For the full list and the reasoning behind each, see An opinionated framework in the architecture doc.
Variables exist where projects legitimately need to vary behavior, things like approval rules, MR settings, common-ci-tasks toggles, share groups, and a few migration-only escape hatches. The defaults are the defaults for a reason; if you find yourself fighting them, open a discussion before working around them.
Prerequisites
Before you start:
- You can open MRs against
gitlab-com/gl-infra/infra-mgmt. - You know the canonical group path your project lives under (e.g.,
gitlab-org/software-delivery). - You’re aware that Atlantis applies the change after merge, review the plan output before applying.
Decide your path
flowchart TD
A{Does the canonical<br/>project already exist?}
A -- No --> B[New project flow]
A -- Yes --> C[Importing existing projects]
- Greenfield (nothing exists yet): New project flow.
- Anything already exists (canonical, security mirror, or both): Importing existing projects. The staged-import guidance applies whether you’re importing one resource or many.
New project flow
1. Add a YAML entry
Edit
data/projects/release-platform/repos.yaml
in infra-mgmt. The header comment at the top of that file is the
authoritative schema, covering required and optional fields, defaults, and
inline documentation for every option. Read it.
A minimal entry looks like this:
my-project:
name: "My Project"
description: "What my project does"
canonical_group: "gitlab-org/your-group"
share_groups:
delivery:
path: "gitlab-org/delivery"
access: "developer"
common_ci_tasks:
enabled: true
danger: true2. Pick the shape that matches your project
A few common cases:
Project that deploys via runway. Add runway_service_ids. This
field is the opt-in signal that infra-mgmt reads to grant the bot
subgroup developer access on the runway deployment project(s), which is
what lets the security mirror’s pipelines trigger the downstream deploy
bridge. Each entry in the list maps to one deployment project; the path
is derived automatically as
gitlab-com/gl-infra/platform/runway/deployments/<service-id>:
my-service:
name: "My Service"
description: "..."
canonical_group: "gitlab-org/your-group"
runway_service_ids: ["my-service"]
common_ci_tasks:
enabled: true
goreleaser: truePrivate canonical with a module-managed security namespace. If your
security namespace is a group that infra-mgmt itself manages
(e.g., gitlab-org/security/auth), set both canonical_visibility and
security_namespace:
my-private-project:
name: "My Private Project"
description: "..."
canonical_group: "gitlab-org/auth"
canonical_visibility: "private"
security_namespace: "gitlab-org/security/auth"If your security namespace doesn’t exist yet and isn’t already module-managed, read the Module-managed vs pre-existing security namespaces section in the architecture doc before proceeding.
Custom Security namespace set security_namespace.
3. Open the MR
Open a merge request in infra-mgmt. Atlantis will run a plan and post the
output back to the MR.
4. Review the Atlantis plan
Look for:
- All the resources you expect to be created (canonical project, security project, bot subgroup, service account, access token, group shares, deploy token, CI/CD variables, etc.).
- No
destroyactions on anything pre-existing. - The runway-deployment share resource(s) if you set
runway_service_ids.
If the plan looks wrong, fix the YAML and push again. Don’t merge a plan you don’t understand.
5. Apply
Atlantis applies the change when you comment atlantis apply. Watch the apply output for
errors. After a successful apply, Atlantis will Merge.
6. Sanity-check after the apply
- The bot subgroup exists at
gitlab-com/gl-infra/release-framework-bots/<your-project-name>, with the service account user as a developer. - The push mirror on the canonical project is configured and shows as successful (canonical project -> Settings -> Repository -> Mirroring).
- The security mirror is reachable and contains your default branch.
- The security mirror has the protected CI/CD variables
CANONICAL_REGISTRY_TOKENandCANONICAL_REGISTRY_TOKEN_USERNAME. - If you set
runway_service_ids, the bot subgroup is shared with the runway deployment project(s) at developer access (one share per service ID).
7. Wire your CI
It is mandatory that you scaffold your project using
common-template-copier.
Answering release_platform: true during the copier prompts generates a
.gitlab-ci.yml with release_platform_enabled: true already set on the
relevant common-ci-tasks component includes — no manual wiring needed.
Importing existing projects
The module supports importing existing projects. For the common case where
only the canonical project already exists, set import: true in your
YAML entry and the module generates the canonical project’s import block for
you (see
Importing the canonical project with import: true
below). For any other pre-existing resource (approval rules, archived
projects from prior failed applies, security mirror, and so on) you still add
Terraform import {} blocks by hand alongside your YAML entry in
environments/release-platform/projects_release_platform.tf.
Importing existing projects is something we do today, but it is fragile. The WARNING below is not theoretical, it is the lived experience that shaped this section. To contain the blast radius, prefer staging the work across two MRs:
- MR 1 - state alignment. Add the YAML entry with values that match
the current state of the project as closely as possible (existing
approval rules, share groups, MR settings, common-ci-tasks toggles)
plus the
import {}blocks. The Atlantis plan after import should show zero or near-zero diffs. This MR is about getting Terraform’s view of the world to agree with reality. - MR 2 - configuration. Once the project is cleanly under module management, edit the YAML to apply the Release Framework configuration you actually want. If this MR’s apply fails, you fail forward on configuration, not on Terraform’s import/destroy decision tree.
A single-MR import is still allowed for trivial cases (e.g., a brand-new project with default settings and no existing approval rules), but when in doubt, split it.
The order of operations below is the same as the new-project flow, with one extra step. For a staged import, run through it once per MR; the YAML diff is the only thing that changes between them.
Add your YAML entry to
data/projects/release-platform/repos.yaml. If the canonical project already exists, setimport: truein that entry (see Importing the canonical project withimport: true).Add
import {}blocks for any other pre-existing resource to the bottom ofenvironments/release-platform/projects_release_platform.tf. Real-world examples already in that file (artifact-registry, knowledge-graph, iam) are the templates to copy. Each block looks like:import { to = module.release_platform_project["my-project"].module.canonical.gitlab_project.project[0] id = "<numeric-project-id>" }With
import: trueset, the canonical project’s block is generated for you, so you do not hand-write that one. You still need a block for every other pre-existing resource the module would otherwise try to create: any pre-existing approval rules, the security mirror, archived projects from prior failed applies, and so on. Walk the module’s resource graph and the existing real-world imports to make sure you cover everything.Open the MR.
Read the Atlantis plan carefully. Imports should appear as
importactions, notcreate. There should be nodestroyactions on anything that already exists.alantis apply
Warning
If your import fails for any reason, treat it as urgent.
Once an import is in flight, the project is in a fragile state. A failed apply, whether from a state mismatch, a missing import for a resource that already exists, or unexpected drift, leaves Atlantis in a position where the next reconcile may interpret the unmanaged-but-already-existing resources as drift to reconcile. Depending on the resource, that reconciliation can include archival or destruction of the project.
Do not leave a half-imported project sitting overnight. If you need help
unwinding the failure, reach out to Delivery in #g_delivery on Slack
or ping a Delivery engineer directly, no need to open an issue first
if it’s blocking.
Importing the canonical project with import: true
When the canonical project already exists on gitlab.com, a plain apply fails
with a 400 has already been taken from the GitLab API, because the module
tries to create a project that is already there. Setting import: true in
your repos.yaml entry tells the module to import that project into Terraform
state instead of creating it:
my-project:
name: "My Project"
description: "..."
canonical_group: "gitlab-org/your-group"
# The canonical project already exists; import it rather than creating it.
import: trueThe import ID is derived as <canonical_group>/<path>, where path defaults
to the map key if you don’t set it explicitly. The field defaults to false,
so leave it out for greenfield projects.
import: true covers the canonical project only. It does not import the
security mirror, approval rules, or any other pre-existing resource; those
still need hand-written import {} blocks as described above. Because the
canonical-project import is the most common one, the flag removes the step
that was most error-prone to write by hand, but it does not remove the need to
read the Atlantis plan carefully before merging.
Runway deployment opt-in
If your project deploys via runway, wire it up in the order below. The steps
span two repos, and the runway/provisioner change lands in two MRs
with the infra-mgmt opt-in applied in between. This ordering is not
optional.
Important
Do not add extra_main_push_groups in the same runway/provisioner MR
that first provisions the deployment project. It will fail with a partial
apply — the deployment project is created, but its main branch-protection
fails with 422 Push access levels group does not have access to the project. Follow the phased sequence below; the “Why this order” note at the
end explains the underlying circular dependency.
1. Phase 1 — add your workload entry in runway/provisioner (without extra_main_push_groups)
Open an MR against
config/runtimes/gke/workloads.yml
in runway/provisioner. This registers your service and provisions the
deployment project. Leave extra_main_push_groups out for now — it is added
in Phase 2, after the bot subgroup has been shared into the project:
- runway_service_id: <service-id>
project_id: <security-mirror-project-id>
regions:
- us-east1
groups:
- <deploy-repo-access-group>project_idis your security mirror project id ongitlab.com(the security mirror of the project you onboarded), not the canonical project, and not the runway deployment project id. The provisioner creates the deployment project itself.groupsmaps toshare_groupsat maintainer access on the deployment repo. Use it for a human-facing owner group (for example your team’s group); it is optional.
This apply creates the deployment project cleanly, because its main
branch-protection does not yet reference any group it cannot resolve.
2. Set runway_service_ids in infra-mgmt
In your repos.yaml entry:
runway_service_ids: ["<service-id>"]This shares your project’s bot subgroup with each now-existing runway
deployment project at developer access (one share per service ID), which is
the permission the security mirror’s pipeline needs to trigger the downstream
Runway deployment bridge. For Runway v2, the generated job is
Trigger deployment pipeline; legacy v1 projects may use Create deployment Helm chart. Each deployment path is derived
automatically as gitlab-com/gl-infra/platform/runway/deployments/<service-id>.
This apply must run after Phase 1: its data.gitlab_project lookup
requires each deployment project to already exist.
For the full picture of why this share exists and where it lives, see The service account and Identity and permissions in the architecture doc.
3. Phase 2 — add extra_main_push_groups in runway/provisioner
Open a second MR against workloads.yml adding extra_main_push_groups to
the entry from Phase 1. The bot subgroup path is always
gitlab-com/gl-infra/release-framework-bots/<project-key> where
<project-key> matches the map key you used in repos.yaml:
- runway_service_id: <service-id>
project_id: <security-mirror-project-id>
regions:
- us-east1
groups:
- <deploy-repo-access-group>
extra_main_push_groups:
- gitlab-com/gl-infra/release-framework-bots/<project-key>extra_main_push_groups adds the named group to the deployment project’s
main branch-protection allowed_to_push list. The Release Framework bot
subgroup must go here (not under groups), because triggering the
downstream Runway deployment bridge requires the bot to be in
allowed_to_push; a maintainer share alone does not satisfy that check. This
apply now succeeds because Step 2 already shared the bot subgroup into the
project.
4. Configure your .runway/ service files
This step lives in your project’s own (canonical) repository and is
independent of the provisioning sequence above — you can do it at any point.
Runway v2 uses a flat .runway/ layout with these files at the root:
fairway.yaml— the Fairway manifest used to build and publish the service Helm chart.deployment.yaml— the RunwayManifest v2 deployment configuration.values.yaml— Helm overrides, including the registry pull secret.
The release-platform CI component uses fairway.yaml to publish the chart,
then the Runway v2 bridge passes the chart and deployment inputs to the
downstream deployment project.
.runway/deployment.yaml — declare the RunwayManifest, pointing chartRef
at your security mirror’s chart registry. chartPullSecret and vaultAccessTokens
allow Runway to pull the Helm chart (an OCI artifact) from the security mirror
registry — this is a control-plane concern, separate from the pod’s own image
pull authentication which is configured in values.yaml:
apiVersion: runway/v2
kind: RunwayManifest
metadata:
name: <service-id>
spec:
chartRef: "oci://registry.gitlab.com/<security-mirror-path>/chart/<chart-name>:{% .ChartVersion %}"
chartPullSecret: "security-registry-reader"
vaultAccessTokens:
- name: security-registry-reader
mount: ci
path: access_tokens/gitlab-com/<security-mirror-path>/read-registryReplace <service-id> with your runway service ID, <security-mirror-path>
with the full path of your security mirror project on gitlab.com using /
as the separator (e.g., gitlab-org/security/ops/artifact-registry), and
<chart-name> with the Helm chart name published to that registry.
.runway/values.yaml — provide Helm values including imagePullSecrets
so the pod can pull images from the security mirror’s container registry:
imageOverride: registry.gitlab.com/<security-mirror-path>
imagePullSecrets:
- name: security-registry-readerAdd any additional Helm overrides (scalability, resources, probes, labels,
etc.) here as needed.
See the
release-platform-canary example
for a complete reference.
Why this order
The provisioner sets allowed_to_push on main, but GitLab rejects any group
in allowed_to_push unless that group is already shared into the project.
That share is owned by infra-mgmt (via runway_service_ids) and its
data.gitlab_project lookup needs the deployment project to already exist. So
the steps chain: Phase 1 creates the project, infra-mgmt then creates the
share, and Phase 2 finally references the group. This mirrors the two-MR
staging idiom already used for
Importing existing projects.
Publish to charts.gitlab.io and the CNG registry
Runway deploys your service inside GitLab.com. Shipping it to customers who run GitLab themselves is another part of the release platform. There are two independent opt-ins:
- charts.gitlab.io is the source of truth for
customer-facing chart releases, the index customers point
helm repo addat. - The CNG registry (
registry.gitlab.com/gitlab-org/build/cng) is where the GitLab Helm chart resolves its default image references. Copying your released image there is what makes the chart’s image reference work for Self-Managed customers, without adding a Dockerfile and a build trigger togitlab-org/build/CNGfor every service.
These two opt-in toggles are specific to projects that use Runway. Once enabled, they make your Helm chart and container image installable by customers who run GitLab themselves.
Each opt-in has two halves: a provisioning flag in infra-mgmt, which mints
the credential or allowlist entry, and a CI input on the release-platform
component, which adds the job. You need both. The flag alone provisions
something nothing uses; the input alone gives you a job that fails on
permissions.
Important
Both jobs run on the canonical project only, never on the security mirror.
flowchart LR
S[Security mirror<br/>tag pipeline] -->|smart sync| C[Canonical<br/>tag pipeline]
C -->|cng-image-copy| N[CNG registry]
C -->|charts-gitlab-io-release| G[charts.gitlab.io]
1. Turn on provisioning in infra-mgmt
Add the flags you need to your repos.yaml entry:
my-service:
name: "My Service"
canonical_group: "gitlab-org/your-group"
charts_gitlab_io_release: true
cng_image_copy: truecharts_gitlab_io_release adds the canonical project to the CI/CD job token
allowlist on
charts/charts.gitlab.io, so the
release job’s CI_JOB_TOKEN trigger is accepted.
cng_image_copy creates a rotating project access token on
gitlab-org/build/CNG, scoped to read_registry + write_registry, and writes
its username and token to Vault under the
ci/<your canonical VAULT_SECRETS_PATH>/protected/shared/cng-image-copy path.
Important
The CNG project is not managed by infra-mgmt, so this apply needs the
provider credentials to hold Maintainer there. If the plan or apply fails on
the CNG deploy token, that is the reason. Raise it in #g_delivery rather
than working around it.
Open the MR, review the plan and apply exactly as in step 5.
2. Wire the CI inputs
Set the matching inputs on the release-platform component in your canonical
repository’s .gitlab-ci.yml. These inputs need release-platform@v5.23.0 or
newer.
include:
- component: $CI_SERVER_FQDN/gitlab-com/gl-infra/common-ci-tasks/release-platform@v5.23.0
inputs:
runway_service_id: my-service-gke
runtime: gke
release_platform_enabled: true
canonical_project_id: "12345678"
canonical_registry_path: gitlab-org/your-group/my-service
# Release the Fairway-generated chart to charts.gitlab.io
charts_gitlab_io_release_enabled: true
# Copy the released image into the CNG registry
cng_image_name: gitlab-my-service| Input | Needed for | Notes |
|---|---|---|
charts_gitlab_io_release_enabled | chart release | Defaults to false. |
canonical_registry_path | chart release | Required when the chart release is on. Only canonical’s registry is publicly pullable, so the component fails rather than guessing. |
charts_gitlab_io_release_unstable_channel | chart release | Publish to the unstable channel instead of stable. Defaults to false. |
charts_gitlab_io_manual_approval | chart release | Hold the release behind a manual play instead of releasing on tag. Defaults to false. See
Gating the release behind a human. |
cng_image_name | image copy | The name to publish under registry.gitlab.com/gitlab-org/build/cng. Empty (the default) means the copy component is not included at all, i.e. no copying is done. |
Two things this deliberately does not give you:
- No
chart_name,chart_versionoroci_refinputs. Aderive-chart-metadatajob in thevalidatestage computes all three and hands them to the trigger job as a dotenv artifact. The chart name comes from.metadata.namein<runway_dir>/fairway.yaml, the version is${CI_COMMIT_TAG}with any leadingvstripped, and the OCI reference is${CI_REGISTRY}/<canonical_registry_path>/chart/<name>:<version>, which is where the sync pipeline lands the chart. - No way to release without a Fairway manifest. The chart release reads
<runway_dir>/fairway.yaml, sorunway_dir: ""is incompatible withcharts_gitlab_io_release_enabled: true.
The chart release only fires on tags with a non-zero major version (v1.0.0
and up), matching Fairway’s own chart push condition. A v0.x.y tag still
builds and syncs the chart; it just isn’t released to customers.
Note the version-format asymmetry, which the components handle for you: CNG
image tags keep the v prefix (gitlab-my-service:v1.2.3), chart versions are
bare semver (1.2.3).
3. What runs, and when
On a mirrored project the publishing pipeline is the canonical tag pipeline,
which exists because the sync pipeline’s sync-commits job pushed the tag
there. By that point sync-registry-artifacts has already copied both the image
and the chart to canonical, which is what these jobs read.
derive-chart-metadata(validatestage) resolves the chart coordinates and fails the pipeline if any of them is missing.- The chart trigger (
publishstage) triggers a pipeline incharts/charts.gitlab.iowithstrategy: depend, so a downstream failure fails your pipeline. This replaces the old fire-and-forgetcurltrigger that reported nothing back. cng-image-copy(publishstage) runsskopeo copy --all, so manifest lists stay intact, multi-arch images stay multi-arch, no layers are rebuilt, and the destination digest matches the source. It reads its credential from the Vault path provisioned in step 1 and logsCNG_REGISTRY_AUTH_SOURCE: Vaultto confirm it.
The component picks its credential source from $VAULT_SECRETS_PATH: set means
Vault, unset means CI/CD variables. Release Framework projects always have Vault
integration, so the Vault path is the one that applies here. The CI/CD variable
route is for projects outside the Release Framework and never comes into play on
a project onboarded through this guide.
Gating the release behind a human
By default the chart releases as soon as the tag pipeline reaches it. Set
charts_gitlab_io_manual_approval: true to hold it behind a manual play instead,
so the team that owns the chart decides when it goes out:
inputs:
charts_gitlab_io_release_enabled: true
charts_gitlab_io_manual_approval: trueThe input forwards to manual_approval on the underlying
charts-gitlab-io-release component. Two consequences worth knowing before you
turn it on:
- The gate lives in your pipeline, not downstream. The trigger job is simply
left unplayed, so no pipeline is created in
charts/charts.gitlab.iojust to hold a button. On a mirrored project the button is in canonical’s tag pipeline, which is where the trigger job runs. - It is a blocking manual job. Your tag pipeline stays unfinished until someone plays or cancels it. It will not go green with the chart unreleased, which is the point: an unreleased chart should not look like a completed release.
Everything else is unchanged. The image copy is not gated by this input, and
derive-chart-metadata still runs in the validate stage, so a coordinate
problem fails the pipeline before anyone is asked to approve anything.
4. Sanity-check after your first tag
# The chart is in the public index
helm repo add gitlab https://charts.gitlab.io
helm repo update
helm search repo gitlab/<chart-name> --versions
# The image landed in CNG (note the v prefix)
docker manifest inspect registry.gitlab.com/gitlab-org/build/cng/<cng_image_name>:v<version>- The
cng-image-copyjob log showsCNG_REGISTRY_AUTH_SOURCE: Vault. There are noCNG_REGISTRY_*CI/CD variables to check for; the credential lives only in Vault. - The canonical project appears on the job token allowlist of
charts/charts.gitlab.io.
Troubleshooting
403on the chart trigger. Eithercharts_gitlab_io_release: truehas not been applied yet, or the job ran from the security mirror; checkrelease_platform_enabled: trueis set.helm pull404s in the downstream pipeline. The chart has not reached canonical yet. The trigger’s stage is running before the sync that copies it.derive-chart-metadatafails. Missingfairway.yaml, no.metadata.name, an empty tag, orcanonical_registry_pathunset. The job message names which is missing.no credential for <path>.cng_image_copy: truehas not been applied, or the tag ref is not protected, so Vault will not release the secret.unauthorizedpushing to CNG. The token in Vault has expired. It is provisioned with a 30-day rotation, so aninfra-mgmtapply that has not run in a while can leave a stale one behind.- The chart job never appears. Most often a
v0.x.ytag, which is below the non-zero-major threshold. - The chart job is sitting there unplayed. That is
charts_gitlab_io_manual_approval: truedoing its job. Play it when the chart is ready, or set the input back tofalseto release on tag. - The job hangs after being played.
strategy: dependwaits for the downstream pipeline, so this ischarts.gitlab.iostill working. Follow the link on the job to watch it.
For the full input reference and further troubleshooting, see the upstream
component docs:
release-platform,
charts-gitlab-io-release,
and
cng-image-copy.
Day-2 changes
Most ongoing changes are YAML edits in repos.yaml:
- Approval rules: edit the
approval_rulesmap. - Share groups: edit
share_groups. - common-ci-tasks toggles: edit the
common_ci_tasksmap. - MR setting overrides: edit the
merge_requestsmap. - Vault paths: edit the
vaultmap. - Customer-facing publishing: toggle
charts_gitlab_io_releaseandcng_image_copy, then match them with the component inputs in your.gitlab-ci.yml.
Open an MR, review the Atlantis plan, run atlantis apply to merge.
Where to get help
- Bug, broken mirroring, broken pipelines, broken token rotation: Delivery tracker.
- Failed import (urgent): ping
#g_deliveryon Slack or contact a Delivery engineer directly. - Need a new YAML field, module variable, or platform feature: open a discussion on the Delivery tracker describing what you need and why.
- Background reading: the Architecture and Internals doc covers what each component does and why.