Get a remote GitLab environment with Hangar
Hangar gives you an isolated vcluster on a
shared GKE host, and
Caproni deploys GitLab into
it exactly as it does on your laptop. The result is a production-like CNG
GitLab at https://gitlab.<namespace>.<baseDomain>, reachable by you in a
browser or by an agent through the API, that ends itself after a TTL.
This page is the paved path through Hangar’s own runbook, deploying GitLab to a remote vcluster. Each step links the section that explains it; where this page and Hangar’s docs disagree, theirs are current.
Before you start
Access. You need to be a tenant on the host, which is granted by hand
today. Open an
access request issue
and work through its checklist: sign in once at
gitlabsandbox.cloud so you have an identity to
bind, and request NordLayer Permission:Dedicated Gateway through Lumos (the
Public WiFi permission does not reach the host). An operator grants you
roles/container.clusterViewer and adds you to the caproni-tenant binding.
Tooling. gcloud with the GKE auth plugin, and
mise. Hangar’s .mise.toml pins caproni,
vcluster, helm and kubectl at versions known to work together, and a
checkout under ~/hangar/envs/ inherits that pin.
Identity check. Connect to NordLayer, fetch credentials, and confirm who the cluster thinks you are:
gcloud components install gke-gcloud-auth-plugin # once per machine
gcloud container clusters get-credentials hangar-host \
--zone us-central1-a --project caproni-remote-envs-4fb1f3fd
kubectl auth whoami # your GCP IAM member
kubectl auth can-i '*' '*' --all-namespaces # want: noThe identity printed is what the ownership policy stamps on your namespace. If
the last command prints yes, say so on your
access request issue
before you provision anything. See
before you start.
1. One checkout per environment
The environment’s name is derived from the checkout directory, and the namespace must equal the vcluster’s name for the host’s generated route to point at it. So one checkout is one environment, and a second environment is a second checkout, not an edited config. Name the directory for what it is.
Or from ~/hangar, ./scripts/gitlab-env.sh up my-feature does this step and
the next, and ./scripts/gitlab-env.sh list shows what you have running. See
setup using the script.
git clone git@gitlab.com:gitlab-org/quality/hangar.git ~/hangar
git clone git@gitlab.com:gitlab-org/gitlab-caproni.git ~/hangar/envs/my-feature
cd ~/hangar/envs/my-feature
mise trust && mise install
cp ~/hangar/vcluster.values.yaml .
cp ~/hangar/examples/caproni.local.yaml .
export CAPRONI_HOST_CONTEXT=gke_caproni-remote-envs-4fb1f3fd_us-central1-a_hangar-hostKeep the directory name under 17 characters; longer names are cut. The two
copied files are Hangar’s overrides for a gitlab-caproni checkout: the
namespaced syncer RBAC and API server source ranges, and a trimmed
caproni.local.yaml that swaps the dev CA for the host’s wildcard certificate,
adds a ClusterIP Traefik inside the vcluster, and turns HTTPS on for the
GitLab hostnames. Nothing in them needs editing for the current host. What each
override is for is in
deploying GitLab to a remote vcluster.
2. Bring it up
From the checkout root:
caproni config validate && caproni upValidate first: it catches a needs entry pointing at a deployer the override
removed, which is the easy mistake. GitLab serves 5 to 10 minutes after
caproni up returns. To watch it, from ~/hangar:
./scripts/gitlab-env.sh wait my-featureExpected while it starts: ERR command error command="kubectl version ..."
lines until the API server answers, webservice and sidekiq restarting while
gitlab-migrations runs, and gitlab-runner crash-looping until GitLab
answers. If you see Unable to connect to the server: context deadline exceeded right after cluster started successfully, wait a couple of minutes
and run caproni up again.
3. Check it is yours, and not public
Your namespace and URL come from the checkout (yq is pinned by
gitlab-caproni, so step 1’s mise install provides it). The script saves
both to hangar.env in the checkout.
NS=$(caproni config print | yq .cluster.vcluster.namespace)
URL=https://$(caproni config print | yq .cluster.network.hostname)
kubectl get ns -l caproni.gitlab.io/managed=true \
-o custom-columns='NAME:.metadata.name,OWNER:.metadata.annotations.caproni\.gitlab\.io/owner'
kubectl -n $NS get rolebindings # caproni-owner-admin, caproni-owner-vcluster-rbac
kubectl -n $NS get svc $NS -o jsonpath='{.spec.loadBalancerSourceRanges}'Your namespace should list your address as owner, and the two RoleBindings are
generated by the policy, not by you. The last command is the one with no
symptom when it fails: it must print the NordLayer gateway ranges. If it prints
nothing, your vcluster’s API server is open to the internet. Run
caproni destroy and raise it in
Hangar’s issues
rather than leaving it up.
4. Reach it
./scripts/gitlab-env.sh status my-feature # from ~/hangar
curl -kso /dev/null -w '%{http_code}\n' $URL/-/readiness # or by hand: 200
mise run gitlab:rootpassword # from the checkout
caproni kubectl get ns # talks to your own API serverOpen $URL in a browser on NordLayer and sign in as root with that
password. Certificates come from Let’s Encrypt staging until a team onboards,
so expect a trust warning; read the issuer rather than the exit code. Inside
your vcluster you are cluster-admin. On the host you hold admin on your own
namespace and nothing else.
5. Set how long it lives
The reaper destroys an environment 24 hours after its vcluster was created,
whether or not you are still in it. Ask for longer, up to a 720-hour cap, at
up or any time before it expires:
./scripts/gitlab-env.sh up my-feature 168h # at up
./scripts/gitlab-env.sh ttl my-feature 7d # any time after
kubectl annotate ns $NS caproni.gitlab.io/ttl=168h --overwrite # without the scriptA TTL is a positive integer followed by s, m, h or d. Expiry counts
from creation, not from the annotation, so setting a TTL later does not restart
the clock, and a shorter one can end the environment at the next sweep. How the
cutoff is derived and clamped is in
reaping environments.
6. Tear it down
From ~/hangar:
./scripts/gitlab-env.sh list # what you still have up
./scripts/gitlab-env.sh destroy my-feature # only that oneThe checkout stays, so up with the same name brings the same environment
back. Without the script, from the checkout:
caproni destroy
gcloud compute disks list --project caproni-remote-envs-4fb1f3fd \
--filter="name~'^pvc-' AND -users:*"The host is shared and LoadBalancers bill by the hour, so do run one of these. Teardown clears the namespace, the LoadBalancer, the PVC, the forwarding rule and the target pool. The disk behind the PVC is not guaranteed to go with them and is invisible from inside Kubernetes, which is what the second command checks. Anything it lists is yours to delete.
Deploy your own service into it
The environment is a Caproni rig, so your service goes in the way it does
locally: add a deployer to caproni.local.yaml and, for hot reload, an edit
mode repository. Name the checkout for the mix it runs, such as
envs/gitlab-with-orbit. See
Run your service locally with Caproni and the
service fragments in the
gitlab-caproni README.
Agents reach the instance through the GitLab API at the same hostname, and the Hangar repo ships a deploy-gitlab agent skill that drives the same script. A team grants its agent access to the vcluster by creating and binding a ServiceAccount inside its own namespace.
Known limits today
gitlab-migrationsmay report Error on its ClickHouse step. Postgres migrations complete first, so the instance comes up and serves.- One
e2-standard-8node runs about two GitLab environments; the second one’s pods restart a few times under memory pressure before settling. - A
gitlab-capronicheckout outside~/hangar/envs/gets novclusterCLI, becausegitlab-capronidoes not pin it. Clone underenvs/or use the script. - The NordLayer allowlist is a hand-copied snapshot. If connections to your API
server time out and nothing in the cluster looks wrong, compare
curl -s https://ifconfig.mewith the ranges invcluster.values.yaml. - Ownership is per engineer. Team-owned environments, per-team caps and the environment registry are Phase 2.
Everything else the team has run into is in Hangar’s troubleshooting pages. Something unclear or slow? Say so on the feedback issue.
Related reading
- Hangar: the component page and its roadmap entry
- Tenancy model: who owns a namespace, who can reach it, what ends it
- Test a modular feature in isolation: the layer below this, with no cluster at all