Skip to content
Get a remote GitLab environment with Hangar

Get a remote GitLab environment with Hangar

Hangar gives you an isolated vcluster on a shared GKE host, and Caproni deploys GitLab into it exactly as it does on your laptop. The result is a production-like CNG GitLab at https://gitlab.<namespace>.<baseDomain>, reachable by you in a browser or by an agent through the API, that ends itself after a TTL.

This page is the paved path through Hangar’s own runbook, deploying GitLab to a remote vcluster. Each step links the section that explains it; where this page and Hangar’s docs disagree, theirs are current.

Reach is controlled at the network layer, not per environment: anyone on NordLayer can open any environment on the host. Treat yours as shared with the company. Do not put real credentials or customer data in it.

Before you start

Access. You need to be a tenant on the host, which is granted by hand today. Open an access request issue and work through its checklist: sign in once at gitlabsandbox.cloud so you have an identity to bind, and request NordLayer Permission:Dedicated Gateway through Lumos (the Public WiFi permission does not reach the host). An operator grants you roles/container.clusterViewer and adds you to the caproni-tenant binding.

Tooling. gcloud with the GKE auth plugin, and mise. Hangar’s .mise.toml pins caproni, vcluster, helm and kubectl at versions known to work together, and a checkout under ~/hangar/envs/ inherits that pin.

Identity check. Connect to NordLayer, fetch credentials, and confirm who the cluster thinks you are:

gcloud components install gke-gcloud-auth-plugin   # once per machine
gcloud container clusters get-credentials hangar-host \
  --zone us-central1-a --project caproni-remote-envs-4fb1f3fd
kubectl auth whoami                              # your GCP IAM member
kubectl auth can-i '*' '*' --all-namespaces      # want: no

The identity printed is what the ownership policy stamps on your namespace. If the last command prints yes, say so on your access request issue before you provision anything. See before you start.

1. One checkout per environment

The environment’s name is derived from the checkout directory, and the namespace must equal the vcluster’s name for the host’s generated route to point at it. So one checkout is one environment, and a second environment is a second checkout, not an edited config. Name the directory for what it is.

Or from ~/hangar, ./scripts/gitlab-env.sh up my-feature does this step and the next, and ./scripts/gitlab-env.sh list shows what you have running. See setup using the script.

git clone git@gitlab.com:gitlab-org/quality/hangar.git ~/hangar
git clone git@gitlab.com:gitlab-org/gitlab-caproni.git ~/hangar/envs/my-feature
cd ~/hangar/envs/my-feature
mise trust && mise install
cp ~/hangar/vcluster.values.yaml .
cp ~/hangar/examples/caproni.local.yaml .
export CAPRONI_HOST_CONTEXT=gke_caproni-remote-envs-4fb1f3fd_us-central1-a_hangar-host

Keep the directory name under 17 characters; longer names are cut. The two copied files are Hangar’s overrides for a gitlab-caproni checkout: the namespaced syncer RBAC and API server source ranges, and a trimmed caproni.local.yaml that swaps the dev CA for the host’s wildcard certificate, adds a ClusterIP Traefik inside the vcluster, and turns HTTPS on for the GitLab hostnames. Nothing in them needs editing for the current host. What each override is for is in deploying GitLab to a remote vcluster.

2. Bring it up

From the checkout root:

caproni config validate && caproni up

Validate first: it catches a needs entry pointing at a deployer the override removed, which is the easy mistake. GitLab serves 5 to 10 minutes after caproni up returns. To watch it, from ~/hangar:

./scripts/gitlab-env.sh wait my-feature

Expected while it starts: ERR command error command="kubectl version ..." lines until the API server answers, webservice and sidekiq restarting while gitlab-migrations runs, and gitlab-runner crash-looping until GitLab answers. If you see Unable to connect to the server: context deadline exceeded right after cluster started successfully, wait a couple of minutes and run caproni up again.

3. Check it is yours, and not public

Your namespace and URL come from the checkout (yq is pinned by gitlab-caproni, so step 1’s mise install provides it). The script saves both to hangar.env in the checkout.

NS=$(caproni config print | yq .cluster.vcluster.namespace)
URL=https://$(caproni config print | yq .cluster.network.hostname)
kubectl get ns -l caproni.gitlab.io/managed=true \
  -o custom-columns='NAME:.metadata.name,OWNER:.metadata.annotations.caproni\.gitlab\.io/owner'
kubectl -n $NS get rolebindings       # caproni-owner-admin, caproni-owner-vcluster-rbac
kubectl -n $NS get svc $NS -o jsonpath='{.spec.loadBalancerSourceRanges}'

Your namespace should list your address as owner, and the two RoleBindings are generated by the policy, not by you. The last command is the one with no symptom when it fails: it must print the NordLayer gateway ranges. If it prints nothing, your vcluster’s API server is open to the internet. Run caproni destroy and raise it in Hangar’s issues rather than leaving it up.

4. Reach it

./scripts/gitlab-env.sh status my-feature            # from ~/hangar
curl -kso /dev/null -w '%{http_code}\n' $URL/-/readiness   # or by hand: 200
mise run gitlab:rootpassword                         # from the checkout
caproni kubectl get ns                               # talks to your own API server

Open $URL in a browser on NordLayer and sign in as root with that password. Certificates come from Let’s Encrypt staging until a team onboards, so expect a trust warning; read the issuer rather than the exit code. Inside your vcluster you are cluster-admin. On the host you hold admin on your own namespace and nothing else.

5. Set how long it lives

The reaper destroys an environment 24 hours after its vcluster was created, whether or not you are still in it. Ask for longer, up to a 720-hour cap, at up or any time before it expires:

./scripts/gitlab-env.sh up my-feature 168h      # at up
./scripts/gitlab-env.sh ttl my-feature 7d       # any time after
kubectl annotate ns $NS caproni.gitlab.io/ttl=168h --overwrite   # without the script

A TTL is a positive integer followed by s, m, h or d. Expiry counts from creation, not from the annotation, so setting a TTL later does not restart the clock, and a shorter one can end the environment at the next sweep. How the cutoff is derived and clamped is in reaping environments.

6. Tear it down

From ~/hangar:

./scripts/gitlab-env.sh list                    # what you still have up
./scripts/gitlab-env.sh destroy my-feature      # only that one

The checkout stays, so up with the same name brings the same environment back. Without the script, from the checkout:

caproni destroy
gcloud compute disks list --project caproni-remote-envs-4fb1f3fd \
  --filter="name~'^pvc-' AND -users:*"

The host is shared and LoadBalancers bill by the hour, so do run one of these. Teardown clears the namespace, the LoadBalancer, the PVC, the forwarding rule and the target pool. The disk behind the PVC is not guaranteed to go with them and is invisible from inside Kubernetes, which is what the second command checks. Anything it lists is yours to delete.

Deploy your own service into it

The environment is a Caproni rig, so your service goes in the way it does locally: add a deployer to caproni.local.yaml and, for hot reload, an edit mode repository. Name the checkout for the mix it runs, such as envs/gitlab-with-orbit. See Run your service locally with Caproni and the service fragments in the gitlab-caproni README.

Agents reach the instance through the GitLab API at the same hostname, and the Hangar repo ships a deploy-gitlab agent skill that drives the same script. A team grants its agent access to the vcluster by creating and binding a ServiceAccount inside its own namespace.

Known limits today

  • gitlab-migrations may report Error on its ClickHouse step. Postgres migrations complete first, so the instance comes up and serves.
  • One e2-standard-8 node runs about two GitLab environments; the second one’s pods restart a few times under memory pressure before settling.
  • A gitlab-caproni checkout outside ~/hangar/envs/ gets no vcluster CLI, because gitlab-caproni does not pin it. Clone under envs/ or use the script.
  • The NordLayer allowlist is a hand-copied snapshot. If connections to your API server time out and nothing in the cluster looks wrong, compare curl -s https://ifconfig.me with the ranges in vcluster.values.yaml.
  • Ownership is per engineer. Team-owned environments, per-team caps and the environment registry are Phase 2.

Everything else the team has run into is in Hangar’s troubleshooting pages. Something unclear or slow? Say so on the feedback issue.

Related reading

Last updated on