Tutorial: Run GitLab with Helm on Kubernetes


../../_images/banner-twitter.jpg

This tutorial shows how to run GitLab Community Edition from your Kubernetes cluster. GitLab Community Edition is an open-source Git server for managing Git repositories.

We use the Helm package manager to install the software. Helm is a tool for managing pre-configured Kubernetes objects. It’s an easy way to install popular software on Kubernetes.

Note that you’ll be using default settings from both Heptio’s AWS Quick Start and Helm, so make sure the security options fit your use case.

Take a look at the GitLab Helm configuration for Kubernetes.

Prerequisites

This tutorial assumes the following:

  • You have a local Linux/Unix environment (like a MacBook)
  • You have successfully created the CloudFormation stacks from the AWS Kubernetes Quick Start (or followed the CLI walkthrough if you prefer)
  • You have copied the resulting kubeconfig file to your local machine, and can successfully run kubectl commands against the cluster (see Step 4 of the setup guide)

Try running kubectl get nodes. Your output should look similar to this:

NAME               STATUS         AGE
ip-10-0-0-0     Ready,master      1h
ip-172-172-172-172 Ready          1h
ip-192-192-192-192 Ready          1h

If it is, your cluster is ready to go! If the output looks substantially different, try running through the setup walkthrough again.

Have a domain name in mind that you would like to use for the GitLab web dashboard. This can be one of the following:

  • a placeholder domain, which you will enter during the GitLab setup, but you will access the dashboard through an assigned AWS domain instead
  • a domain that you would like to use to access the dashboard, meaning that you will need to set up a CNAME DNS entry from your chosen domain name to the AWS domain later in the tutorial, once the AWS domain is generated

1. Install or upgrade Helm locally

On your local machine, install Helm, a popular package manager for Kubernetes.

On OS X, we recommend using Homebrew to install Helm.

brew install kubernetes-helm

If you’ve already installed Helm, it’s a good idea to make sure you’re on the latest version:

brew upgrade kubernetes-helm

Note that the latest Helm release is often a bit ahead of Homebrew, and may address issues from previous releases.

Check the Helm website for instructions on installing Helm for other platforms.

2. Initialize Helm and Tiller

On your local machine, initialize Helm for your Kubernetes cluster. This will install Tiller (the Helm server-side component) onto the Kubernetes cluster and set up your local configuration. This depends on your kubeconfig having the credentials to connect to the Kubernetes cluster, as described in the prerequisites. For more details, see the Helm documentation.

Install Tiller

Initialize Helm and configure Tiller with this command:

helm init

This works with kubectl to connect to your cluster.

Note

If kubectl isn’t configured to connect to your cluster, you might see an error like Error: error installing: Post http://localhost:8080/apis/extensions/v1beta1/namespaces/kube-system/deployments: dial tcp [::1]:8080: getsockopt: connection refused. Check your prerequisites if you see this error.

You should see output like:

$HELM_HOME has been configured at /Users/varMyUser/.helm.

Tiller (the helm server side component) has been installed into your Kubernetes Cluster.
Happy Helming!

If you’ve already installed Tiller on your cluster, it’s a good idea to make sure you’re on the latest version:

helm init --upgrade

Optionally, check your Helm and Tiller versions:

helm version

If they’re not both the latest, run through the previous steps to update Helm and Tiller.

Implement Tiller RBAC fix

At the time of writing, the version of Tiller installed with Homebrew has not caught up to the new default permission structure in Kubernetes 1.6+. On the 1.6+ Kubernetes cluster built by the AWS Quick Start, RBAC is turned on by default, and the default service account for each namespace does not have admin access. Read more about the RBAC updates and a general approach to Kubernetes permission fixes in our RBAC workaround guide. Read more about this specific workaround on the Helm issue.

To give Tiller permission to install and configure resources on our cluster, first we create a new ServiceAccount named tiller in the system-wide kube-system Namespace.

kubectl create serviceaccount --namespace kube-system tiller

Create a ClusterRoleBinding with cluster-admin access levels for this new service account named tiller. Note that these are very broad permissions, because we want Tiller to be able to install and configure a wide variety of cluster resources.

kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller

Edit the existing Tiller Deployment named tiller-deploy.

kubectl edit deploy --namespace kube-system tiller-deploy

This will open the YAML configuration for this deployment in a text editor. Insert the line serviceAccount: tiller in the spec: template: spec section of the file:

...
spec:
  replicas: 1
  selector:
    matchLabels:
      app: helm
      name: tiller
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: helm
        name: tiller
    spec:
      serviceAccount: tiller
      containers:
      - env:
        - name: TILLER_NAMESPACE
          value: kube-system
...

Now Tiller is using the new tiller service account we just created, which has expanded permissions. You should be able to successfully use Helm to install your software now.

3. Install GitLab

Create a new Kubernetes Namespace with:

kubectl create namespace varmygitlab
namespace "varmygitlab" created

This sets up a new namespace in your Kubernetes cluster to contain all the objects for the GitLab site.

Use Helm to install GitLab into your new namespace. This configures everything that GitLab needs to run, including:

Replace http://gitlab.varmydomain.com/ with your chosen domain name.

Note: the command will work with any domain, whether or not it resolves to the cluster. If you have a domain you want to use for accessing GitLab, though, please use that one.

helm install --namespace varmygitlab --name gitlab --set externalUrl=http://gitlab.varmydomain.com/ stable/gitlab-ce

You will see a large amount of output, starting with:

NAME:   gitlab
LAST DEPLOYED: Thu Apr 27 17:17:23 2017
NAMESPACE: varmygitlab
STATUS: DEPLOYED
...

Read through the rest of the output at your leisure. It contains instructions for accessing your site, but these may not be 100% compatible with AWS because of how the Kubernetes stack’s networking is configured. (With AWS, Kubernetes stores a hostname for the load balancer, not the IP.)

We’ll show you how to access your GitLab dashboard in the next step.

4. Connect to your GitLab dashboard

You’ll need to wait 2-10 minutes for GitLab to finish provisioning.

One issue at time of writing: the command Helm gives for finding the site’s IP address may not work. Instead, use the following command to find the URL to the new GitLab dashboard:

echo http://$(kubectl get svc --namespace varmygitlab gitlab-gitlab-ce -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

It should be a lengthy auto-generated URL, like:

http://xxxxxx.us-west-2.elb.amazonaws.com

It will typically take a few minutes for the new site’s DNS to be ready. After this time, you should be able to visit the new site in your browser.

The first time you access GitLab, you will need to change the root password. Then, you can log in as the root user with the following credentials:

  • Username: root
  • Password: the password you created

We recommend that you register a new GitLab account other than root for day to day use of the repository manager.

Log in with the new non-root GitLab user. Then, select the New group or New project button to get started. Read more about using GitLab Community Edition in their documentation.

If you want to set up DNS for your domain, you should now create a CNAME record from your chosen domain, like gitlab.varmydomain.com, to the AWS domain that was generated during the deployment, which will look like xxxxxx.us-west-2.elb.amazonaws.com.

Now you can start start managing Git repos with your GitLab Community Edition installation running on Kubernetes.