Deploying a Gitlab Runner on AWS with Packer

Introduction

Gitlab.com allows its users to run their own, self-hosted, Gitlab Runners as part of their Gitlab CI feature. This comes in handy when the CI jobs have to run in a user-owned environment or more powerful runners are needed.

If that environment is AWS and you are looking for an automated way of setting up the Runner(s) on EC2, then stay tuned for a 56k.cloud recipe!

Requirements

a gitlab.com account
a repository that needs a CI / CD set up
the Gitlab runner registration token. This can be found in the above mentioned repository settings -> CI / CD -> Runners -> Specific Runners -> Set up a specific runner manually
install Hashicorp's Packer on your build machine

Steps

clone the 56k.cloud Packer repository:

git clone https://gitlab.com/56k/packer.git

build the Gitlab runner image using Packer:

cd packer/gitlab-runner
packer build -var 'aws_access_key=<your_access_key_id>' -var 'aws_secret_key=<your_access_key>' gitlab-runner.json

login the AWS Console and check the EC2 -> AMIs section to confirm that the new Gitlab runner AMI is there. Make a note of the AMI ID.
launch a new EC2 instance using the AMI ID found in the previous step. Call the following script from the User data section of EC2's instance launch options to register the runner with gitlab.com:

/home/ubuntu/gitlab-runner-register.sh <environment (e.g.: dev)> <gitlab_cicd_token>

going back to the repository's settings -> CI / CD -> Runners -> Specific Runners, this new runner should be available in the list. Click on Enable for this project to make the runner usable by the repository.

Next steps

At this point, the Gitlab runner is attached to the project. When defining the Gitlab CI / CD pipeline in .gitlab-ci.yml each job will have to use a tag to run that job on the new Gitlab runner. This tag matches the environment passed to the gitlab-runner-register.sh script above. Example:

job:
  script:
    - echo "Hello world!"
  tags:
    - <environment (e.g. dev)>

Summary

Now that we have a reusable image (AMI) for a self-hosted Gitlab runner, it can be used as a building block for further automation.

One possible next improvement is to use this AMI and the User Data script in an infrastructure-as-code DSL like Terraform or Cloudformation. But that's a subject for a future blog post!

Find out more about 56K.Cloud

We love Cloud, Containers, DevOps, and Infrastructure as Code. If you are interested in chatting connect with us on Twitter or drop us an email: info@56K.Cloud We hope you found this article helpful. If there is anything you would like to contribute or you have questions, please let us know!