Continuous Integration using Gitlab CI with Docker

Stories about project management and coding

Continuous Integration using Gitlab CI with Docker

Gitlab.com

Most people I know prefer Github or a private server for hosting their private git repositories.
Nothing wrong with that, everyone has their own workflow.
I’ve been using Gitlab for a few years now, privately hosted. A few months ago, my Gitlab server had some problems after upgrading to a newer version. I needed the repositories back online fast, so I switched to using gitlab.com – and never looked back. A free account on gitlab.com gives you most features you’ll ever want, like built-in CI, issue tracker or even cycle analytics.
This wont be another Github vs. Gitlab article, in this post I want to highlight one of the features I’ve come to enjoy the most, Gitlab CI.

What we’re gonna build

For most of my projects I use Maven for building the packages and even for building the docker images.
I was always to lazy to set up a continuous integration system for my private projects, mostly because I was the only one commiting to the repositories, so nothing to integrate. For a recent project I was working together with someone and wanted automate the Junit tests and builds to catch any discrepancies early.

For me, the most annoying part of setting up unit tests was, that if you wanted to run tests against the database, you’d also have to manage that stuff. Still, it’s well worth the effort to make sure stuff like migration scripts work. Sure, you can just use a H2 in-memory database, which is fine for most simple databases, but as soon as you want to use specific database types (like Postgres JSONB), you’re stuck.
One of the advantages of Gitlab CI is that you can use docker images to run your test. So instead of setting up a special database instance / schema for our unit tests, we can just pull in a Postgres Docker image, spin it up and run our tests against that one!

You’ll also have to choose a docker image compatible to your production environment, because Gitlab CI will run the tests in them.

Setup

So to test the features we’ll talk about here, go to gitlab.com, register a free user and push a repository with a maven enabled project.
Got to your repository in the Gitlab web UI and click on Pipeline. Gitlab will help you setup your config.

.gitlab-ci.yml

If you add a .gitlab-ci.yml  (notice the preceding dot) file to the root directory of your repository, and configure your GitLab project to use a Runner, then each commit or push triggers your CI pipeline.
In GitLab CI, Runners run the code defined in .gitlab-ci.yml . They are isolated (virtual) machines that pick up jobs through the coordinator API of GitLab CI.

A Runner can be specific to a certain project or serve any project in GitLab CI. A Runner that serves all projects is called a shared Runner.

We’ll use a shared runner, running docking. You can see which runners are available for your project by going to Setting -> Pipelines.

Gitlab CI without config

You can define when you want to trigger the build, on stuff like push to a specific branch or when pushing a tag or just timed.

To configure the build pipelines in Gitlab CI, create a file .gitlab-ci.yml  in the root of your repository.

The .gitlab-ci.yml  file tells the GitLab runner what to do. By default it runs a pipeline with three stages: build, test, and deploy.
You don’t need to use all three stages; stages with no jobs are simply ignored.

You’ll also have to choose a docker image compatible to your production environment, because Gitlab CI will run the tests in them.

As I use Maven 3.5 with JDK8 I’ll use the tag maven:3.5.0-jdk-8-alpine , but have a look at Docker Hub to find one that’s right for you.
Next, I want to make use of the new parallel query features in Postgres 9.6, so I’ll run my test against the tag postgres:latest .

Example

Here’s an example .gitlab-ci.yml  – it’s a modified version of the maven template available on gitlab.com:

Let’s go through the example line by line:

  • 1. This tell Gitlab CI what docker image we want our script to run on
  • 3. Here we define any docker images our tests depend on. In this case, the latest available Postgres image.
  • 6. When receiving a job from GitLab CI, the Runner prepares the build environment. Variables allow you to configure certain aspects of this environment.
  • 8-10 The database and credentials config for our Postgres docker image. Naturally, your tests need to be configured to use these credentials, for more see below. Docker will provide use with a hostname, it will just be “postgres”
  • 14-18 Common maven options
  • 22. Caching the dependencies will speed up the build times.
  • 24. A job definition, the job names are arbitrary.

Put this .gitlab-ci.yml  into your repository root, push it and in the Gitlab web UI, go to Pipelines.
As soon as Gitlab sees the config it will run the pipeline.
Gitlab CI runs a linter on your config, so if you have a typo in it you’ll notice it immediately.

Gitlab CI Pipeline

To view the log of the running process click on “Running”

Gitlab CI Log View

Bonus – configure your tests in a Spring project to use the CI database

I make use of Spring Profiles here, as you can see in the parameter -DargLine="-Dspring.profiles.active=ci  above.
This tells Spring to use the profile CI. For more on Spring Profiles see my post here.
I have a file application-ci.properties  under src/test/ressource :

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *