Create a CI/CD environment with Docker

Let’s talk about creating a continuous integration/continuous deployment with gitlab/gitlab-runner/docker.

If you don’t know what CI/CD is, let me just explain it like that:  CI is the part where you will build automatically the changes pushed by the developers and CD is the automated deployment of a software when after a build (for example). It’s maybe not totally accurate or maybe only a part of what CI/CD covers but you can get an idea why I find it cool.

Why Docker? Because it can be reproduced on someone else’s machine and this is helpful if  a colleague want to play with that for example and it can be less painful to deploy that infrastructure on a server as well.

Goal

Our goal today is deploying a CI/CD infrastructure which will deploy our dockerized application on a docker-machine every time we push a change.

You can find below a schema of the flow and the different actors.

(Yep, not beautiful)

Let’s deploy the different elements

Let’s presume you have docker already installed on your machine. You will have to deploy three different elements:

  • a Gitlab server which contains our git repository
  • a Gitlab-runner which builds automatically our test software
  • a private docker registry which holds our docker images

For the gitlab server, here is my command (please, note that the server can take several minutes before being up):

docker run --detach \
--hostname my.domain.com \
--publish 4443:443 --publish 8000:80 --publish 2222:22 \
--name gitlab \
--volume gitlab/config:/etc/gitlab \
--volume gitlab/logs:/var/log/gitlab \
--volume gitlab/data:/var/opt/gitlab \
gitlab/gitlab-ce:latest

Now, let’s do the same for the gitlab-runner:

docker run -d --name gitlab-runner \
-v gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
-v .docker/certs.d:/etc/gitlab-runner/certs \
gitlab/gitlab-runner:latest

For our docker registry, before deploying it, you need to create a certificate:
openssl req -newkey rsa:2048 -nodes -keyout domain.key -x509 -days 365 -out domain.crt
Openssl will ask you some questions, answer them and now you can enjoy your self-signed certificate and you can deploy your register:

docker run -d --name registry \
-e TCP_PORTS="5000" -e VIRTUAL_HOST="*:5000, https://*:5000" \
-e FORCE_SSL="true" -e REGISTRY_STORAGE_DELETE_ENABLED="true" \
-v registry/certs:/certs \
-e REGISTRY_HTTP_ADDR=my.registry.com:5000 \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
-e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
-e ENV_DOCKER_REGISTRY_USE_SSL=1 \
-p 5000:5000 registry:latest

And finally, our target machine which is in this case a docker-machine:

docker-machine create --driver virtualbox target

You will have to connect to it and add an entry in /etc/hosts. Put your registry ip for my.registry.com URL (docker-machine is not in the same VLAN as the registry and a regitry needs to be accessed with its FQDN and not an IP).

Now that we have our different elements running, we can continue.

Shitty code creation

Before pushing our code on the repository, we need a code. You can choose whatever language you want, it’s not important.

Note that we need a dockerfile for our use case. Here is mine:
FROM golang:latest
ADD . /code
WORKDIR /code/
RUN go build main.go
EXPOSE 8080
CMD ["/code/main"]

Push also your registry certificate. Our runner will push our dockerized application to the registry. In order to do that, we need to update its trusted CA by adding domain.crt to it. A way to do that is to put the crt file inside the repo because it is accessible to the runner. Another way (but haven’t tested it yet) is to put the certificate in a environment variable inside gitlab.
And here is the final structure of my project:

Gitlab config

Now that we have a fresh gitlab server, we need to configure it. Basically, for your first visit, gitlab will ask you to change your password. You can afterwards create a new user.

Once it’s done, create a new git repository and follow the instructions. At the end, you should have your code and the Dockerfile in your repo.

We need to configure as well some environment variables in order to deploy flawlessly. First, grab your docker-machine client certificate/client key/CA certificate (you can see that with docker-machine env target). Then  go to Settings > Secret variables and fill up the key/value fields with the informations you have.

Finally, you will have to fetch your registration token from Settings > CI/CD > Runner Settings. You will need it when we will register our gitlab-runner in the next step.

Run for your life

Let’s configure a simple runner.
First, you need to have a shell:
docker exec -ti gitlab-runner /bin/bash

Register a runner:
sudo gitlab-runner register -n \
--url http://my.domain.com/ \
--registration-token YOUR_REGISTRATION_TOKEN \
--executor docker \
--description "POC CICD" \
--docker-image "docker:latest" \
--docker-volumes /var/run/docker.sock:/var/run/docker.sock \
--docker-volumes /etc/gitlab-runner/certs:/etc/docker/certs.d

As you can see, this is a “docker in docker” configuration meaning that the docker gitlab-runner will spawn another docker runner for the build.

At this step, we now have a git repository on our gitlab server, a runner which is registered on the gitlab. We now need to tell to the runner how to build our project.

Aaaaand finally

Create a .gitlab-ci.yml file inside your application tree. Mine seems like this:

image: docker:latest

stages:
  - build
  - deploy

services:
- docker:dind

build:
  stage: build
  script:
  - cp my.registry.com.crt /usr/local/share/ca-certificates/my.registry.com.crt
  - mkdir -p /etc/ssl/certs
  - update-ca-certificates
  - docker build -t my.registry.com:5000/go_test:latest .
  - docker push my.registry.com:5000/go_test

deploy_prod:
  stage: deploy
  script:
    - mkdir $DOCKER_CERT_PATH
    - echo "$CA" > $DOCKER_CERT_PATH/ca.pem
    - echo "$CLIENT_CERT" > $DOCKER_CERT_PATH/cert.pem
    - echo "$CLIENT_KEY" > $DOCKER_CERT_PATH/key.pem
    - docker stop go_test && docker rm go_test
    - docker run -d --name go_test -p 8001:8080 my.registry.com:5000/go_test:latest # modified /etc/hosts on the docker machine in order to have my.registry.com = 192.168.99.1 = my local machine => because docker-machine could not communicate with private registry (not the same VLAN) + had to be a FQDN and not an IP...
    - rm -rf $DOCKER_CERT_PATH
  variables:
    DOCKER_TLS_VERIFY: "1"
    DOCKER_HOST: "tcp://<YOUR_DOCKER_MACHINE_IP>:2376"
    DOCKER_CERT_PATH: "certs"
  environment:
    name: production
  when: manual

Add it to your repo and push that. Normally, you should have a CI/CD infrastructure with one environment (the target machine which is our “prod”). You have to go to the environment tab in gitlab in order to deploy (note the when: manual inside my gitlab-ci file).

I hope it works for you (had a lot of painful moments in order to make it works). Let me know if you have some questions, I will try to answer them.

Ciao!

Leave a Reply