Deploy gitlab-runner

CI : Continuous Integration Continuous Integration
Developers frequently “integrate” local code into the trunk branch every day and need to ensure that the trunk branch is available. These new submissions need to be verified through compilation and automated testing flows before they are finally merged into the main line. In the traditional R&D process In general, compilation, testing, and deployment are performed manually. After automation, the frequency of integration can be greatly increased and low-level problems caused by maintaining manual scripts can be reduced.

CD：Continuous Delivery Continuous Delivery
The next step of continuous integration is continuous delivery. After CI completes the automated processes of construction, unit testing and integration testing, continuous delivery can automatically publish the verified code to the enterprise’s own repository. In order to achieve an efficient continuous delivery process, it is important to ensure that CI is built into the development pipeline (pipeline automation pipeline). The goal of continuous delivery is to have a code base that is ready to be deployed to production. At the end of the process, operations teams can quickly and easily deploy the application to production.

CD: Continuous Deployment Continuous Deployment
For a complete CI/CD pipeline, the final stage is continuous deployment. The next step in continuous delivery is continuous deployment, which “automatically” deploys code to the production environment. Continuous deployment means that changes made by developers to the application can take effect within a very short time after being integrated into the main branch (provided that it passes automated testing), which can greatly accelerate the feedback loop with customers.

This blog post will first talk about the CI tool gitlab-runner. CD tools such as Argo CD will be introduced in other blog posts.

1. What is gitlab-runner

Everyone must have understood the meaning of CI. Gitlab CICD is a tool for continuous deployment. gitlab-runner cooperates with gitlab CICD to achieve automation. gitlab-runner is a machine used to execute CI/CD tasks (jobs) on gitlab. This machine can be a server, docker or k8s, etc. gitlab-runner is written in Go language and has only one binary file and does not require other dependencies. Easy to install and use.

Three types of Runner

1. shared: Run jobs for the entire platform project (gitlab)
2. group: Run jobs for all projects under a specific group (group)
3. specific: Run the specified project job (project)

Two states of Runner

1. locked: Unable to run project job
2. paused: The job will not run

Runner’s executor Executor

Runner is only responsible for reading the task (job) content to the machine where it is located. But it is not the Runner that actually performs the task, but the Executor. When registering the Runner, you will be asked to select an executor, such as shell, docker, k8s, etc. Today we will mainly talk about shell and docker.

Tags for Runner Tags

When registering a Runner, you can add a label to it. After adding the tag, you can call the specific Runner through the tag in the CI/CD configuration file (gitlab-ci.yaml).

2. Deploying gitlab-runner on Linux (CentOS)

Automatic installation

1. Add gitlab repository

curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash

2. Install gitlab-runner

yum install -y gitlab-runner

Manual installation

1. Download the gitlab-runner installation package

# Replace ${arch} with amd64, arm, arm64
curl -LJO "https://gitlab-runner-downloads.s3.amazonaws.com/latest/rpm/gitlab-runner_${arch}.rpm"

2. Install gitlab-runner

# Replace <arch> with amd64, arm, arm64
rpm -i gitlab-runner_<arch>.rpm

3. Check whether gitlab-runner is installed successfully

gitlab-runner --version

3. Docker deployment gitlab-runner

1. Install the Docker image and start the container,

# This example uses the local system as the configuration volume mounted into the gitlab-runner container. This volume is used for configuration and other resources.
docker run -d --name gitlab-runner --restart always \
  -v /srv/gitlab-runner/config:/etc/gitlab-runner \
  -v /var/run/docker.sock:/var/run/docker.sock \
  gitlab/gitlab-runner:latest

2. View container running status

docker ps | grep gitlab-runner

4. Register Runner

1. First find Runners under the settings –> CI/CD directory of gitlab

The picture above shows Specific type Runners. You can also choose Group (in the same directory of the selected group page) or Shared type Runners. We will need to use Url and token for registration later.

2. linux Server registration runner

gitlab-runner register

Enter Register the runner with this URL and registration token as shown above

Then enter the name of the Runer, Tag, optional maintenance instructions, and then select an executor (currently selected shell) type.

You can see on the Runners configuration page that the registration is Runner

You can also use non-interactive mode to register the runner with additional parameters

gitlab-runner register --non-interactive \
--executor "shell" \ #executor
--url "http://192.168.37.13/" \ #Corresponding to gitlab address
--registration-token "uo7sXcM4F66XA-6qv6GM" \ #corresponds to the token shown below
--description "caitlin-demo" \ #runner name
--tag-list "caitlin" \ #runner tag
--run-untagged="true" \
--locked="false" \ # runner status
--access-level="not_protected"

After Runner registration is completed, a config.toml file will be generated in the /etc/gitlab-runner directory. This is the runner configuration file.

3. Register Runner in docker container

You can go inside the container and follow the steps above to register the Runner.

# Enter container command
docker exec -it gitlab-runner bash

You can also use non-interactive mode to register the runner with additional parameters

docker run --rm -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register \
  --non-interactive \
  --url "https://gitlab.com/" \
  --registration-token "$PROJECT_REGISTRATION_TOKEN" \
  --executor "docker" \
  --docker-image alpine:latest \ #When selecting docker as the executor, you need to enter the base image
  --description "docker-runner" \
  --maintenance-note "Free-form maintainer notes about this runner" \
  --tag-list "docker,aws" \
  --run-untagged="true" \
  --locked="false" \
  --access-level="not_protected"

5. Test Runner to run CI/CD task pipeline

Create a new project, and then submit a .gitlab-ci.yml file in the project root directory. When submitted, the pipeline will be triggered. I will not introduce the syntax of the .gitlab-ci.yml file in detail here.

stages:
  -maven
  - build
  -deploy
  
.maven_job:
  Stage: maven
  script:
    - echo "Package project"
    
.build_job:
  stage: build
  script:
    - echo "build project"

.deploy_job:
  stage:deploy
  script:
    - echo "deployment project"

You can see the status of the pipeline in the Pipelines directory under CI/CD of the gitlab project.

Summary

At this point, the introduction, installation and simple application of gitlab-runner are completed. I will continue to write the entire CICD automation blog post based on gitlab argocd k8s later, so please pay more attention to it. If there are any shortcomings, please feel free to enlighten me.