Article directory
- Preface
- 1. Gitlab Runner deployment
-
- 1. Obtain Runner registration token
- 2. Register Runner
- 3. Configure Runner
- 2. Configure GitLab CI
- 3. Run the pipeline
- 4. Use scheduled scripts for backup & deployment
- Reference URL
- Summarize
Foreword
The software and versions used in this article are as follows:
Gitlab: 14.6.1; Gitlab-Runner: 16.3.1
This article uses Gitlab CI/CD + Gitlab Runner to realize the function of automatically packaging and deploying springboot projects.
Using automatic CI/CD can reduce the burden of maintenance and avoid problems such as human operation errors.
- Both Gitlab and Gitlab-Runner use Docker for deployment, and Runner also uses Docker as the executor.
- In this article, Runner is only responsible for packaging and outputting to the specified directory. The springboot project is backed up & deployed by a scheduled script.
- This article contains the author’s CI configuration, backup & deployment timing scripts
- Regarding the deployment of Gitlab, this article will not go into details.
1. Gitlab Runner deployment
1. Obtain Runner registration token
Use Gitlab’s administrator account (default is root) to open the Gitlab-Runner management interface in the sequence shown below
Copy the Runner’s Registration Token, which will be used when deploying the Runner later.
2. Register Runner
Use the following command to pull the Runner image. Note: This article does not use the official image
docker pull bitnami/gitlab-runner
Create and run the Runner using the following commands
When the Runner is registered, config.toml will be generated to /etc/gitlab-runner/config.toml, but when it is actually run, /home/gitlab-runner/.gitlab-runner/config.toml is used, so it is mapped to the same file. Easy to modify
-d runs in the background, -names it as [Gitlab_Runner], --restart always starts automatically, -v mounts the host directory
[Host directory] is modified according to the actual situation. The docker.sock path generally does not need to be modified
docker run -d --name Gitlab_Runner --restart always \ -v [host directory]/.gitlab-runner/config.toml:/etc/gitlab-runner/config.toml \ -v [host directory]:/home/gitlab-runner \ -v /var/run/docker.sock:/var/run/docker.sock \ bitnami/gitlab-runner:latest
Next, enter the container as root to perform operations ([Gitlab_Runner] needs to be replaced with the actual container name)
docker exec -it -u root Gitlab_Runner bash
As shown in the figure below, enter the command to start registering the Runner:
Because the author's Gitlab and Runner are under the same Docker network, the Gitlab address here is the intranet address. Please modify it according to the actual situation
Three configurations that can be empty and can be modified in Gitlab at any time
gitlab-runner register
3. Configure Runner
After the Runner is registered, check the corresponding files in the host as follows:
docker in docker mounting path needs to be modified according to the actual situation, the syntax is consistent with the docker instruction: host path: path within the container
Please note that the former path here is the path of the host, not the path of the Runner container
Under [runners.docker], add: pull_policy = “if-not-present” to enable the Runner to pull only when the image does not exist, as follows:
pull_policy = "if-not-present" #Default is always, optional: always, if-not-present, never
It is recommended to map maven and its warehouse path so that the local warehouse can be reused. The configuration is as follows:
volumes = ["/cache", "[maven path]:/root/.m2", "[jar package storage path]:/app/build"] # Note: The maven warehouse path is: [maven path]/repository
The author has configured multiple caches, as follows:
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
After restarting the Runner container, you can see the newly registered Runner in Gitlab
At this point, Gitlab Runner has been deployed and configured.
2. Configure GitLab CI
As shown below, enter the CI configuration file editing interface, and then modify the preset CI configuration file:
Gitlab has provided a large number of templates for reference, including maven; there are also detailed syntax instructions in the official documentation. Therefore, this article will not go into details, but only introduces the author's CI configuration ideas
Due to actual needs, the author did not use Runner to encapsulate the project into a Docker image and run it, but only output the jar package. Actions such as backup and deployment are performed by scheduled tasks.
So the CI configuration only executes mvn package and moves the jar package to the specified directory, so only the build phase is needed
The author’s CI configuration example is as follows:
# Script variables
variables:
# Project port number
PORT_NUM: "[port number]"
# Information printed during the build phase
BUILD_PREFIX_INFO: "$CI_COMMIT_BRANCH build started\
Build creator: $GITLAB_USER_NAME\
Branch committer: $CI_COMMIT_AUTHOR\
Packaging start time: $CI_PIPELINE_CREATED_AT"
BUILD_SUFFIX_INFO: "$CI_COMMIT_BRANCH build completed"
# Project file root directory
# "[jar package storage path]:/app/build"] in the Runner container specifies that /app/build is mapped to the host machine
BASE_PATH: "/app/build/$CI_PROJECT_NAMESPACE/$CI_PROJECT_TITLE"
# New package path
PACKAGE_PATH: "$BASE_PATH/new"
# New package name: project name-port number.jar
PACKAGE_NAME: "$CI_PROJECT_TITLE-$PORT_NUM.jar"
# Backup path
BACKUP_PACKAGE_PATH: "$BASE_PATH/backup"
# maven packaging parameters
MAVEN_PACKAGE_OPTS: "-Djansi.passthrough=true -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8 -Dsun.stderr.encoding=UTF-8
-DskipTests=true -Dmaven.test.skip=true"
# During the construction phase, package the project and place it in the corresponding path, and the server will automatically deploy the script for backup and deployment.
# Jar package naming rules: project name-port number.jar
#Final product file structure:
# / Project file root directory, that is, /app/build/
# ╚═══╦══Group or username/
# ╚═══╦══Project name/
# ╠═══╦══new/
# ║ ╚══════new.jar output jar package
# ╠═══╦══backup/
# ║ ╚══════backup.jar
# ╚══════old.jar
build: # stage name, can be modified arbitrarily
stage: build
# Only the master branch is executed
only:
- master
before_script:
- echo -e "$BUILD_PREFIX_INFO"
script:
- mvn $MAVEN_PACKAGE_OPTS clean package
- if [ ! -d $PACKAGE_PATH ];
then echo -e "Create a new package path for the project"; mkdir -p $PACKAGE_PATH;
fi
- if [ ! -d $BACKUP_PACKAGE_PATH ];
then echo -e "Create project backup path"; mkdir -p $BACKUP_PACKAGE_PATH;
fi
- if [ -f target/*.jar ];
then cp target/*.jar $PACKAGE_PATH/$PACKAGE_NAME; # Output the jar package to [jar package storage path]
else echo -e "New project package not found, migration will not be performed"; exit 1;
fi
# If you do not need to output the jar package to the Gitlab pipeline product, you do not need the following steps
- cp target/*.jar $CI_PROJECT_DIR/$PACKAGE_NAME # Output to the Gitlab artifact directory
# Output the packaged jar package to Gitlab pipeline product
artifacts:
name: $CI_PROJECT_NAMESPACE-$CI_PROJECT_TITLE-$CI_COMMIT_BRANCH
paths:
- $CI_PROJECT_DIR/$PACKAGE_NAME
expire_in: 1 week
after_script:
- echo -e "$BUILD_SUFFIX_INFO"
3. Run the pipeline
When the .gitlab-ci.yml file exists in the git root directory of the project, if the relevant branches are submitted or merged, Gitlab will automatically create and execute the pipeline and related jobs.
The picture below shows a successful job. The generated jar package is also found in the [jar package storage path] folder of the host machine:
4. Use scheduled scripts for backup & deployment
An example of the file structure of the author’s server is as follows:
Therefore, the timing script needs to have the following functions:
Iterate through all items
Project Y
Continue to traverse other items in the group
Continue to traverse other groups
Iterate through all groups
GroupX
Out of service
old.jar migrated to backup
Migrate new.jar to the project Y root directory
Start service
The author’s scheduled backup & deployment script is as follows:
BaseDir="[parent directory of group folder]"
BackupDirName="backup"
NewPackageDirName="new"
LogDirName="[Log]" # Log folder name, skip when traversing
RunScriptName="[start and stop script].sh"
StopCommand="stop" # Stop service command
StartCommand=" start" # Start service command
cd $BaseDir
echo "************Start deploying************"
if (ls -A $BaseDir/ | grep -q ".")
then
for groupName in $(ls -A $BaseDir/)
do
echo -----Start checking group:$groupName-----
for projectName in $(ls -A $BaseDir/$groupName)
do
thisBaseDir=$BaseDir/$groupName/$projectName
projectPath=$groupName/$projectName
if [ $projectName == $LogDirName ]
then continue
fi
if [ ! -d $thisBaseDir/$BackupDirName ]
then echo $projectPath Backup folder not created; mkdir -p $thisBaseDir/$BackupDirName;
fi
if [ ! -d $thisBaseDir/$NewPackageDirName ]
then echo $projectPath did not create new package folder; mkdir -p $thisBaseDir/$NewPackageDirName;
fi
if [ -f $thisBaseDir/$NewPackageDirName/* ]
then
New files have been found in echo $projectPath and deployment begins;
cd $thisBaseDir;
if [ -f $thisBaseDir/$RunScriptName ]
then $thisBaseDir/$RunScriptName$StopCommand;
else echo $projectPath not found stop script! ;
fi
if [ -f $thisBaseDir/*.jar ]
then
for file in $thisBaseDir/*.jar
do
# Backup file name format: old_YYYYmmdd-HHMM.jar
mv $file $thisBaseDir/$BackupDirName/$(basename ${<!-- -->file%.*})_$(date + %Y%m%d-%H%M).jar;
done
echo "Old package backup completed";
else echo $projectPath did not find the old package, no backup required;
fi
mv $thisBaseDir/$NewPackageDirName/*.jar $thisBaseDir/; #The situation that the new package does not exist has been ruled out
echo "New package migration completed";
if [ -f $thisBaseDir/$RunScriptName ]
then $thisBaseDir/$RunScriptName$StartCommand;
else echo $projectPath startup script not found! ;
fi
echo $projectPath deployed;
else echo No new file found in $projectPath;
fi
done
echo -----End checking group:$groupName-----
done
else
echo "No new files found"
fi
echo "************Deployment completed************"
exit 0
The author’s java service start and stop script is as follows:
#!/bin/bash
# jar package file name
PROJECT_NAME="[jar package file name].jar"
# jdk path, the author did not install the java environment, but directly used the java in the jdk folder
JavaBaseDir="[jdk path]/jdk/bin"
function check() {<!-- -->
PID=$(ps aux | grep $1 | grep -v grep | awk '{print $2}')
if [[ "${PID[@]}" != "" ]];then
echo "Service already exists: $1"
exit 1
fi
}
function check_start() {<!-- -->
PID=$(ps aux | grep $1 | grep -v grep | awk '{print $2}')
if [[ "${PID[@]}" != "" ]];then
echo "Service started successfully: $1"
else
echo "Service startup failed: $1"
fi
}
function stop_service() {<!-- -->
PID=$(ps aux | grep $1 | grep -v grep | awk '{print $2}')
if [[ "${PID[@]}" != "" ]];then
ps -ef | grep $1 | grep -v grep | cut -c 9-15 | xargs kill -9
test $? -eq 0 & amp; & amp; echo "Service shutdown successful: $1"
else
echo "Corresponding service not found: $1"
fi
}
function start_service() {<!-- -->
check ${<!-- -->PROJECT_NAME}
# Run the service. If the java environment has been configured, you can use the java command directly.
nohup $JavaBaseDir/java -Xms1000m -Xmx1000m -XX:-UseGCOverheadLimit -Dfile.encoding=utf-8 -jar ${<!-- -->PROJECT_NAME} >/dev/null 2> & amp;1 & amp;
check_start ${<!-- -->PROJECT_NAME}
}
case $1 in
start)
start_service
;;
stop)
stop_service ${<!-- -->PROJECT_NAME}
;;
restart)
stop_service ${<!-- -->PROJECT_NAME}
sleep 1
start_service
;;
*)
echo "Usage: bash $0 start | stop | restart"
;;
esac
The output of one of my timing scripts is as follows:
Reference URL
Official documentation
Gitlab CICD environment variables
GitLab CI/CD + GitLab Runner in Docker fully automatic deployment server web page
Automated build: gitlab gitlab-run, maven cache and gitea drone drone-run
Summary
At this point, Gitlab’s function of automatically deploying springboot projects has been implemented. Please choose what kind of Runner executor, CI/CD content, etc. to use based on your actual needs.
Thank you for reading.