CI/CD: GitLab-CI automated integration/deployment JAVA microservice application collection

CI/CD: GitLab-CI automated integration/deployment application collection for JAVA microservices

  • CI/CD: GitLab-CI automated integration/deployment JAVA microservice application collection
    • Install
      • Docker
      • GitLab
      • GitLab-Runner
      • Alibaba Cloud Container Warehouse
    • GitLab-CI
      • GitLab-CI application for Java microservices
    • other problems
      • Maven local repository cache

CI/CD: GitLab-CI automated integration/deployment application collection for JAVA microservices

In daily development, every time the code is written, it needs to be manually packaged and uploaded to the server. Regardless of the local packaging time or uploading files to the server, it takes a lot of time to complete. It is repetitive and meaningless and should be Time is spent on more valuable time; so I wrote this article to summarize the steps or experiences I collected, built, and tested. Everyone is welcome to propose or discuss better solutions together.

Prerequisite skills:

  • Shell simple scripting (satisfying this item means you understand and can use Linux)
  • Docker
  • Git
  • Java (optional, since you read this article, you can do this, but no matter what automated deployment, the principles are the same)

Installation

Because they are all local tests, Docker containers are used for quick installation to reduce the time spent on deployment. Most of the corresponding installation articles have been written before, so I will not paste them again here.

Docker

Docker offline installation

GitLab

Docker GitLab installation

GitLab-Runner

Docker GitLab-Runner installation

Alibaba Cloud Container Warehouse

(Skip if not used). I originally planned to use Harbor image private server warehouse or Alibaba Cloud container warehouse, but found that this step can be omitted and the packaged docker image can be run directly through the host (Runner needs to be configured with docker in docker).

GitLab-CI

Entering the core of CI/CD, the grammar part is all dead. Here is an article recommended. It was originally a video on station b. When I found a corresponding article, I directly browsed the article and then tested it. Article link.

GitLab-CI application for Java microservices

  1. Pull the gitlab-ci template repository and upload it to your own gitlab (note that the code repository for CI/CD to introduce templates requires permissions for this repository)

gitlab-ci template warehouse (develop branch): https://github.com/zsl0/devops-ci-template.git

  1. Create the .gitlab-ci.yml file in the Java warehouse project directory

Please add image description

The simplified version can automatically run docker after code submission. Note that the auxiliary configuration stage is deploy job task that needs to be run as needed, such as the gateway and admin server automation configured below. Deployment requires configuring the three variables IMAGE_NAME, MODULE_PATH, and JAR_NAME of variables, which correspond to the service name and service module respectively. Relative path, packaged jar package name.

Project relative path:

Please add image description

.gitlab-ci.yml configuration:

include:
  - project: 'zsl0/devops-ci-template'
    ref: develop
    file: 'templates/default-pipeline.yml'

variables:
  ## Global configuration
  GIT_CLONE_PATH: ${<!-- -->CI_BUILDS_DIR}/builds/${<!-- -->CI_PROJECT_NAMESPACE}/${<!-- -->CI_PROJECT_NAME}/${<!-- -->CI_PIPELINE_ID }
  GIT_CHECKOUT: "false"
  CACHE_DIR: "**/target/**.jar"

  ## Job Control
  RUN_PIPELINE_BUILD: "yes" #Whether to run the build yes/no
  RUN_PIPELINE_TEST: "no" #Whether to run the test yes/no
  RUN_CODE_ANALYSIS: "no" #Whether to scan the code yes/no
  RUN_BUILD_IMAGE: "no" #Whether to generate a mirror yes/no
  RUN_DEPLOY_ARTIFACTS: "no" #Whether to upload products yes/no
  RUN_DEPLOY_K8S: "no" #Whether to release K8S yes/no
  RUN_DEPLOY_DOCKER: "yes" #Whether to release DOCKER yes/no

  ## Depends on container image
  BUILD_IMAGE: "registry.cn-hangzhou.aliyuncs.com/zsl0/maven-build:0.0.2"
  CURL_IMAGE: "curlimages/curl:7.70.0"
  SONAR_IMAGE: "sonarsource/sonar-scanner-cli:latest"
  KUBECTL_IMAGE: "lucj/kubectl:1.17.2"


  ## Build test parameters
  MAVEN_OPTS: "-Dmaven.repo.local=/home/gitlab-runner/ci-build-cache/maven " #maven build parameters
# BUILD_SHELL: 'mvn clean package -DskipTests -P zsl0-prod' #Build command
  BUILD_SHELL: 'mvn clean package -DskipTests -P hxkj-dev' #Build command
  #GRADLE_OPTS: "" #gradle build parameters

  ## Unit test parameters
  TEST_SHELL : 'mvn test ' #Test command
  JUNIT_REPORT_PATH: 'target/surefire-reports/TEST-*.xml' #Unit test report #Project scan parameters

  ## Build image
  #CI_REGISTRY: 'registry.cn-beijing.aliyuncs.com' #Mirror warehouse address
  #CI_REGISTRY_USER: '610556220zy' #Warehouse user information
  #CI_REGISTRY_PASSWD: 'xxxxxxxx.' #Warehouse user password
  IMAGE_NAME: "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}" #Image name
  DOCKER_FILE_PATH: "./Dockerfile" #Dockerfile location

  ## Upload artifact library (artifactory)
  #ARTIFACTORY_URL: "http://192.168.1.200:30082/artifactory" #Artifact library address
  #ARTIFACTORY_NAME: "${CI_PROJECT_NAMESPACE}" #Product library name
  #ARTIFACT_PACKAGE: "jar" #Product type
  #ARTIFACT_PATH: "target/*.${ARTIFACT_PACKAGE}" #Product location
  #TARGET_FILE_PATH: "${CI_PROJECT_NAMESPACE}/${CI_PROJECT_NAME}/${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}" #Target product location (directory structure)
  #TARGET_ARTIFACT_NAME: "${CI_PROJECT_NAME}-${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}.${ARTIFACT_PACKAGE}" #Target product name

  ## Deploy application k8s
  APP_NAME: "$CI_PROJECT_NAME" #Application name <-->deploymentName
  CONTAINER_PORT: "8081" #Service port <--> servicesPort
  NAMESPACE: "$CI_PROJECT_NAME-$CI_PROJECT_ID-$CI_ENVIRONMENT_SLUG" #Namespace
  ENV_URL: "${ENV_NAME}.${CI_PROJECT_NAMESPACE}.${CI_PROJECT_NAME}.devops.com" #IngressHosts

cache:
  paths:
    # Subsequent modifications will solve the problem of mvn packaging repository caching. You need to set the local warehouse path according to the path.
    - /root/.m2/repository
    -/opt/repository
    - ~/.m2/repository
    - ${<!-- -->CACHE_DIR}

## Build job
build:
  #Specify Runner, if not specified, the shared Runner will be used by default
  tags:
    -runner-java

## Deployment
## gateway Docker automated deployment
deploy-gateway:
  stage:deploy
  image: docker:latest
  tags:
    -runner-java
  extends: .deploy_docker
  variables:
    # Image name, service name is used by default
    IMAGE_NAME: "gateway"
    #Module path
    MODULE_PATH: "data-center/gateway/"
    # jar package name
    JAR_NAME: gateway.jar

## gateway Docker automated deployment
deploy-admin-server:
  stage:deploy
  image: docker:latest
  tags:
    -runner-java
  extends: .deploy_docker
  variables:
    # Image name, service name is used by default
    IMAGE_NAME: "admin-server"
    #Module path
    MODULE_PATH: "data-center/admin/admin-server/"
    # jar package name
    JAR_NAME: admin-server.jar
  1. Create Dockerfile_template template
FROM openjdk:11

MAINTAINER [email protected]

WORKDIR/opt/jar

RUN mkdir -p /opt/shell/
#ADD /zsl0/shell/ /opt/shell/

RUN mkdir -p /opt/jar/
ADD MODULE_PATH/target/JAR_NAME /opt/jar/java-program.jar

ENTRYPOINT ["java", "-jar"]
CMD ["java-program.jar"]

Other questions

Maven local warehouse cache

According to the reference article, choose to add the /.m2 cache directory in the configuration file, and specify the /.m2/repository warehouse directory in the specified mvn:

variables:
  MAVEN_OPTS: "-Dmaven.repo.local=/.m2/repository " #maven build parameters