Implement GitOps with GitHub, Argo CD, and KCL to simplify DevOps

Using GitOps, developers and operations teams can manage application deployment by modifying application and configuration code respectively, and the GitOps toolchain will automatically synchronize changes to the configuration, enabling continuous deployment and ensuring consistency. If something goes wrong, you can use the GitOps toolchain to quickly roll back.

Workflow

In this example, we use a Python Flask app and Github Actions as the CI example, Argo CD as the CD example, and KCL to define the Kubernetes resources that need to be deployed.

Note: You can use any containerized application and different CI and CD systems such as Gitlab CI, Jenkins CI, FluxCD, etc. in this solution. We will publish more sample articles to illustrate this in the future.

We split the Python Flask application code and configuration code into two repositories to achieve separation of concerns for different roles such as developers and operations teams:

• Business code repository: https://github.com/kcl-lang/flask-demo

• Configuration manifest repository: https://github.com/kcl-lang/flask-demo-kcl-manifests

The overall workflow is as follows:

1. Pull application code from GitHub

2. Application code development and submission to GitHub repository

3. Trigger GitHub Actions to compile the application code, generate a container image, and push the container image to the Dockerhub container registry

4. Trigger GitHub Actions based on the version number of the container image in the docker.io container registry and synchronously update the KCL-defined Kubernetes manifest deployment file

5. Argo CD gets KCL-defined Kubernetes manifest changes and updates are deployed to the Kubernetes cluster

Specific steps

0. Prerequisites

• Familiar with basic Unix/Linux commands

• Familiar with Git and using GitHub Action

• Learn the basics of Kubernetes

• Learn about tools like Argo CD

• Learn the basics of KCL

1. Configure Kubernetes cluster

• Install K3d (https://github.com/k3d-io/k3d) and create a cluster

k3d cluster create mycluster

Note: You can use other methods to create your own Kubernetes cluster in this scenario, such as kind, minikube, etc.

2. Configure Argo CD

Configuring the Argo CD Controller

• Install K3d (https://github.com/k3d-io/k3d) and create a cluster

k3d cluster delete mycluster & amp; & amp; k3d cluster create mycluster

Note: You can use other methods to create your own Kubernetes cluster in this scenario, such as kind, minikube, etc.

• Install Argo CD (https://github.com/argoproj/argo-cd/releases/)

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

• Install the Argo CD KCL plugin

kubectl apply -f ./install/kcl-cmp.yaml & amp; & amp; kubectl -n argocd patch deploy/argocd-repo-server -p "$(cat ./install/patch-argocd-repo -server.yaml)"

• Use the kubectl get command to check whether the Argo CD controller container has been initialized and entered the running state.

kubectl get pod -n argocd -l app.kubernetes.io/name=argocd-repo-server

• Open the Argo CD UI with the following command

kubectl port-forward svc/argocd-server -n argocd 8080:443

• Open the browser https://localhost:8080 and enter the username “admin” and password to log in to the Argo CD UI. The password can be obtained through the following command:

kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

Configuring the ArgoCD client tool

• Install the Argo CD client tool (https://github.com/argoproj/argo-cd/releases)

• Log in using the username “admin” and the password you just got.

argocd login localhost:8080

Create an Argo CD KCL application with the following command

argocd app create flaskdemo \
--repo https://github.com/kcl-lang/flask-demo-kcl-manifests \
--path .\
--dest-namespace default \
--dest-server https://kubernetes.default.svc \
--config-management-plugin kcl-v1.0

If the creation is successful, you can see the following output:

application 'flaskdemo' created

If you are using a private repository, you need to configure dedicated private repository access using private key credentials before executing the create command. See https://argo-cd.readthedocs.io/en/stable/user-guide/private-repositories/ for more details.

Through the Argo CD UI, you can see that the created application has not yet been synced, and you can manually configure the sync or set it to automatic synchronization.

For more information about synchronization strategies, see https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/

3. Obtain business code

git clone https://github.com/kcl-lang/flask-demo.git/
cd flask-demo

This is a web application written in Python. We can use the Dockerfile in the application directory to generate the container image of this application. At the same time, we can automatically build the flask_demo image through GitHub CI. CI The configuration is as follows:

# This is a basic workflow to help you get started with Actions


name:CI


# Controls when the workflow will run
on:
  # Triggers the workflow on push or pull request events but only for the main branch
  push:
    branches: [main]
  pull_request:
    branches: [main]


  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:


# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest


    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v2
      
      - name: Docker Login
        uses: docker/[email protected]
        with:
          username: ${<!-- -->{ secrets.DOCKER_USERNAME }}
          password: ${<!-- -->{ secrets.DOCKER_PASSWORD }}
          logout: true


      # Runs a set of commands using the runners shell
      - name: build image
        run: |
          make image
          docker tag flask_demo:latest ${<!-- -->{ secrets.DOCKER_USERNAME }}/flask_demo:${<!-- -->{ github.sha }}
          docker push ${<!-- -->{ secrets.DOCKER_USERNAME }}/flask_demo:${<!-- -->{ github.sha }}


      # Trigger KCL manifest
      - name: Trigger CI
        uses: InformaticsMatters/[email protected]
        with:
          ci-owner:kcl-lang
          ci-repository: flask-demo-kcl-manifests
          ci-ref: refs/heads/main
          ci-user:kcl-bot
          ci-user-token: ${<!-- -->{ secrets.DEPLOY_ACCESS_TOKEN }}
          ci-name:CI
          ci-inputs: >-
            image=${<!-- -->{ secrets.DOCKER_USERNAME }}/flask_demo
            sha-tag=${<!-- -->{ github.sha }}

We need the workflow of the source code warehouse to automatically trigger the workflow in the deployment manifest warehouse. At this time, we need to create secrets.DEPLOY_ACCESS_TOKEN with GitHub CI operation permissions and the account information secrets pushed by the Docker Hub image. DOCKER_USERNAME and secrets.DOCKER_PASSWORD, these can be configured in the Secrets and variables settings of the GitHub repository, as shown below:

4. Submit application code

After submitting the code to the flask-demo warehouse, GitHub will automatically build the container image and push the product to Docker Hub. It will then trigger the Action of the flask-demo-kcl-manifests warehouse and modify the image address in the deployment manifest warehouse through the KCL automation API. . Now let us create a commit for the flask-demo warehouse. We can see that the business warehouse GitHub CI process is triggered after the code is submitted.

5. Configure automatic updates

When the GitHub CI process of the business warehouse is executed, a CI will be automatically triggered in the warehouse where the KCL resource configuration is stored to automatically update the configuration and submit it to the flask-demo-kcl-manifests main branch. The commit information is as follows:

• We can obtain the deployment manifest source code for compilation verification

git clone https://github.com/kcl-lang/flask-demo-kcl-manifests.git/
cd flask-demo-kcl-manifests
git checkout main & amp; & amp; git pull & amp; & amp; kcl

The output YAML is:

apiVersion: apps/v1
Kind: Deployment
metadata:
  name: flask_demo
  labels:
    app: flask_demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: flask_demo
  template:
    metadata:
      labels:
        app: flask_demo
    spec:
      containers:
        - name: flask_demo
          image: "kcllang/flask_demo:6428cff4309afc8c1c40ad180bb9cfd82546be3e"
          ports:
            - protocol: TCP
              containerPort: 5000
---
apiVersion: v1
Kind: Service
metadata:
  name: flask_demo
  labels:
    app: flask_demo
spec:
  type: NodePort
  selector:
    app: flask_demo
  ports:
    - port: 5000
      protocol:TCP
      targetPort: 5000

From the above configuration, we can see that the resource image is indeed automatically updated with the newly built image content.

In addition, we can set the synchronization policy to automatic synchronization in the Argo CD UI interface, so that we can achieve a complete automated process of e2e business code submission and deployment of Kubernetes.

Summary

Through this article, we can use GitHub, Argo CD and KCL to create a GitOps automated pipeline, which can efficiently and stably build containerized applications, automatically update the latest Docker image tags, and keep the Git configuration consistent with the cluster configuration. In addition, combining KCL and Argo CD can help us better implement Infrastructure as Code (IaC), improve deployment efficiency, achieve separation of concerns of different roles and simplify application configuration management .

Participate in KCL open source co-construction?

Star?:
https://github.com/kcl-lang/kcl

https://github.com/KusionStack/kusion

Welcome to play

https://kcl-lang.io

https://kusionstack.io/

Recommended reading this week

Good news! The CNCF Foundation announces KCL as a sandbox project!

SOFARegistry | Large-scale cluster optimization practice

SOFARegistry | Let’s talk about data consistency in service discovery

Exploration and practice of Ant SOFAServerless new microservice architecture

The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge. Cloud native entry-level skills treeHomepageOverview 16,745 people are learning the system