Gitops Handbook 5: Building GitOps Workflows with ArgoCD

Introduction

This lab series consists of seven topics, covering implementation and advanced features from application containerization to GitOps workflows. Through these lab playbooks, you’ll learn how to build, deploy, and manage applications, and how to implement an efficient continuous delivery process on a Kubernetes cluster.

The following are the seven chapters of the lab manual:

  1. Application Containerization Lab Manual: Learn how to containerize applications and build images for mainstream programming languages. Also, learn how to reduce image size and automate image builds with GitHub Actions.
  2. Single-node Kubernetes cluster deployment experiment manual: Use kind to deploy a single-node Kubernetes cluster locally, install basic services, and get familiar with the basic functions of Kubernetes, such as running Pods, scheduling workloads, service publishing, and automatic scaling.
  3. Deployment and Analysis Experiment Manual of Sample Application: Get an in-depth understanding of each component of the sample application, learn how to deploy, run, and manage these components on a Kubernetes cluster, and implement functions such as service invocation, application configuration, and scaling.
  4. Using Helm to Define an Application Lab Playbook: Define a sample application with Helm to meet the needs of a GitOps deployment. Learn about creating and using Helm Charts, and deploying to staging and production environments.
  5. Build a GitOps workflow experiment manual: Install ArgoCD on the Kubernetes cluster, and use ArgoCD to define sample applications for deployment to implement GitOps workflows triggered based on image version changes.
  6. Implementing GitOps Advanced Release Strategy Lab Playbook: Combine advanced features of ArgoCD, such as Argo Rollout, AnalysisTemplate, and Argo Rollout Dashboard, to implement blue-green releases, canary releases, and automated progressive delivery.
  7. ArgoCD Advanced Management Features Lab Manual: Explores advanced features of ArgoCD in GitOps scenarios, including multi-environment deployment with ApplicationSet and secret protection with SealedSecret.

Through these seven lab manuals, you will have a comprehensive knowledge of application containerization, Kubernetes, and GitOps, providing reliable and efficient solutions for continuous delivery of applications.

Based on the previous experimental scenarios, this lab manual will guide you to install ArgoCD on the Kubernetes cluster, define and deploy sample applications on ArgoCD, and realize the GitOps workflow triggered by image version changes.

This lab manual will cover the following main lab links:

  1. Installing ArgoCD on a Kubernetes Cluster: Learn how to install ArgoCD and how to deploy it on a Kubernetes cluster.
  2. Defining a sample application on ArgoCD: Learn how to create an application on the ArgoCD console, and how to configure information such as the application source, target cluster, and target namespace.
  3. Deploy the sample app: Learn how to deploy a Helm Chart to a Kubernetes cluster on ArgoCD and watch the deployment process.
  4. Implement GitOps workflows: Learn how to trigger GitOps workflows when image versions change to automate deployment and rollback.

Through this lab manual, you will master how to use ArgoCD to build a GitOps workflow to further improve the deployment, update and management efficiency of Kubernetes applications.

Deploy ArgoCD

create namespace

kubectl create namespace argocd

Install

kubectl apply -n argocd -f https://ghproxy.com/https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Check the installation

kubectl wait --for=condition=Ready pods --all -n argocd --timeout 300s

root@node1:~/kubernetes-example# kubectl wait --for=condition=Ready pods --all -n argocd --timeout 300s
pod/argocd-application-controller-0 condition met
pod/argocd-applicationset-controller-69c4b965dc-mcgd9 condition met
pod/argocd-dex-server-64d856b94c-8jntz condition met
pod/argocd-notifications-controller-f7c967bc9-7nb8p condition met
pod/argocd-redis-598f75bc69-7pg4j condition met
pod/argocd-repo-server-df7f747b4-48cj7 condition met
pod/argocd-server-59d9b8cb46-wns97 condition met

Install argocd-cli

curl -sSL -o argocd-linux-amd64 https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64

sudo install -m 555 argocd-linux-amd64 /usr/local/bin/argocd
rm argocd-linux-amd64

forward to port 8080 (it is recommended to open another session)

kubectl port-forward service/argocd-server 8080:80 -n argocd

Create ingress manifest

nano argocd-ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: argocd-server-ingress
  namespace: argocd
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
  rules:
  - host: argocd.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: argocd-server
            port:
              name: https
  tls:
  -hosts:
    - argocd.example.com
    secretName: argocd-secret # do not change, this is provided by Argo CD

kubectl apply -f argocd-ingress.yaml

Obtaining the ArgoCD Admin initial password

kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

amsgK3UobDG6VsVd

Modify the local host file

192.168.1.231 argocd.example.com

Visit https://argocd.example.com/ to suggest changing the password

(Optional) Install cert-manager

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

helm repo add jetstack https://charts.jetstack.io

 helm repo update

helm install cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v1.10.0 \
--set ingressShim.defaultIssuerName=letsencrypt-prod \
--set ingressShim.defaultIssuerKind=ClusterIssuer \
--set ingressShim.defaultIssuerGroup=cert-manager.io \
--set installCRDs=true

nano cluster-issuer.yaml

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: "[email protected]"
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: nginx

kubectl apply -f cluster-issuer.yaml

Creating an ArgoCD application

Make sure to delete the previous example project

kubectl delete ns example

Login to Argo CD

argocd login localhost:8080 --insecure

root@node1:~# argocd login localhost:8080 --insecure
Username: admin
Password:
'admin:login' logged in successfully
Context 'localhost:8080' updated

Optional $PASSWORD here is the previously defined github token

argocd repo add https://github.com/chengzh/kubernetes-example.git --username cloudzun --password $PASSWORD


argocd app create example --sync-policy automated --repo https://github.com/cloudzun/kubernetes-example.git --revision main --path helm --dest-namespace gitops-example --dest -server https://kubernetes.default.svc --sync-option CreateNamespace=true

This command uses Argo CD to create and manage an application named example in a Kubernetes cluster. The following is a detailed explanation of each part of the command:

  1. argocd app create: This is the Argo CD command to create a new application.
  2. example: Assign a name to the created Argo CD application. In this case, the name of the application is example.
  3. --sync-policy automated: Set the synchronization policy to be automated, which means that when the configuration in the Git repository changes, Argo CD will automatically apply these changes to the Kubernetes cluster.
  4. --repo https://github.com/cloudzun/kubernetes-example.git: Specify the Git repository URL of the application. In this example, the Git repository is located at https://github.com/cloudzun/kubernetes-example.git.
  5. --revision main: Specifies the branch or revision of the Git repository. Here, use the main branch.
  6. --path helm: Specifies the path in the Git repository that contains Kubernetes resources. In this case, the path is helm.
  7. --dest-namespace gitops-example: Specify the Kubernetes namespace for deploying applications. In this case, the namespace is gitops-example.
  8. --dest-server https://kubernetes.default.svc: Specifies the URL of the Kubernetes API server where Argo CD will deploy the application. Here, use the default API server address https://kubernetes.default.svc of the Kubernetes cluster.
  9. --sync-option CreateNamespace=true: Set a sync option to automatically create the target namespace (if it doesn’t exist) when the app syncs.

In summary, this command uses Argo CD to create an application named example in the specified Kubernetes namespace, which fetches Kubernetes resources from the specified Git repository and branch. The sync policy is set to automatic so that when the configuration in the Git repository changes, those changes are automatically applied.

Achieving a GitOps workflow triggered by mirror version changes

Installing and configuring ArgoCD Image Updater

Install Argo CD Image Updater

kubectl apply -n argocd -f https://ghproxy.com/https://raw.githubusercontent.com/argoproj-labs/argocd-image-updater/stable/manifests/install.yaml

Set the docker warehouse secret (note the timeliness of docker-password)

kubectl create -n argocd secret docker-registry dockerhub-secret \
  --docker-username chengzh \
  --docker-password dckr_pat_UxCRddCJXMg9_HNyHA0gsE3BSZA \
  --docker-server "https://registry-1.docker.io"

Create a Helm Chart repository

Create a new Git repository, copy the helm directory of the existing repository to the new repository

 $ cp -r ./kubernetes-example/helm ./kubernetes-example-helm

Give ArgoCD Image Updater permission to write back to the kubernetes-example-helm repository. To configure repository access, you can use the argocd repo add command

argocd repo add https://github.com/cloudzun/kubernetes-example-helm.git --username $USERNAME --password $PASSWORD

Note to change the warehouse address to the newly created GitHub warehouse address for storing Helm Chart, and replace $USERNAME with GitHub account ID, and $PASSWORD is replaced with GitHub Personal Token. You can create a GitHub Personal Token on this page: https://github.com/settings/tokens and grant relevant permissions to the repository.

argocd repo add https://github.com/cloudzun/kubernetes-example-helm.git --username cloudzun --password $PASSWORD

Creating an ArgoCD application

(Optional) Delete old apps

argocd app delete example --cascade

The next step is to formally create the ArgoCD application. In the previous lesson, an ArgoCD application was created using the argocd app create command. In fact, it will create a special type of resource, that is, ArgoCD Application, which, like other K8s standard resource objects, is also defined using YAML.
Here, directly use YAML to create a new Application, and save the content of the following file as application.yaml.

nano application.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: example
  annotations:
    argocd-image-updater.argoproj.io/backend.allow-tags: regexp:^main
    argocd-image-updater.argoproj.io/backend.helm.image-name: backend.image
    argocd-image-updater.argoproj.io/backend.helm.image-tag: backend.tag
    argocd-image-updater.argoproj.io/backend.pull-secret: pullsecret:argocd/dockerhub-secret
    argocd-image-updater.argoproj.io/frontend.allow-tags: regexp:^main
    argocd-image-updater.argoproj.io/frontend.helm.image-name: frontend.image
    argocd-image-updater.argoproj.io/frontend.helm.image-tag: frontend.tag
    argocd-image-updater.argoproj.io/frontend.pull-secret: pullsecret:argocd/dockerhub-secret
    argocd-image-updater.argoproj.io/image-list: frontend=chengzh/frontend, backend=chengzh/backend
    argocd-image-updater.argoproj.io/update-strategy: latest
    argocd-image-updater.argoproj.io/write-back-method: git
spec:
  destination:
    namespace: gitops-example-updater
    server: https://kubernetes.default.svc
  project: default
  source:
    path: .
    repoURL: https://github.com/cloudzun/kubernetes-example-helm.git
    targetRevision: main
  syncPolicy:
    automated: {}
    syncOptions:
      -CreateNamespace=true

This manifest file defines an Argo CD Application named example, which is used to deploy and manage a Helm-based application in a Kubernetes cluster. At the same time, this manifest also contains some configurations of Argo CD Image Updater, which is used to automatically update the container image. Here is a detailed explanation of each part of the file:

  1. apiVersion and kind: These two fields define the type of Kubernetes resource. In this example, the resource type is argoproj.io/v1alpha1 and Application, indicating that this is an Argo CD Application resource.
  2. metadata: Defines metadata about the Application, such as name and comments.
    • name: defines the name of the application, which is example.
    • annotations: defines the configuration of Argo CD Image Updater. These configurations allow automatic updating of container images, while specifying information such as image names, tag policies, etc.
  3. spec: defines the specific configuration of the application.
    • destination: defines the target Kubernetes cluster and namespace for application deployment.
      • namespace: defines the Kubernetes namespace for deploying applications, which is gitops-example-updater.
      • server: defines the URL of the Kubernetes API server, which is https://kubernetes.default.svc.
    • project: defines the Argo CD project, which is default.
    • source: defines the source code information of the application.
      • path: defines the path that contains Kubernetes resources, in this example it is ., which means the current directory.
      • repoURL: defines the URL of the Git repository, which is https://github.com/cloudzun/kubernetes-example-helm.git.
      • targetRevision: defines the branch or revision of the Git repository, which is main.
    • syncPolicy: defines the synchronization policy of the application.
      • automated: Specifies the automatic synchronization strategy, which means that when the configuration in the Git repository changes, Argo CD will automatically apply these changes to the Kubernetes cluster.
      • syncOptions: defines synchronization options, such as automatically creating namespaces during synchronization.

In summary, this manifest file defines an Argo CD application named example, which is used to deploy and manage a Helm-based application in a specified Kubernetes namespace. At the same time, it also contains the configuration of Argo CD Image Updater, which is used to automatically update container images.

  • The annotations section contains the Argo CD Image Updater configuration. These configurations are used to specify policies, parameters, and related information for automatically updating container images. Here is a detailed explanation of these annotations:

    1. argocd-image-updater.argoproj.io/backend.allow-tags and argocd-image-updater.argoproj.io/frontend.allow-tags: these two annotations For the backend (backend) and frontend (frontend) images, specify the image tags that allow updates. The regular expression regexp:^main is used here, which means that tags starting with “main” are allowed.
    2. argocd-image-updater.argoproj.io/backend.helm.image-name and argocd-image-updater.argoproj.io/frontend.helm.image-name: These two annotations specify the image name field in the Helm chart for the backend (backend) and frontend (frontend) images respectively. The values here are backend.image and frontend.image.
    3. argocd-image-updater.argoproj.io/backend.helm.image-tag and argocd-image-updater.argoproj.io/frontend.helm.image-tag: These two annotations specify the image tag field in the Helm chart for the backend (backend) and frontend (frontend) images respectively. The values here are backend.tag and frontend.tag.
    4. argocd-image-updater.argoproj.io/backend.pull-secret and argocd-image-updater.argoproj.io/frontend.pull-secret: these two annotations For the backend (backend) and frontend (frontend) mirrors, the Secret used to pull the mirror is specified. Here, the Secret name is dockerhub-secret, located under the argocd namespace.
    5. argocd-image-updater.argoproj.io/image-list: This annotation defines the list of images used in the application. Here are the images of frontend (frontend) and backend (backend), corresponding to chengzh/frontend and chengzh/backend respectively.
    6. argocd-image-updater.argoproj.io/update-strategy: This annotation defines the image update strategy. The value here is latest, indicating that the latest image tag is used for updating.
    7. argocd-image-updater.argoproj.io/write-back-method: This annotation defines the updated configuration writeback method. The value here is git, indicating that the updated configuration will be written back to the Git repository.

    These annotations are related to Argo CD Image Updater and are used to configure policies, parameters and related information for automatically updating container images. This way, Argo CD Image Updater can automatically update applications deployed in Kubernetes clusters when a new version of the image is available.

Then, use the kubectl apply command to create an ArgoCD Application, which is equivalent to using the argocd app create command to create an application.

kubectl apply -n argocd -f application.yaml

Trigger GitOps workflow

Next, you can try to modify the frontend/src/App.js file, such as modifying the “Hi! I am abraham” content on line 49 of the file. After the modification is complete, push the code to the main branch of GitHub. Two GitHub Action workflows are triggered at this point. Among them, when the build-every-branch workflow is triggered, it will build a mirror version whose Tag starts with main and push it to the mirror warehouse,

#22 pushing manifest for docker.io/chengzh/backend:main-2fff0b2@sha256:20ae4681cebb5c3096a43f067f53e5255777b4de52d1b636a2143e8ed130845e

[1034](https://github.com/cloudzun/kubernetes-example/actions/runs/4252208283/jobs/7395500319#step:8:1034)#22 pushing manifest for docker.io/chengzh/backend:main-2fff0b2 @sha256:20ae4681cebb5c3096a43f067f53e5255777b4de52d1b636a2143e8ed130845e 0.9s done

Note the backend:main-2fff0b2 in the above log

Go to docker hub to view the newly launched image tag:
![[Pasted image 20230223194606.png]]
It can be observed that main-2fff0b2

At the same time, ArgoCD Image Updater will retrieve the image versions of frontend and backend from the mirror warehouse every 2 minutes. Once a new one is found, it will start with main code>, it will automatically update the image of the workload in the cluster with the new version, and write the image version back to the kubernetes-example-helm repository. When writing back, ArgoCD Image Updater will not directly modify the values.yaml file of the warehouse, but will create a .argocd-source-example.yaml specially used to overwrite Helm Chart values.yaml document

helm:
  parameters:
  - name: frontend.image
    value: chengzh/frontend
    forcestring: true
  - name: frontend.tag
    value: main-2fff0b2
    forcestring: true
  - name: backend.image
    value: chengzh/backend
    forcestring: true
  - name: backend.tag
    value: main-2fff0b2
    forcestring: true

Use the following command to view the deployment

kubectl get deployment -n gitops-example-updater -o wide

root@node1:~# kubectl get deployment -n gitops-example-updater -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
backend 1/2 2 1 13m flask-backend chengzh/backend:main-2fff0b2 app=backend
frontend 2/7 7 2 13m react-frontend chengzh/frontend:main-2fff0b2 app=frontend
postgres 1/1 1 1 13m postgres postgres app=database

Focus on the mirror image here: chengzh/backend:main-2fff0b2

Demo tips

Use a simplified version of the application manifest that only responds to frontend updates

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: example
  annotations:
    argocd-image-updater.argoproj.io/frontend.allow-tags: regexp:^main
    argocd-image-updater.argoproj.io/frontend.helm.image-name: frontend.image
    argocd-image-updater.argoproj.io/frontend.helm.image-tag: frontend.tag
    argocd-image-updater.argoproj.io/frontend.pull-secret: pullsecret:argocd/dockerhub-secret
    argocd-image-updater.argoproj.io/image-list: frontend=chengzh/frontend
    argocd-image-updater.argoproj.io/update-strategy: latest
    argocd-image-updater.argoproj.io/write-back-method: git
spec:
  destination:
    namespace: gitops-example-updater
    server: https://kubernetes.default.svc
  project: default
  source:
    path: .
    repoURL: https://github.com/cloudzun/kubernetes-example-helm.git
    targetRevision: main
  syncPolicy:
    automated: {<!-- -->}
    syncOptions:
      -CreateNamespace=true

  • Remember to delete .argocd-source-example.yaml in https://github.com/cloudzun/kubernetes-example-helm/ before doing the demo

  • It takes about 20 minutes from triggering the action to deploying the new version. Patience is a virtue.

Postscript

This course system aims to provide students with a comprehensive learning path of cloud native technologies, including container technology, Kubernetes deployment and management, Ingress management, persistent storage, application deployment, observability, service grid, DevOps tool chain and pipeline And things like GitOps. By studying this course system, students will master the key concepts, principles and practical methods of cloud-native technology, so as to improve the deployment, management and collaboration efficiency of cloud-native applications.

  1. Docker technology in action: Learn the basics of container technology, including how Docker works, core components, and application deployment and monitoring management.
  2. Kubernetes deployment and management practice: In-depth understanding of the core concepts and basic components of Kubernetes, and master the deployment, management and monitoring skills of Kubernetes clusters.
  3. Kubernetes Ingress management and operation and maintenance: learn the concept, working mechanism and operation and maintenance management methods of Kubernetes Ingress, and improve the deployment and management capabilities of Ingress.
  4. Kubernetes persistent storage practice: Discuss the principle, implementation and practical application of Kubernetes persistent storage, and improve the ability to implement persistent storage on Kubernetes.
  5. Kubernetes application deployment and management: learn the basic concepts, principles and implementation methods of Kubernetes application deployment and management, and master the skills of deploying and managing applications on Kubernetes.
  6. Kubernetes Observability Practices: Study the concepts, principles, and implementation methods of Kubernetes observability practices, covering aspects such as monitoring, logging, and analysis.
  7. Istio-based service mesh combat: In-depth understanding of Istio-based service mesh concepts, principles, and implementation methods, including service routing, traffic management, elasticity, and microservice observability.
  8. DevOps tool chain and pipeline practice: Discuss the concepts, principles and implementation methods of DevOps tool chain and pipeline, covering automated construction, testing, deployment and monitoring.
  9. Efficient deployment and management of GitOps cloud-native applications: Comprehensively master GitOps principles, technologies and tools to improve the efficiency of cloud-native application deployment, management and collaboration.

Through the study of these nine courses, students can:

  1. Proficiency in deploying and managing containerized applications using Docker and Kubernetes.
  2. Master the configuration and management methods of Ingress and persistent storage in Kubernetes cluster.
  3. Learn Kubernetes observability practices to improve application monitoring, logging, and analytics.
  4. Explore the principle and implementation of Istio service mesh to achieve high availability, elasticity and security of microservices.
  5. Automate builds, tests, deployments, and monitoring with DevOps toolchains and pipelines.
  6. Master the principles and technologies of GitOps to improve the deployment, management and collaboration efficiency of cloud-native applications.

This course system is suitable for developers, operation and maintenance engineers, architects and other professionals who are interested in cloud native technology. The courses are rich in content and highly practical, aiming to help students better apply cloud-native technologies in practical work and contribute to the digital transformation of enterprises.

If you want to get a more detailed course introduction, please contact: [email protected]