K8S resource configuration management

1. Secret

Secret is a k8s resource used to store sensitive data such as passwords, tokens, and keys. Although such data can also be stored in Pods or images, it is placed in Secrets to make it easier to control how to use data and reduce exposure. risk.

1. There are three types of Secret

  • kubernetes.io/service-account-token: Automatically created by Kubernetes to access the APIServer’s Secret, the Pod will use this Secret to communicate with the APIServer by default, and it will be automatically mounted to the Pod’s /run/secrets/kubernetes.io/serviceaccount directory;
  • Opaque: Secret in base64 encoding format, used to store user-defined passwords, keys, etc., the default Secret type;
  • kubernetes.io/dockerconfigjson : Used to store authentication information for private docker registries.

Pod needs to be referenced before using a certain secret. Pod has 3 ways to use secret:

  • As a file in a volume mounted on one or more containers.
  • As an environment variable for the container.
  • Used by the kubelet when pulling images for pods.

Application Scenario: Credentials
https://kubernetes.io/docs/concepts/configuration/secret/

2. Create a secret instance

2.1 Create a Secret with the kubectl create secret command

echo -n 'zhangsan' > username.txt
echo -n 'abc123' > password.txt

kubectl create secret generic mysecret --from-file=username.txt --from-file=password.txt

kubectl get secrets

Neither the #get nor describe command will display the actual content of the secret, which is out of consideration for data protection
kubectl describe secret mysecret

2.2 Encode the content with base64 and create a secret

echo -n zhangsan | base64
emhhbmdzYW4=

echo -n abc123 | base64
wxya

vim secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: mysecret1
type: Opaque
data:
  username: emhhbmdzYW4=
  password: YWJjMTIz

kubectl create -f secret.yaml

kubectl get secrets

kubectl get secret mysecret1 -o yaml

apiVersion: v1
data:
  password: YWJjMTIz
  username: emhhbmdzYW4=
kind: Secret
metadata:
  creationTimestamp: "2023-03-02T07:15:01Z"
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:data:
        .: {}
        f:password:{}
        f:username:{}
      f:type:{}
    manager: kubectl-create
    operation: Update
    time: "2023-03-02T07:15:01Z"
  name: mysecret1
  namespace: default
  resourceVersion: "50956"
  uid: e611fddd-7ca1-4e75-8d3d-99478ab58a83
type: Opaque

3. How to use

3.1 Mount the Secret to the Volume, and mount it to a certain directory of the Pod in the form of Volume

vim secret-test.yaml

apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  containers:
  -name: nginx
    image: nginx
    volumeMounts:
    -name: secrets
      mountPath: "/etc/secrets"
      readOnly: true
  volumes:
  -name: secrets
    secret:
      secretName: mysecret

kubectl create -f secret-test.yaml

kubectl get pods

kubectl exec -it seret-test bash

3.2 Export secret to environment variables

vim secret-test1.yaml

apiVersion: v1
kind: Pod
metadata:
  name: mypod1
spec:
  containers:
  -name: nginx
    image: nginx
    env:
      - name: TEST_USER
        valueFrom:
          secretKeyRef:
            name: mysecret1
            key: username
      - name: TEST_PASSWORD
        valueFrom:
          secretKeyRef:
            name: mysecret1
            key: password

kubectl apply -f secret-test1.yaml

kubectl get pods

kubectl exec -it mypod bash

2. ConfigMap

Similar to Secret, the difference is that ConfigMap saves configuration information that does not need to be encrypted.
The ConfigMap feature was introduced in Kubernetes version 1.2, and many applications read configuration information from configuration files, command line parameters, or environment variables. Con?gMap API provides us with a mechanism to inject configuration information into the container. Con?gMap can be used to save a single property, or it can be used to save the entire configuration file or JSON binary large object.
Application Scenario: Application Configuration

1. Create ConfigMap

mkdir /opt/configmap/
 
vim /opt/configmap/game.properties
enemies = aliens
lives=3
enemies.cheat=true
enemies.cheat.level=noGoodRotten
secret.code.passphrase=UUDDLRLRBABAS
secret.code.allowed=true
secret.code.lives=30
 
vim /opt/configmap/ui.properties
color.good=purple
color.bad=yellow
allow.textmode=true
how.nice.to.look=fairlyNice

ls /opt/configmap/
game.properties
ui.properties

kubectl create configmap game-config --from-file=/opt/configmap/
#--from-file All files specified in the directory will be used to create a key-value pair in Con?gMap, the name of the key is the file name, and the value is the content of the file

kubectl get cm

kubectl get cm game-config -o yaml

apiVersion: v1
data:
  game.properties: | +
    enemies = aliens
    lives=3
    enemies.cheat=true
    enemies.cheat.level=noGoodRotten
    secret.code.passphrase=UUDDLRLRBABAS
    secret.code.allowed=true
    secret.code.lives=30

  ui.properties: | +
    color.good=purple
    color.bad=yellow
    allow.textmode=true
    how.nice.to.look=fairlyNice

kind: ConfigMap
metadata:
  creationTimestamp: "2023-03-02T10:31:37Z"
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:data:
        .: {}
        f:game.properties: {}
        f:ui.properties: {}
    manager: kubectl-create
    operation: Update
    time: "2023-03-02T10:31:37Z"
  name: game-config
  namespace: default
  resourceVersion: "65700"
  uid: 33ac77c9-8f0f-4d5e-a14d-763ac6e2be04

1.1 Using file creation

A Con?gMap can be created from a single file as long as it is specified as a file
–from-file This parameter can be used multiple times, that is, it can be used twice to specify the two configuration files in the previous example, the effect is the same as specifying the entire directory

kubectl create configmap game-config-2 --from-file=/opt/configmap/game.properties --from-file=/opt/configmap/ui.properties

kubectl get configmaps game-config-2 -o yaml

kubectl describe cm game-config-2

1.2 Create with literal values

Create with a literal value, use the –from-literal parameter to pass configuration information, this parameter can be used multiple times, the format is as follows

kubectl create configmap special-config --from-literal=special.how=very --from-literal=special.type=good

kubectl get configmaps special-config -o yaml

apiVersion: v1
data:
  special.how: very #key-value structure
  special.type: good
kind: ConfigMap
metadata:
  creationTimestamp: "2023-03-02T10:41:48Z"
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:data:
        .: {}
        f:special.how:{}
        f:special.type:{}
    manager: kubectl-create
    operation: Update
    time: "2023-03-02T10:41:48Z"
  name: special-config
  namespace: default
  resourceVersion: "66554"
  uid: 56e431fa-4833-458b-9a35-ab19f7dd3f77

kubectl delete cm --all

kubectl delete pod --all

2. Use Con?gMap in Pod

2.1 Use Con?gMap to replace environment variables

vim env.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: special-config
  namespace: default
data:
  special. how: very
  special.type: good
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: env-config
  namespace: default
data:
  log_level: INFO

kubectl create -f env.yaml

kubectl get cm

Pod creation

vim test-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: test-pod
spec:
  containers:
  -name: busybox
    image: busybox:1.28.4
    command: [ "/bin/sh", "-c", "env" ]
    env:
      - name: SPECIAL_HOW_KEY
        valueFrom:
          configMapKeyRef:
            name: special-config
            key: special.how
      - name: SPECIAL_TYPE_KEY
        valueFrom:
          configMapKeyRef:
            name: special-config
            key: special.type
    envFrom:
      - configMapRef:
          name: env-config
  restartPolicy: Never


kubectl create -f test-pod.yaml

kubectl get pods

kubectl logs pod-test

KUBERNETES_PORT=tcp://10.96.0.1:443
KUBERNETES_SERVICE_PORT=443
HOSTNAME=test-pod
SHLVL=1
SPECIAL_HOW_KEY=very #The value of the variable SPECIAL_HOW_KEY is special-config
NGINX_PORT_80_TCP=tcp://10.105.252.176:80
SPECIAL_TYPE_KEY=good #The value of the variable SPECIAL_TYPE_KEY is special-config
KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
NGINX_SERVICE_HOST=10.105.252.176
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
NGINX_PORT=tcp://10.105.252.176:80
NGINX_SERVICE_PORT=80
log_level=INFO #introduce env-config variable log_level: INFO
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443
KUBERNETES_SERVICE_HOST=10.96.0.1
PWD=/
NGINX_PORT_80_TCP_ADDR=10.105.252.176
NGINX_PORT_80_TCP_PORT=80
NGINX_PORT_80_TCP_PROTO=tcp

2.2 Use Con?gMap to set command line parameters

vim test-pod2.yaml
apiVersion: v1
kind: Pod
metadata:
  name: test-pod2
spec:
  containers:
  -name: busybox
    image: busybox:1.28.4
    command:
- /bin/sh
- -c
- echo "$(SPECIAL_HOW_KEY) $(SPECIAL_TYPE_KEY)"
    env:
      - name: SPECIAL_HOW_KEY
        valueFrom:
          configMapKeyRef:
            name: special-config
            key: special.how
      - name: SPECIAL_TYPE_KEY
        valueFrom:
          configMapKeyRef:
            name: special-config
            key: special.type
    envFrom:
      - configMapRef:
          name: env-config
  restartPolicy: Never


kubectl create -f test-pod2.yaml

kubectl get pods

kubectl logs test-pod2

2.3 Using Con?gMap through the data volume plug-in

Using Con?gMap in the data volume is to fill the file into the data volume. In this file, the key is the file name, and the key value is the file content.

vim test-pod3.yaml

apiVersion: v1
kind: Pod
metadata:
  name: test-pod3
spec:
  containers:
  -name: busybox
    image: busybox:1.28.4
    command: [ "/bin/sh", "-c", "sleep 36000" ]
    volumeMounts:
    - name: config-volume
      mountPath: /etc/config
  volumes:
    - name: config-volume
      configMap:
        name: special-config
  restartPolicy: Never


kubectl create -f test-pod3.yaml

kubectl get pods

kubectl exec -it test-pod3 sh
 # cd /etc/config/
 # ls
special.how special.type
 #vi special.how
 # vi special.type

3. Hot update of Con?gMap

vim test-pod4.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: log-config
  namespace: default
data:
  log_level: INFO
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: my-nginx
spec:
  replicas: 1
  template:
    metadata:
      labels:
        run: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx
        ports:
        - containerPort: 80
        volumeMounts:
        - name: config-volume
          mountPath: /etc/config
      volumes:
        - name: config-volume
          configMap:
            name: log-config

kubectl apply -f test-pod4.yaml

kubectl get pods
NAME READY STATUS RESTARTS AGE
my-nginx-76b6489f44-6dwxh 1/1 Running 0 46s

kubectl exec -it my-nginx-76b6489f44-6dwxh -- cat /etc/config/log_level
INFO

kubectl edit configmap log-config
apiVersion: v1
data:
  log_level: DEBUG #INFO changed to DEBUG
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{"log_level":"DEBUG"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"log- config","namespace":"default"}} #INFO changed to DEBUG
  creationTimestamp: 2021-05-25T07:59:18Z
  name: log-config
  namespace: default
  resourceVersion: "93616"
  selfLink: /api/v1/namespaces/default/configmaps/log-config
  uid: 1b8115de-bd2f-11eb-acba-000c29d88bba
  
#Wait for about 10 seconds, use the data in the Volume mounted by the ConfigMap to update synchronously
kubectl exec -it my-nginx-76b6489f44-6dwxh -- cat /etc/config/log_level
DEBUG

Rolling Pods after #ConfigMap update
Updating the ConfigMap currently does not trigger rolling updates of related Pods. You can add version/config to .spec.template.metadata.annotations and modify version/config each time to trigger rolling updates

kubectl patch deployment my-nginx --patch '{"spec": {"template": {"metadata": {"annotations": {"version/config": "20210525" }}}}}'

kubectl get pods
NAME READY STATUS RESTARTS AGE
my-nginx-665dd4dc8c-j4k9t 0/1 ContainerCreating 0 4s
my-nginx-76b6489f44-6dwxh 0/1 Terminating 0 10m

kubectl get pods
NAME READY STATUS RESTARTS AGE
my-nginx-665dd4dc8c-j4k9t 1/1 Running 0 74s

PS: After updating ConfigMap:

  • Envs mounted using this ConfigMap will not be updated synchronously.
  • It takes a while (about 10 seconds in actual measurement) for the data in the Volume mounted using this ConfigMap to be updated synchronously.