Description of the content of the k8s certificate generated by Kubeadm and the extension of the certificate expiration time

Description of the content of the k8s certificate generated by Kubeadm

• Description of the content of the k8s certificate generated by Kubeadm:
• • certificate grouping
  • Kubernetes cluster root certificate
  • • The certificates issued by this root certificate are:
  • kube-apiserver proxy root certificate (client certificate)
  • etcd cluster root certificate
  • • • The server certificate held by etcd server
      • The client certificate used by the nodes in the peer cluster to communicate with each other
      • Define the client certificate used by the Liveness probe in the pod
      • Configure the client certificate used in kube-apiserver for mutual authentication with etcd server
  • Extend k8s certificate time
  • • To extend the certificate expiration time:

Description of the content of the k8s certificate generated by Kubeadm:

Certificate grouping

Kubernetes puts the certificate in two folders

/etc/kubernetes/pki
/etc/kubernetes/pki/etcd

Kubernetes cluster root certificate

Kubernetes cluster root certificate CA (certificate issuing authority for Kubernetes cluster components)

/etc/kubernetes/pki/ca.crt
/etc/kubernetes/pki/ca.key

The above group of certificates is the root certificate used to issue other Kubernetes component certificates, which can be considered as one of the certificate issuing organizations in the Kubernetes cluster

The certificates issued by this root certificate are:

kube-apiserver apiserver certificate

/etc/kubernetes/pki/apiserver.crt
/etc/kubernetes/pki/apiserver.key

Kubelet client certificate, used as client authentication when kube-apiserver actively initiates a request to kubelet

/etc/kubernetes/pki/apiserver-kubelet-client.crt
/etc/kubernetes/pki/apiserver-kubelet-client.key

kube-apiserver proxy root certificate (client certificate)

Used in the requestheader-client-ca-file configuration option, kube-apiserver uses this certificate to verify whether the client certificate is issued by itself

/etc/kubernetes/pki/front-proxy-ca.crt
/etc/kubernetes/pki/front-proxy-ca.key

There is only one set of certificates issued by this root certificate:
The proxy layer (such as the aggregation layer aggregator) uses this set of proxy certificates to request authentication from the kube-apiserver
The client certificate used by the agent, used to authenticate users on behalf of the kube-apiserver

/etc/kubernetes/pki/front-proxy-client.crt
/etc/kubernetes/pki/front-proxy-client.key

etcd cluster root certificate

The certificates used by the etcd cluster are stored in the path /etc/kubernetes/pki/etcd. Obviously, this set of certificates is used exclusively for etcd cluster services. Design the following certificate files
etcd cluster root certificate CA (issuer of all certificates used by etcd)

/etc/kubernetes/pki/etcd/ca.crt
/etc/kubernetes/pki/etcd/ca.key

The certificates issued by this root certificate issuing authority are:

• The server certificate held by etcd server
• The client certificate used by the nodes in the peer cluster to communicate with each other
• Define the client certificate used by the Liveness probe in the pod
• Configure the client certificate used in kube-apiserver for mutual authentication with etcd server

Server certificate held by etcd server

/etc/kubernetes/pki/etcd/server.crt
/etc/kubernetes/pki/etcd/server.key

The client certificate used by the nodes in the peer cluster to communicate with each other

/etc/kubernetes/pki/etcd/peer.crt
/etc/kubernetes/pki/etcd/peer.key

Note: Peer: the name of another Member in the same etcd cluster

Define the client certificate used by the Liveness probe in the pod

The Kubernetes cluster deployed by kubeadm runs the etcd service as a pod. In the definition of the pod, the Liveness detection probe is configured.

/etc/kubernetes/pki/etcd/healthcheck-client.crt
/etc/kubernetes/pki/etcd/healthcheck-client.key

When you describe the pod of etcd, you will see the following line of configuration:

Liveness: exec [/bin/sh -ec ETCDCTL_API=3 etcdctl --endpoints=https://[127.0.0.1]:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt -- cert=/etc/kubernetes/pki/etcd/healthcheck-client.crt --key=/etc/kubernetes/pki/etcd/healthcheck-client.key get foo] delay=15s timeout=15s period=10s #success=1 #failure=8

Configure the client certificate used in kube-apiserver for two-way authentication with etcd server

/etc/kubernetes/pki/apiserver-etcd-client.crt
/etc/kubernetes/pki/apiserver-etcd-client.key

Extend k8s certificate time

Any server executes
Check the validity period of the certificate:

openssl x509 -in /etc/kubernetes/pki/ca.crt -noout -text |grep Not

The display is as follows. From the following, you can see that the ca certificate is valid for 10 years, from 2023 to 2033:

openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text |grep Not

The display is as follows, through which you can see that the apiserver certificate is valid for 1 year

Extend certificate expiration time:

Upload the update-kubeadm-cert.sh file to the k8smaster1 and k8smaster2 nodes, and execute the following on k8smaster1 and k8smaster2:

#!/bin/bash
#Script reproduced from https://github.com/yuyicai/update-kube-cert

set -o errexit
set -o pipefail
# set -o xtrace

log::err() {<!-- -->
  printf "[$(date + '%Y-%m-%dT%H:%M:%S.%N%z')]: \033[31mERROR: \033[0m$@\
"
}

log::info() {<!-- -->
  printf "[$(date + '%Y-%m-%dT%H:%M:%S.%N%z')]: \033[32mINFO: \033[0m$@\
"
}

log::warning() {<!-- -->
  printf "[$(date + '%Y-%m-%dT%H:%M:%S.%N%z')]: \033[33mWARNING: \033[0m$@\
"
}

check_file() {<!-- -->
  if [[ ! -r ${<!-- -->1} ]]; then
    log::err "can not find ${1}"
    exit 1
  the fi
}

# get x509v3 subject alternative name from the old certificate
cert::get_subject_alt_name() {<!-- -->
  local cert=${<!-- -->1}.crt
  check_file "${cert}"
  local alt_name=$(openssl x509 -text -noout -in ${<!-- -->cert} | grep -A1 'Alternative' | tail -n1 | sed 's/[[:space:]]*Address/ /g')
  printf "${alt_name}\
"
}

# get subject from the old certificate
cert::get_subj() {<!-- -->
  local cert=${<!-- -->1}.crt
  check_file "${cert}"
  local subj=$(openssl x509 -text -noout -in ${<!-- -->cert} | grep "Subject:" | sed 's/Subject:/\//g;s/\,/\/ /;s/[[:space:]]//g')
  printf "${subj}\
"
}

cert::backup_file() {<!-- -->
  local file=${<!-- -->1}
  if [[ ! -e ${<!-- -->file}.old-$(date + %Y%m%d) ]]; then
    cp -rp ${<!-- -->file} ${<!-- -->file}.old-$(date + %Y%m%d)
    log::info "backup ${file} to ${file}.old-$(date + %Y%m%d)"
  else
    log::warning "does not backup, ${file}.old-$(date + %Y%m%d) already exists"
  the fi
}

# generate certificate whit client, server or peer
# Args:
# $1 (the name of certificate)
# $2 (the type of certificate, must be one of client, server, peer)
# $3 (the subject of certificates)
# $4 (the validity of certificates) (days)
# $5 (the x509v3 subject alternative name of certificate when the type of certificate is server or peer)
cert::gen_cert() {<!-- -->
  local cert_name=${<!-- -->1}
  local cert_type=${<!-- -->2}
  local subj=${<!-- -->3}
  local cert_days=${<!-- -->4}
  local alt_name=${<!-- -->5}
  local cert=${<!-- -->cert_name}.crt
  local key=${<!-- -->cert_name}.key
  local csr=${<!-- -->cert_name}.csr
  local csr_conf="distinguished_name = dn\
[dn]\
[v3_ext]\
keyUsage = critical, digitalSignature, keyEncipherment\
"

  check_file "${key}"
  check_file "${cert}"

  # backup certificate when certificate not in ${kubeconf_arr[@]}
  # kubeconf_arr=("controller-manager.crt" "scheduler.crt" "admin.crt" "kubelet.crt")
  # if [[ ! "${kubeconf_arr[@]}" =~ "${cert##*/}" ]]; then
  # cert::backup_file "${cert}"
  #fi

  case "${cert_type}" in
    client)
      openssl req -new -key ${<!-- -->key} -subj "${subj}" -reqexts v3_ext \
        -config <(printf "${csr_conf} extendedKeyUsage = clientAuth\
") -out ${<!-- -->csr}
      openssl x509 -in ${<!-- -->csr} -req -CA ${<!-- -->CA_CERT} -CAkey ${<!-- -->CA_KEY} -CAcreateserial -extensions v3_ext \
        -extfile <(printf "${csr_conf} extendedKeyUsage = clientAuth\
") -days ${<!-- -->cert_days} -out ${<!-- -->cert}
      log::info "generated ${cert}"
    ;;
    server)
      openssl req -new -key ${<!-- -->key} -subj "${subj}" -reqexts v3_ext \
        -config <(printf "${csr_conf} extendedKeyUsage = serverAuth\
subjectAltName = ${alt_name}\
") -out ${<!-- -->csr}
      openssl x509 -in ${<!-- -->csr} -req -CA ${<!-- -->CA_CERT} -CAkey ${<!-- -->CA_KEY} -CAcreateserial -extensions v3_ext \
        -extfile <(printf "${csr_conf} extendedKeyUsage = serverAuth\
subjectAltName = ${alt_name}\
") -days ${<!-- -->cert_days} -out ${<!-- -->cert}
      log::info "generated ${cert}"
    ;;
    peer)
      openssl req -new -key ${<!-- -->key} -subj "${subj}" -reqexts v3_ext \
        -config <(printf "${csr_conf} extendedKeyUsage = serverAuth, clientAuth\
subjectAltName = ${alt_name}\
") -out ${<!-- -->csr}
      openssl x509 -in ${<!-- -->csr} -req -CA ${<!-- -->CA_CERT} -CAkey ${<!-- -->CA_KEY} -CAcreateserial -extensions v3_ext \
        -extfile <(printf "${csr_conf} extendedKeyUsage = serverAuth, clientAuth\
subjectAltName = ${alt_name}\
") -days ${<!-- -->cert_days} -out ${<!-- --> cert}
      log::info "generated ${cert}"
    ;;
    *)
      log::err "unknow, unsupported etcd certs type: ${cert_type}, supported type: client, server, peer"
      exit 1
  esac

  rm -f ${<!-- -->csr}
}

cert::update_kubeconf() {<!-- -->
  local cert_name=${<!-- -->1}
  local kubeconf_file=${<!-- -->cert_name}.conf
  local cert=${<!-- -->cert_name}.crt
  local key=${<!-- -->cert_name}.key

  # generate certificate
  check_file ${<!-- -->kubeconf_file}
  # get the key from the old kubeconf
  grep "client-key-data" ${<!-- -->kubeconf_file} | awk {<!-- -->'print$2'} | base64 -d > ${<!-- -->key}
  # get the old certificate from the old kubeconf
  grep "client-certificate-data" ${<!-- -->kubeconf_file} | awk {<!-- -->'print$2'} | base64 -d > ${<!-- -->cert}
  # get subject from the old certificate
  local subj=$(cert::get_subj ${<!-- -->cert_name})
  cert::gen_cert "${cert_name}" "client" "${subj}" "${CAER_DAYS}"
  # get certificate base64 code
  local cert_base64=$(base64 -w 0 ${<!-- -->cert})

  # backup kubeconf
  # cert::backup_file "${kubeconf_file}"

  # set certificate base64 code to kubeconf
  sed -i 's/client-certificate-data:.*/client-certificate-data: '${<!-- -->cert_base64}'/g' ${<!-- -->kubeconf_file}

  log::info "generated new ${kubeconf_file}"
  rm -f ${<!-- -->cert}
  rm -f ${<!-- -->key}

  # set config for kubectl
  if [[ ${<!-- -->cert_name##*/} == "admin" ]]; then
    mkdir -p ~/.kube
    cp -fp ${<!-- -->kubeconf_file} ~/.kube/config
    log::info "copy the admin.conf to ~/.kube/config for kubectl"
  the fi
}

cert::update_etcd_cert() {<!-- -->
  PKI_PATH=${<!-- -->KUBE_PATH}/pki/etcd
  CA_CERT=${<!-- -->PKI_PATH}/ca.crt
  CA_KEY=${<!-- -->PKI_PATH}/ca.key

  check_file "${CA_CERT}"
  check_file "${CA_KEY}"

  # generate etcd server certificate
  # /etc/kubernetes/pki/etcd/server
  CART_NAME=${<!-- -->PKI_PATH}/server
  subject_alt_name=$(cert::get_subject_alt_name ${<!-- -->CART_NAME})
  cert::gen_cert "${CART_NAME}" "peer" "/CN=etcd-server" "${CAER_DAYS}" "${subject_alt_name}"

  # generate etcd peer certificate
  # /etc/kubernetes/pki/etcd/peer
  CART_NAME=${<!-- -->PKI_PATH}/peer
  subject_alt_name=$(cert::get_subject_alt_name ${<!-- -->CART_NAME})
  cert::gen_cert "${CART_NAME}" "peer" "/CN=etcd-peer" "${CAER_DAYS}" "${subject_alt_name}"

  # generate etcd healthcheck-client certificate
  # /etc/kubernetes/pki/etcd/healthcheck-client
  CART_NAME=${<!-- -->PKI_PATH}/healthcheck-client
  cert::gen_cert "${CART_NAME}" "client" "/O=system:masters/CN=kube-etcd-healthcheck-client" "${CAER_DAYS}"

  # generate apiserver-etcd-client certificate
  # /etc/kubernetes/pki/apiserver-etcd-client
  check_file "${CA_CERT}"
  check_file "${CA_KEY}"
  PKI_PATH=${<!-- -->KUBE_PATH}/pki
  CART_NAME=${<!-- -->PKI_PATH}/apiserver-etcd-client
  cert::gen_cert "${CART_NAME}" "client" "/O=system:masters/CN=kube-apiserver-etcd-client" "${CAER_DAYS}"

  # restart etcd
  docker ps | awk '/k8s_etcd/{print$1}' | xargs -r -I '{}' docker restart {<!-- -->} || true
  log::info "restarted etcd"
}

cert::update_master_cert() {<!-- -->
  PKI_PATH=${<!-- -->KUBE_PATH}/pki
  CA_CERT=${<!-- -->PKI_PATH}/ca.crt
  CA_KEY=${<!-- -->PKI_PATH}/ca.key

  check_file "${CA_CERT}"
  check_file "${CA_KEY}"

  # generate apiserver server certificate
  # /etc/kubernetes/pki/apiserver
  CART_NAME=${<!-- -->PKI_PATH}/apiserver
  subject_alt_name=$(cert::get_subject_alt_name ${<!-- -->CART_NAME})
  cert::gen_cert "${CART_NAME}" "server" "/CN=kube-apiserver" "${CAER_DAYS}" "${subject_alt_name}"

  # generate apiserver-kubelet-client certificate
  # /etc/kubernetes/pki/apiserver-kubelet-client
  CART_NAME=${<!-- -->PKI_PATH}/apiserver-kubelet-client
  cert::gen_cert "${CART_NAME}" "client" "/O=system:masters/CN=kube-apiserver-kubelet-client" "${CAER_DAYS}"

  # generate kubeconf for controller-manager, scheduler, kubectl and kubelet
  # /etc/kubernetes/controller-manager,scheduler,admin,kubelet.conf
  cert::update_kubeconf "${KUBE_PATH}/controller-manager"
  cert::update_kubeconf "${KUBE_PATH}/scheduler"
  cert::update_kubeconf "${KUBE_PATH}/admin"
  # check kubelet.conf
  # https://github.com/kubernetes/kubeadm/issues/1753
  set + e
  grep kubelet-client-current.pem /etc/kubernetes/kubelet.conf > /dev/null 2> &1
  kubelet_cert_auto_update=$?
  set -e
  if [[ "$kubelet_cert_auto_update" == "0" ]]; then
    log::warning "does not need to update kubelet.conf"
  else
    cert::update_kubeconf "${KUBE_PATH}/kubelet"
  the fi

  # generate front-proxy-client certificate
  # use front-proxy-client ca
  CA_CERT=${<!-- -->PKI_PATH}/front-proxy-ca.crt
  CA_KEY=${<!-- -->PKI_PATH}/front-proxy-ca.key
  check_file "${CA_CERT}"
  check_file "${CA_KEY}"
  CART_NAME=${<!-- -->PKI_PATH}/front-proxy-client
  cert::gen_cert "${CART_NAME}" "client" "/CN=front-proxy-client" "${CAER_DAYS}"

  # restart apiserve, controller-manager, scheduler and kubelet
  docker ps | awk '/k8s_kube-apiserver/{print$1}' | xargs -r -I '{}' docker restart {<!-- -->} || true
  log::info "restarted kube-apiserver"
  docker ps | awk '/k8s_kube-controller-manager/{print$1}' | xargs -r -I '{}' docker restart {<!-- -->} || true
  log::info "restarted kube-controller-manager"
  docker ps | awk '/k8s_kube-scheduler/{print$1}' | xargs -r -I '{}' docker restart {<!-- -->} || true
  log::info "restarted kube-scheduler"
  systemctl restart kubelet
  log::info "restarted kubelet"
}

main() {<!-- -->
  local node_tpye=$1
  
  KUBE_PATH=/etc/kubernetes
  CAER_DAYS=3650

  # backup $KUBE_PATH to $KUBE_PATH.old-$(date + %Y%m%d)
  cert::backup_file "${KUBE_PATH}"

  case ${<!-- -->node_tpye} in
    etcd)
# update etcd certificates
      cert::update_etcd_cert
    ;;
    master)
# update master certificates and kubeconf
      cert::update_master_cert
    ;;
    all)
      # update etcd certificates
      cert::update_etcd_cert
      # update master certificates and kubeconf
      cert::update_master_cert
    ;;
    *)
      log::err "unknown, unsupported certs type: ${cert_type}, supported types: all, etcd, master"
      printf "Documentation: https://github.com/yuyicai/update-kube-cert
  example:
    '\033[32m./update-kubeadm-cert.sh all\033[0m' update all etcd certificates, master certificates and kubeconf
      /etc/kubernetes
      ├── admin.conf
      ├── controller-manager.conf
      ├── scheduler.conf
      ├── kubelet.conf
      └── pki
          ├── apiserver.crt
          ├── apiserver-etcd-client.crt
          ├── apiserver-kubelet-client.crt
          ├── front-proxy-client.crt
          └── etcd
              ├── healthcheck-client.crt
              ├── peer.crt
              └── server. crt

    '\033[32m./update-kubeadm-cert.sh etcd\033[0m' update only etcd certificates
      /etc/kubernetes
      └── pki
          ├── apiserver-etcd-client.crt
          └── etcd
              ├── healthcheck-client.crt
              ├── peer.crt
              └── server. crt

    '\033[32m./update-kubeadm-cert.sh master\033[0m' update only master certificates and kubeconf
      /etc/kubernetes
      ├── admin.conf
      ├── controller-manager.conf
      ├── scheduler.conf
      ├── kubelet.conf
      └── pki
          ├── apiserver.crt
          ├── apiserver-kubelet-client.crt
          └── front-proxy-client.crt
"
      exit 1
    esac
}

main "$@"

chmod +x update-kubeadm-cert.sh

Execute the following command to modify the certificate expiration time and extend it to 10 years

./update-kubeadm-cert.sh all

Query whether the Pod is normal on the k8smaster1 node, and the data can be queried to indicate that the certificate has been issued

kubectl get pods -n kube-system
#The display is as follows, you can see the pod information, indicating that the certificate is issued normally:

openssl x509 -in /etc/kubernetes/pki/ca.crt -noout -text |grep Not

openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text |grep Not

The display is as follows. From the following, you can see that the apiserver certificate is valid for 10 years, from 2025 to 2035: