Deploy heterogeneous clusters for existing TiDB clusters through TiDB Operator

Author: lqbyz Original source: https://tidb.net/blog/21dee5a3

This document describes how to deploy a cluster built with different service components for an existing tidb cluster. A heterogeneous cluster is a cluster composed of nodes with different configurations from the existing TiDB cluster.

Applicable scenarios

Applicable to scenarios where you need to create instance nodes with differentiated configurations based on an existing TiDB cluster, for example:

Create TiKV clusters with different configurations and labels for hotspot scheduling
Create TiDB clusters with different configurations for OLTP and OLAP queries respectively

Preconditions

A TiDB cluster already exists. If you have not deployed a TiDB cluster, please build a TiDB cluster in the K8S environment now.

Deploying heterogeneous clusters

Depending on whether TLS needs to be enabled for heterogeneous clusters, it can be divided into method 1 and method 2, which are optional.

Method 1, non-TLS deployment

Create a new cluster configuration file for heterogeneous clusters.

Execute the following command to create a heterogeneous cluster configuration file, where origin_cluster_name is the name of the original cluster to be joined, and heterogeneous_cluster_name is the name of the heterogeneous cluster, for subsequent viewing in Grafana of TidbMonitor For the monitoring data of the original cluster and heterogeneous cluster, please name the heterogeneous cluster with the prefix of the original cluster name.

origin_cluster_name=basic
heterogeneous_cluster_name=basic-heterog
cat > cluster.yaml << EOF
apiVersion: pingcap.com/v1alpha1
kind: TidbCluster
metadata:
  name: ${heterogeneous_cluster_name}
spec:
  configUpdateStrategy: RollingUpdate
  version: v6.5.0
  timezone: UTC
  pvReclaimPolicy: Delete
  discovery: {}
  cluster:
    name: ${origin_cluster_name}
  tikv:
    baseImage: pingcap/tikv
    maxFailoverCount: 0
    replicas: 1
    # If storageClassName is not set, TiDB Operator will use the default Storage Class of the Kubernetes cluster
    # storageClassName: local-storage
    requests:
      storage: "100Gi"
    config: {}
  tidb:
    baseImage: pingcap/tidb
    maxFailoverCount: 0
    replicas: 1
    service:
      type: ClusterIP
    config: {}
  tiflash:
    baseImage: pingcap/tiflash
    maxFailoverCount: 0
    replicas: 1
    storageClaims:
      - resources:
          requests:
            storage: 100Gi
EOF

Compared with the common TiDB cluster configuration file, the only difference between the heterogeneous cluster configuration file is that you need to additionally configure the spec.cluster.name field as the existing TiDB cluster name. Through this field, TiDB Operator will add the heterogeneous cluster to the existing TiDB cluster.

Modify the configuration items of each component node in the heterogeneous cluster configuration file as needed

If you do not need or delete unnecessary components, you can not add related configuration or replicas is 0

apiVersion: pingcap.com/v1alpha1
kind: TidbCluster
metadata:
  name: idc
  namespace: lqbyz
spec:
  configUpdateStrategy: RollingUpdate
  version: v6.5.0
  timezone: UTC
  pvReclaimPolicy: Delete
  discovery: {}
  cluster:
    name: tidb-test
  tikv:
    baseImage: pingcap/tikv
    maxFailoverCount: 0
    replicas: 3
# # If storageClassName is not set, TiDB Operator will use the default Storage Class of the Kubernetes cluster
    storageClassName: local-hostpath
    requests:
      storage: "20Gi"
    config: {}
  tidb:
    baseImage: pingcap/tidb
    maxFailoverCount: 0
    replicas: 3
    service:
      type: ClusterIP
    config: {}
  tiflash:
    baseImage: pingcap/tiflash
    maxFailoverCount: 0
    replicas: 1
    storageClaims:
      - resources:
          requests:

Create a heterogeneous cluster

kubectl create -f cluster.yaml -n ${namespace}

Method 2, TLS deployment

To enable heterogeneous cluster TLS, you need to display the statement, and you need to create a new Secret certificate file, which is issued by the same CA (Certification Authority) as the target cluster. If you use the cert-manager method, you need to use the same Issuer as the target cluster to create a Certificate .

For detailed steps to create certificates for heterogeneous clusters, refer to the following documents:

Enable TLS between TiDB components
Enable TLS for MySQL client

After creating the certificate, to deploy a heterogeneous cluster with TLS enabled, do the following:

Create a new cluster configuration file for heterogeneous clusters.

apiVersion: pingcap.com/v1alpha1
kind: TidbCluster
metadata:
  name: ${heterogeneous_cluster_name}
spec:
  tlsCluster:
    enabled: true
  configUpdateStrategy: RollingUpdate
  version: v6.5.0
  timezone: UTC
  pvReclaimPolicy: Delete
  discovery: {}
  cluster:
    name: ${origin_cluster_name}
  tikv:
    baseImage: pingcap/tikv
    maxFailoverCount: 0
    replicas: 1
    # If storageClassName is not set, TiDB Operator will use the default Storage Class of the Kubernetes cluster
    # storageClassName: local-storage
    requests:
      storage: "100Gi"
    config: {}
  tidb:
    baseImage: pingcap/tidb
    maxFailoverCount: 0
    replicas: 1
    service:
      type: ClusterIP
    config: {}
    tlsClient:
      enabled: true
  tiflash:
    baseImage: pingcap/tiflash
    maxFailoverCount: 0
    replicas: 1
    storageClaims:
      - resources:
          requests:
            storage: 100Gi

Modify the configuration items of each node in the heterogeneous cluster configuration file as needed.

For example, you can modify the number of replicas for each component in the cluster.yaml file or delete unnecessary components.

Execute the following command to create a heterogeneous cluster with TLS enabled. You need to replace `cluster.yaml` with your heterogeneous cluster configuration file name.

kubectl create -f cluster.yaml -n ${namespace}

If the execution is successful, the output will prompt tidbcluster.pingcap.com/${heterogeneous_cluster_name} created . TiDB Operator will create a correspondingly configured TiDB cluster according to the cluster configuration file.

Deployment cluster monitoring

If you need to deploy monitoring for heterogeneous clusters, please add the name of the heterogeneous cluster to the TidbMonitor CR file of the existing TiDB cluster. The specific operation is as follows:

Edit the TidbMonitor Custom Resource (CR) of an existing TiDB cluster:

kubectl edit tm ${cluster_name} -n ${namespace}

Refer to the following example, replace `${origin_cluster_name}` with the name of the cluster you want to join, and replace `${heterogeneous_cluster_name}` with the name of the heterogeneous cluster:

apiVersion: pingcap.com/v1alpha1
kind: TidbMonitor
metadata:
name: heterogeneous
spec:
clusters:
    - name: ${origin_cluster_name}
    - name: ${heterogeneous_cluster_name}
prometheus:
    baseImage: prom/prometheus
    version: v2.27.1
grafana:
    baseImage: grafana/grafana
    version: 7.5.11
initializer:
    baseImage: pingcap/tidb-monitor-initializer
    version: v6.5.0
reloader:
    baseImage: pingcap/tidb-monitor-reloader
    version: v1.0.1
prometheusReloader:
    baseImage: quay.io/prometheus-operator/prometheus-config-reloader
    version: v0.49.0
imagePullPolicy: IfNotPresent

The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge Java skill treeHomepageOverview 108572 people are studying systematically

Applicable scenarios

Preconditions

Deploying heterogeneous clusters

Method 1, non-TLS deployment

Create a new cluster configuration file for heterogeneous clusters.

Modify the configuration items of each component node in the heterogeneous cluster configuration file as needed

Create a heterogeneous cluster

Method 2, TLS deployment

Create a new cluster configuration file for heterogeneous clusters.

Modify the configuration items of each node in the heterogeneous cluster configuration file as needed.

Execute the following command to create a heterogeneous cluster with TLS enabled. You need to replace cluster.yaml with your heterogeneous cluster configuration file name.

Deployment cluster monitoring

Edit the TidbMonitor Custom Resource (CR) of an existing TiDB cluster:

Refer to the following example, replace ${origin_cluster_name} with the name of the cluster you want to join, and replace ${heterogeneous_cluster_name} with the name of the heterogeneous cluster:

Execute the following command to create a heterogeneous cluster with TLS enabled. You need to replace `cluster.yaml` with your heterogeneous cluster configuration file name.

Refer to the following example, replace `${origin_cluster_name}` with the name of the cluster you want to join, and replace `${heterogeneous_cluster_name}` with the name of the heterogeneous cluster: