Redis Cluster (Redis Cluster), uses the cluster function that comes with Redis to build a masterless mode cluster

Article directory

    • I. Overview
    • 2. Simulation configuration instructions
    • 3. Create Redis Cluster using script
      • 3.1 Configuration creation script
      • 3.2 Start the cluster instance
      • 3.3 Create a cluster
      • 3.4 Test cluster
      • 3.5 Stop the cluster instance
      • 3.6 Delete (clear) the cluster
    • 4. Manually create a cluster Redis Cluster
      • 4.1 Start the cluster instance
      • 4.2 Manually create a cluster
      • 4.4 Test cluster
    • 5. Cluster management
      • 5.1 Move slot
      • 5.2 View basic information of the cluster
      • 5.3 View cluster details

If your understanding of Redis is not deep enough, please pay attention to this column. This column includes Redis installation, Redis configuration file description, Redis command and data type description, Redis persistence configuration, Redis master-slave replication and sentinel mechanism.

1. Overview

  • Redis is an open source in-memory database that supports distributed deployment. Redis Clusteris a distributed solution for Redis that provideshigh availability and scalability. Redis Cluster achieves data distribution and redundancy through sharding and data replication to provide high performance and fault tolerance.
  • The principle of clustering is data partitioning, that is, data is dispersed to multiple nodes to achieve distributed storage and processing. However, this dispersion of data can make aggregation operations (such as transactions) more complex in some cases.
  • In Redis applications, to enhance availability you can use master-standby mode and master-slave mode (please see Redis master-slave mode configuration). This is a full master-slave replication mechanism, where the slave is a mirror of the master. However, this method cannot solve the problem of increasing data volume as business increases (because they are mirrors and cannot expand capacity). To solve the capacity problem, cluster is needed. strong>.
  • Redis Cluster distributes business data to different master-slave nodes through logical rules, solving the problem of a single master-slave node (equivalent to a high-availability configuration of the master node). On the one hand, it improves scalability, and on the other hand, it also improves availability (because the data is not on the same master-slave node, there is generally no problem of all data failing at the same time).
  • If there is no Redis cluster, our manual implementation of data partitioning logic is generally as follows:
    1. Segment according to business scenarios. For example, split by function: users, orders, products, etc. are assigned to different Redis master-slave nodes.
    2. If the requirements cannot be met after splitting according to business, it can also be split according to priority and logic (for example, one Redis master-slave node for every 10,000 IDs).
  • The default Redis Cluster uses the concept of Hash Ring to fragment and manage data. A hash ring is a data structure that maps data nodes (Redis instances) to a ring space. In Redis Cluster, each node is assigned a slot, with a total of 16384 slots. Data is mapped to the corresponding slot based on the hash value of the key. Assume that there are three master-slave Redis clusters, and the slot distribution is as follows:
  • Redis Cluster uses a sharding strategy called Hash Slot Sharding. Each node is responsible for managing a part of the slots and the data in the corresponding slots. When the client sends a command to Redis Cluster, the hash value is first calculated based on the key in the command, and the corresponding slot is determined based on the hash value. Then, the client sends the command to the node responsible for the slot to complete the data operation (so it is a masterless mode cluster, that is, no master-slave node is the master node, but based on slot calculation). In this way, data is evenly distributed across the nodes in the entire cluster.
  • The use of hash rings enables Redis Cluster to have distributed capabilities while ensuring high availability and scalability. When a node joins or leaves the cluster, Redis Cluster automatically reallocates slots to maintain data balance and consistency. This sharding method enables Redis Cluster to scale horizontally and support processing large-scale data and loads.

2. Simulation configuration instructions

  • Below we use three master-slave Redis to form a Redis Cluster (officially, a Redis Cluster requires at least three master-slave Redis. Please see here for the Redis master-slave mode configuration). The ports of the Redis Cluster created by default are 30001, 30002, 30003, 30004, 30005, and 30006, as shown in the diagram below (the actual port of the cluster may not match yours, but the idea is this):

3. Create Redis Cluster via script

  • First, we use the official Redis script to create a Redis Cluster to see the effect. The advantage of using the official Redis script is that it is convenient, because it will automatically create three master-slave Redis, and then create a cluster based on the three master-slave Redis.

3.1 Configuration creation script

  • There is a create-cluster script in the utils/create-cluster/ directory of the Redis source code. Open this script and modify the configuration (if you do not modify it, some create 9 Redis instances, and the host address is 127.0.0.1. If you don’t think so You don’t have to modify it if it’s troublesome. Just access it according to the default configuration after creation. I’ve provided a demonstration here for easy modification). For information on Redis source code compilation and installation, please see here.
cd utils/create-cluster/
vi create-cluster
  • The main contents of modifying the create-cluster script are as follows:
# Total number of test nodes
NODES=6
# Number of slaves, which is equivalent to three masters and three slaves (master=total/slave + 1)
REPLICAS=1

# If it is not a local test, the following configuration is very important (for details, please refer to the Redis configuration file description in this column)
# Configure the cluster host information, 192.168.8.60. Change the IP of your host as needed.
CLUSTER_HOST=192.168.8.60
# Whether to enable protected mode
PROTECTED_MODE=no

3.2 Start a cluster instance

  • Use the create-cluster script in the utils/create-cluster/ directory and add the start parameter to start the cluster instance.
./create-cluster start

[root@yiqifu-redis create-cluster]# ./create-cluster start
Starting 30001
Starting 30002
Starting 30003
Starting 30004
Starting 30005
Starting 30006

  • Because to build a Redis cluster, at least 3 master nodes are required. Generally there are 3 masters and 3 slaves, so at least 6 nodes are required. This script directly creates 6 instances for us.

3.3 Create a cluster

  • Use the create-cluster script in the utils/create-cluster/ directory and add the create parameter to create a Redis instance as a cluster.
./create-cluster create

[root@yiqifu-redis create-cluster]# ./create-cluster create
>>> Performing hash slots allocation on 6 nodes…
Master[0] -> Slots 0 – 5460
Master[1] -> Slots 5461 – 10922
Master[2] -> Slots 10923 – 16383
Adding replica 127.0.0.1:30005 to 127.0.0.1:30001
Adding replica 127.0.0.1:30006 to 127.0.0.1:30002
Adding replica 127.0.0.1:30004 to 127.0.0.1:30003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: cf12868ca9b7049ba7b73757aed83cf56e3e6b09 127.0.0.1:30001
slots:[0-5460] (5461 slots) master
M: f8521587351d126ee6618fee70bdc994c598fab0 127.0.0.1:30002
slots:[5461-10922] (5462 slots) master
M: 764d51f5a7a986f424cb218794a3fcbe493913b1 127.0.0.1:30003
slots:[10923-16383] (5461 slots) master
S: 46233ddb3462efcbf37b2d1e7229f9a54e36d9e8 127.0.0.1:30004
replicates f8521587351d126ee6618fee70bdc994c598fab0
S: 31d9d8aa6927509ec33b7534a3ddeb3e921ce171 127.0.0.1:30005
replicates 764d51f5a7a986f424cb218794a3fcbe493913b1
S: 89415c9c43b4515441f49ac285d2e0e1a80b4525 127.0.0.1:30006
replicates cf12868ca9b7049ba7b73757aed83cf56e3e6b09

Can I set the above configuration? (type yes’ to accept): yes

>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join

Performing Cluster Check (using node 127.0.0.1:30001)
M: cf12868ca9b7049ba7b73757aed83cf56e3e6b09 127.0.0.1:30001
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: 31d9d8aa6927509ec33b7534a3ddeb3e921ce171 127.0.0.1:30005
slots: (0 slots) slave
replicates 764d51f5a7a986f424cb218794a3fcbe493913b1
M: 764d51f5a7a986f424cb218794a3fcbe493913b1 127.0.0.1:30003
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
M: f8521587351d126ee6618fee70bdc994c598fab0 127.0.0.1:30002
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 46233ddb3462efcbf37b2d1e7229f9a54e36d9e8 127.0.0.1:30004
slots: (0 slots) slave
replicates f8521587351d126ee6618fee70bdc994c598fab0
S: 89415c9c43b4515441f49ac285d2e0e1a80b4525 127.0.0.1:30006
slots: (0 slots) slave
replicates cf12868ca9b7049ba7b73757aed83cf56e3e6b09
[OK] All nodes agree about slots configuration.
>>> Check for open slots…
>>> Check slots coverage…
[OK] All 16384 slots covered.

3.4 Test Cluster

  • Use redis-cli to connect to the cluster for testing. Since Redis Cluster is a masterless cluster, the cluster we created has three masters and slaves. So you can use any main port when connecting (such as: redis-cli -c -p 30001, redis-cli -c -p 30002, redis-cli -c -p 30003). When used, the cluster will calculate the slot based on the hash of the key, and then locate the corresponding master and slave to access the data. The following test example
# -c means connecting in a cluster mode
# -p specifies port
redis-cli -c -p 30001

[root@yiqifu-redis create-cluster]# redis-cli -c -p 30001
127.0.0.1:30001> set aaa 111
-> Redirected to slot [10439] located at 127.0.0.1:30002
OK
127.0.0.1:30002> set bbb 222
-> Redirected to slot [5287] located at 127.0.0.1:30001
OK
127.0.0.1:30001> set ccc 333
OK
127.0.0.1:30001> set {g1}aaa 111
-> Redirected to slot [13519] located at 127.0.0.1:30003
OK
127.0.0.1:30003> set {g1}bbb 222
OK
127.0.0.1:30003> set {g1}ccc 333
OK
127.0.0.1:30003> set {g1}ddd 444
OK
127.0.0.1:30003> set {g1}eee 555
OK

127.0.0.1:30003> multi
OK
127.0.0.1:30003> set {g1}bbb 222222
QUEUED
127.0.0.1:30003> set {g1}ccc 333333
QUEUED
127.0.0.1:30003>exec

1) OK
2) OK

127.0.0.1:30003>

  • When using, you can also use “{group name} key” to group different keys (Key) into a group and put them in the same Redis master. You can use transactions, monitoring and other functions in the same Redis master. For example: “{g1}” is used here for namespace isolation, which means that keys with “{g1}” are assigned to the same redis, so that operations such as opening transactions in the same Redis can be facilitated.
  • Through the above test, we can find that the Redis Cluster masterless mode cluster will calculate and store the master-slave Redis based on the key value you enter, and will automatically jump to the corresponding master-slave Redis node.

3.5 Stop the cluster instance

  • Use the create-cluster script in the utils/create-cluster/ directory and add the stop parameter to stop Redis Cluster.
./create-cluster stop

[root@yiqifu-redis create-cluster]# ./create-cluster stop
Stopping 30001
Stopping 30002
Stopping 30003
Stopping 30004
Stopping 30005
Stopping 30006

3.6 Delete (clear) cluster

  • Use the create-cluster script in the utils/create-cluster/ directory with the clean parameter to clear all Redis Cluster configurations (return to the original state).
./create-cluster clean

4. Manually create a cluster Redis Cluster

  • In the actual production environment, the Redis master and slave services are manually configured, and the Redis Cluster also needs to be created manually. The following demonstrates manually creating a Redis Cluster.

4.1 Start a cluster instance

  • When testing here, I still used a script to start 6 instances first (starting the instance only starts the Redis service and does not create a cluster). In the production environment, these Redis instances are configured by you on different servers.
  • The IPs and ports of my six Redis instances are as follows: 127.0.0.1:30001 127.0.0.1:30002 127.0.0.1:30003 127.0.0.1:30004 127.0.0.1:30005 127.0.0.1:30006. Pay attention to this information, it will be useful later when creating a cluster.
./create-cluster start

[root@yiqifu-redis create-cluster]# ./create-cluster start
Starting 30001
Starting 30002
Starting 30003
Starting 30004
Starting 30005
Starting 30006

  • If you configure the redis.conf file yourself and then start the instance through redis-server redis.conf, then the following configuration must be modified correctly to create a cluster.

    # port 6391

cluster-enabled yes

# Configure the cluster configuration file as nodes.conf
# nodes.conf does not need to be created manually, it will be created automatically when the cluster starts.
cluster-config-file nodes.conf

cluster-node-timeout 5000

appendonly yes

4.2 Manually create a cluster

  • Use the command to manually create a cluster based on existing Redis service instances (these instances are your Redis services distributed on different hosts).

    redis-cli --cluster create 127.0.0.1:30001 127.0.0.1:30002 127.0.0.1:30003 127.0.0.1:30004 127.0.0.1:30005 127.0.0.1:30006 --cluster-replicas 1

  • Parameter Description

    • The –cluster create parameter indicates creating a cluster
    • –cluster-replicas 1 specifies the number of slave nodes under each master-slave Redis. From this, it can be deduced that the number of master-slave in the Redis cluster = total number of nodes / (number of slave nodes in each master-slave mode + 1) = 6 / (1 + 1) = 3 master-slave Redis.

4.4 Test Cluster

  • Same as the script creation method, please test it yourself
# -c means connecting in a cluster mode
redis-cli -c -p 30001

5. Cluster management

5.1 Moving slots

  • Why move slots? I think of the following situations:

    • Suppose you adda new Redis master and slave, you need to allocate certain slots of the original cluster to the new Redis master and slave.
    • Suppose a Redis master-slave in your cluster is broken, then you need to distribute the broken slot equally to other Redis masters-slave in the cluster.
    • Suppose that one of your Redis masters and slaves has a lower hardware configuration and another Redis master and slave has a higher hardware configuration. You can allocate some slots with lower hardware configurations. Go to the Redis master-slave with higher hardware configuration.

  • In Redis, slots are the concept used for sharding data. Redis Cluster uses slots to distribute data to multiple nodes to achieve horizontal expansion and improve system scalability and availability. Each Redis Cluster contains 16384 slots, which can be used to distribute data.

  • There is some information when creating a cluster, which describes the slot information assigned to each master-slave node. Here, for example, the slot information of Master[0] is 0 ~ 5460. And the ID and ownership relationship of each master-slave node. As follows, pay attention to this information, which will be used later.

Master[0] -> Slots 0 – 5460
Master[1] -> Slots 5461 – 10922
Master[2] -> Slots 10923 – 16383

M: fc253a3c9f2c301f7c5f6e72b47a338237b3447d 127.0.0.1:30001
slots:[0-5460] (5461 slots) master
M: ab3db8a2ed5fc4066c3257344618d02059e88648 127.0.0.1:30002
slots:[5461-10922] (5462 slots) master
M: 5d4d3f51cf968ca13718e378b5ffd68452d70479 127.0.0.1:30003
slots:[10923-16383] (5461 slots) master
S: a82a0b7c77addee864259f02a6c38f47745c5da0 127.0.0.1:30004
replicates 5d4d3f51cf968ca13718e378b5ffd68452d70479
S: b2944e78407304948150003bc395de9165354010 127.0.0.1:30005
replicates fc253a3c9f2c301f7c5f6e72b47a338237b3447d
S: 934fc2185bd12ff361fd40ae1b26bd0355bdf6e3 127.0.0.1:30006
replicates ab3db8a2ed5fc4066c3257344618d02059e88648

  • Move slot command syntax: redis-cli –cluster reshard any master node in the cluster

    • The –cluster reshard parameter indicates moving slots

  • As follows, we move the 2000 slots of the host (127.0.0.1:30001) to the host (127.0.0.1:30002).

    In the process, you need to check the ID of the master and slave in the cluster. If there is no record before, you can read the command in Section 5.3 first.

# Enter the cluster move slot command (in my configuration environment, any of the following commands will work)
redis-cli --cluster reshard 127.0.0.1:30001
# redis-cli --cluster reshard 127.0.0.1:30002
# redis-cli --cluster reshard 127.0.0.1:30003
  • After executing the above command, you will be prompted “How many slots do you want to move (from 1 to 16384)?”
    • It means “How many slots do you want to move (the value range is 1~16384)?”, but you can see from the log that it is actually (0 – 5460), for example, I fill in: 2000.
  • Then it will prompt “What is the receiving node ID?”
    • It means “Which master-slave node in the cluster accepts this 2000 slot information?” You can see the ID of the host (127.0.0.1:30002) through the log. For example, if I move to the second node, fill in: ab3db8a2ed5fc4066c3257344618d02059e88648
  • Then it prompts “Source node #1:”
    • It means “move these 2000 slots from that node”. You can see the ID of the host (127.0.0.1:30001) through the log. For example, if I select the first node, fill in: fc253a3c9f2c301f7c5f6e72b47a338237b3447d
  • Then enter “done” when prompted “Source node #2:”. Then fill in yes all the way to complete

5.2 View basic cluster information

  • Basic syntax: redis-cli –cluster info any master node in the cluster
redis-cli --cluster info 127.0.0.1:30001

[root@yiqifu-redis create-cluster]# redis-cli –cluster info 127.0.0.1:30001
127.0.0.1:30001 (fc253a3c…) -> 0 keys | 3461 slots | 1 slaves.
127.0.0.1:30003 (5d4d3f51…) -> 0 keys | 7461 slots | 1 slaves.
127.0.0.1:30002 (ab3db8a2…) -> 0 keys | 5462 slots | 1 slaves.

  • Here you can see the basic information of the cluster and the total number of slots.

5.3 View cluster details

  • Basic syntax: redis-cli –cluster check any master node in the cluster
redis-cli --cluster check 127.0.0.1:30001

[root@yiqifu-redis create-cluster]# redis-cli –cluster check 127.0.0.1:30001
127.0.0.1:30001 (fc253a3c…) -> 0 keys | 3461 slots | 1 slaves.
127.0.0.1:30003 (5d4d3f51…) -> 0 keys | 7461 slots | 1 slaves.
127.0.0.1:30002 (ab3db8a2…) -> 0 keys | 5462 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 127.0.0.1:30001)
M: fc253a3c9f2c301f7c5f6e72b47a338237b3447d 127.0.0.1:30001
slots:[2000-5460] (3461 slots) master
1 additional replica(s)
M: 5d4d3f51cf968ca13718e378b5ffd68452d70479 127.0.0.1:30003
slots:[0-1999],[10923-16383] (7461 slots) master
1 additional replica(s)
S: 934fc2185bd12ff361fd40ae1b26bd0355bdf6e3 127.0.0.1:30006
slots: (0 slots) slave
replicates ab3db8a2ed5fc4066c3257344618d02059e88648
M: ab3db8a2ed5fc4066c3257344618d02059e88648 127.0.0.1:30002
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: a82a0b7c77addee864259f02a6c38f47745c5da0 127.0.0.1:30004
slots: (0 slots) slave
replicates 5d4d3f51cf968ca13718e378b5ffd68452d70479
S: b2944e78407304948150003bc395de9165354010 127.0.0.1:30005
slots: (0 slots) slave
replicates fc253a3c9f2c301f7c5f6e72b47a338237b3447d
[OK] All nodes agree about slots configuration.
>>> Check for open slots…
>>> Check slots coverage…
[OK] All 16384 slots covered.

  • Cluster details can be seen here. The specific range of each master-slave slot, such as: slots:[0-1999],[10923-16383] (7461 slots) master