Redis Twemproxy cluster, horizontal expansion, expansion plan

Article directory

    • I. Overview
    • 2. Twemproxy distribution mode
    • 3. Test planning
    • 4. Redis service instance preparation
      • 4.1 Configure Redis instance
      • 4.2 Create related resources
      • 4.3 Start the Redis service instance
    • 5. Twemproxy installation preparation
    • 6. Twemproxy installation and cluster configuration
      • 6.1 Install Twemproxy
      • 6.2 Configuring Twemproxy
      • 6.3 Start twemproxy
      • 6.4 Testing the twemproxy cluster

If your understanding of Redis is not deep enough, please pay attention to this column. This column includes Redis installation, Redis configuration file description, Redis command and data type description, Redis persistence configuration, Redis master-slave replication and sentry mechanism, Redis Cluster (cluster) configuration , Redis Predixy cluster.

1. Overview

  • Twemproxy (also known as nutcracker) is an open source proxy software used to provide load balancing and high availability of Redis or Memcached data stores. It is a proxy layer that allows clients to send requests to multiple Redis or Memcached backend servers and load balance across multiple nodes. It was also developed by Twitter Inc and has become one of the common tools in the Redis and Memcached communities.

  • I talked about the Redis Cluster cluster configuration before, so why use Twemproxy since the official Redis Cluster comes with it? Because Twemproxy also has functions that Redis Cluster does not have, as follows:

    • Multiple deployments: Redis Cluster only supports one specific hash sharding architecture, while Twemproxy can be applied to multiple deployment topologies. Moreover, after adding a Redis cluster, the application code does not need to be modified, not even the configuration.
    • Dynamic load balancing: Based on real-time node status and load conditions, Twemproxy intelligently routes requests to the best Redis node to achieve load balancing.
    • Read-write separation: Twemproxy supports read-write separation configuration. It can route read operations to slave nodes to share the load on the master node and improve read performance. At the same time, write operations are still routed to the primary node to ensure data consistency.

  • Compared with Predixy, Twemproxy has the advantage of being more lightweight and more efficient. However, the community activity is low and there are relatively few documents.

  • Open source address: https://github.com/twitter/twemproxy

2. Twemproxy distribution mode

  • Twemproxy routes requests to which backend server supports the following three distribution modes:

    1. ketama (consistent hashing): This is the default and recommended distribution mode. It is a distribution method based on a consistent hashing algorithm, ensuring that when backend servers are added or removed, most requests can still be mapped to the correct backend server. This helps keep the load balanced and minimizes data migration.

    2. modula (modulo hashing): This is another distribution mode that uses the modulo (remainder) operation in a hash operation to select a backend server. It usually requires knowing the number of servers in advance and calculating the hash value based on this number. This pattern can result in uneven load distribution, especially when servers are added or removed.

    3. random: This distribution mode is based on randomly selecting backend servers. For each key for each request, it will randomly select a backend server. This can lead to uneven load distribution and is not suitable for situations where a certain degree of load balancing is required.

  • Consistent hashing (ketama) is the most commonly used distribution mode because it provides relatively stable distribution when servers are added or removed, thereby maintaining system availability and load balancing. Other modes may be suitable for specific use cases, but require more careful configuration and monitoring to ensure load balancing and performance. The choice of which distribution mode to choose will often be determined by specific application needs and circumstances.

3. Test planning

  • Here I open two Redis service instances on the same host, and then use the Twemproxy cluster to test. The diagram is as follows:

4. Redis service instance preparation

4.1 Configure Redis instance

  • Create 2 new Redis configuration files to configure 2 Redis service instance nodes. They are: redis_6381.conf, redis_6382.conf.

  • redis_6381.conf

include redis.conf

port 6381
dir /var/lib/redis/6381
pidfile /var/run/redis_6381.pid

supervised no
daemonize no
logfile ""
appendonly no
  • redis_6382.conf
include redis.conf

port 6382
dir /var/lib/redis/6382
pidfile /var/run/redis_6382.pid

supervised no
daemonize no
logfile ""
appendonly no

4.2 Create related resources

mkdir -p /var/lib/redis/6381
mkdir -p /var/lib/redis/6382

4.3 Start Redis service instance

  • Start 2 Redis service instances
redis-server redis_6381.conf
redis-server redis_6382.conf

5. Twemproxy installation preparation

  • Install the Epel software repository

    • Official address: https://developer.aliyun.com/mirror/
    wget -O /etc/yum.repos.d/epel.repo https://mirrors.aliyun.com/repo/epel-7.repo
yum clean all

  • Prepare compilation tools

    yum install automake libtool -y

  • Upgrade autoreconf

    yum search autoconf
yum install autoconf268

6. Twemproxy installation and cluster configuration

6.1 Install Twemproxy

  • On CentOS7 and above systems, execute the following commands in order to install. If you cannot download using git during the installation process, you can download it manually and then copy it to the target machine. The target program generated after the installation is completed is called “nutcracker”. Later configurations will use “nutcracker” instead of “twemproxy”.
git clone https://github.com/twitter/twemproxy.git
cd twemproxy/
autoreconf -fvi

# If you use autoreconf -fvi to report an error, use the following command
#autoreconf268 -fvi

./configure --enable-debug=full
make

mkdir /etc/nutcracker
cp conf/* /etc/nutcracker/
cp /etc/nutcracker/nutcracker.yml /etc/nutcracker/nutcracker.yml.bk

cp src/nutcracker /usr/bin/

# cp scripts/nutcracker.init /etc/init.d/nutcracker
# chmod a + x /etc/init.d/nutcracker

6.2 Configuring Twemproxy

  • Modify the configuration file nutcracker.yml
vi /etc/nutcracker/nutcracker.yml
  • The content is as follows
alpha:
  listen: 127.0.0.1:22121
  hash: fnv1a_64
  distribution: ketama
  auto_eject_hosts: true
  redis: true
  server_retry_timeout: 2000
  server_failure_limit: 1
  servers:
   - 127.0.0.1:6381:1
   - 127.0.0.1:6382:1

6.3 Start twemproxy

nutcracker -d -c /etc/nutcracker/nutcracker.yml

6.4 Test twemproxy cluster

redis-cli -p 22121

[root@yiqifu-centos conf]# redis-cli -p 22121
127.0.0.1:22121> set aaa 111
OK
127.0.0.1:22121> set bbb 222
OK
127.0.0.1:22121>