Spring Cloud Practical Combat | Decrypting the underlying principles of load balancing Ribbon, including practical source code

A collection of columns that you can save for emergencies

Spring Cloud practical column: https://blog.csdn.net/superdangbo/category_9270827.html

Python practical column: https://blog.csdn.net/superdangbo/category_9271194.html

Logback detailed explanation column: https://blog.csdn.net/superdangbo/category_9271502.html

tensorflow column: https://blog.csdn.net/superdangbo/category_8691332.html

Redis column: https://blog.csdn.net/superdangbo/category_9950790.html

1024 Programmers Day special article:

1024 Programmers Carnival Special | ELK + collaborative filtering algorithm builds a personalized recommendation engine to intelligently realize “thousands of people, thousands of faces”

1024 Programmer’s Day Special | Deciphering Spring Cloud Hystrix Meltdown to Improve System Availability and Fault Tolerance

1024 Programmer’s Day Special | ELK + user portraits build a personalized recommendation engine to intelligently realize “thousands of people, thousands of faces”

1024 Programmer’s Day Special | OKR VS KPI, who is more suitable?

1024 Programmers Day Special | Spring Boot Practical MongoDB Sharding or Replica Set Operation

Spring practical series of articles:

Spring Practical | Spring AOP Core Tips – Sunflower Collection

Spring Practice | The secret that Spring IOC cannot tell?

National Day and Mid-Autumn Festival special series of articles:

National Day and Mid-Autumn Festival Special (8) How to use JPA in Spring Boot projects

National Day and Mid-Autumn Festival Special (7) 20 Common Programming Interview Questions for Java Software Engineers

National Day and Mid-Autumn Festival Special (6) 30 Common Treasure Programming Interview Questions for College Students

National Day and Mid-Autumn Festival Special (5) How to performance tune MySQL? Next article

National Day and Mid-Autumn Festival Special (4) How to performance tune MySQL? Previous article

National Day and Mid-Autumn Festival Special (3) Use Generative Adversarial Network (GAN) to generate paintings with a festive atmosphere, implemented by the deep learning frameworks TensorFlow and Keras

National Day and Mid-Autumn Festival Special (2) Romantic Blessings Using Generative Adversarial Networks (GAN) to generate paintings with a festive atmosphere

National Day and Mid-Autumn Festival Special (1) Romantic Blessing Method Use Recurrent Neural Network (RNN) or Long Short-Term Memory Network (LSTM) to generate blessing poems

1. Ribbon development history and underlying principles

Ribbon is an open source project developed by Netflix and is mainly used to achieve client-side load balancing. Its development history can be traced back to 2013. With the development of the Spring Cloud ecosystem, Ribbon has gradually become a part of Spring Cloud and is widely used in microservice architecture.

The underlying principle of load balancing
The underlying principle of Ribbon’s load balancing is to select a service provider based on the configured load balancing policy when a request is initiated through RestTemplate. The following are the load balancing strategies supported by Ribbon:

RoundRobin: The default strategy selects service providers in order.
Random: randomly selects service providers.
BestConnection: Choose the shortest one based on the service provider’s response time.
Sentinel (fence): Degrade based on the service provider’s response time.

Applicable scene
Ribbon is suitable for the following scenarios:

A scenario where each node in the cluster provides equal service capabilities and is stateless.
Scenarios that require load balancing based on request characteristics or response time.
A microservice architecture that implements dynamic discovery and load balancing is required.

specification
Ribbon configuration is mainly set through application.properties or application.yml files. Here are some commonly used parameters:

listOfServers: Specify the address list of service providers, separated by commas.
loadBalancer: Load balancing policy configuration.
- roundRobin: polling strategy configuration, no special parameters.
- random: Random strategy configuration, no special parameters.
- bestConnection: Best connection strategy configuration, the following parameters can be set:
  timeout: connection timeout, in milliseconds.
  sentinel: fence policy configuration, the following parameters can be set:
  stat: Statistics configuration, including:
  enabled: Whether to enable statistics, default false.
  timeout: Statistics timeout, unit is milliseconds, default is 1000.

Development History

In 2013, Netflix developed Ribbon to solve the load balancing problem of internal microservice architecture.
In 2017, the Spring Cloud project acquired Ribbon and maintains and develops it as part of Spring Cloud.
So far, Ribbon has become an important part of the Spring Cloud ecosystem and is widely used in microservice architecture.

Summarize
Ribbon is a mature and practical load balancing component. Its underlying principle is to select a service provider based on the configured load balancing policy when a request is initiated through RestTemplate. By adjusting load balancing strategies and parameters, the needs of different scenarios can be met. As the Spring Cloud ecosystem develops, Ribbon will continue to improve and expand its functionality to provide better support for microservice architectures.

https://github.com/Netflix/ribbon

2. Load balancing strategy

Ribbon supports the following load balancing strategies:

RoundRobin (polling)
The polling strategy is Ribbon’s default load balancing strategy. It selects service providers in sequence. The following is a code analysis of the polling strategy:
In Ribbon, the polling strategy is implemented in the DefaultLoadBalancer class. DefaultLoadBalancer inherits from the AbstractLoadBalancer class. The following is the key code of the polling strategy:

public class DefaultLoadBalancer extends AbstractLoadBalancer {<!-- -->
    private final AtomicInteger index = new AtomicInteger(0);
    @Override
    protected Server choose(Server[] servers) {<!-- -->
        int currentIndex = index.getAndUpdate(i -> (i + 1) % servers.length);
        return servers[currentIndex];
    }
}

In this code, the choose method is the core method of Ribbon load balancing. It receives an array of servers as parameter. The index atomic variable is used to record the currently polled server index. Each time the choose method is called, the index is incremented by 1 and modulo the server array to get a new index. Then the corresponding server is taken from the new index position as the result of load balancing selection.
2. Random
The random policy is another simple load balancing policy. It randomly selects a service provider. The following is a code analysis of the random strategy:

public class RandomLoadBalancer extends AbstractLoadBalancer {<!-- -->
    @Override
    protected Server choose(Server[] servers) {<!-- -->
        int randomIndex = new Random().nextInt(servers.length);
        return servers[randomIndex];
    }
}

In this code, the choose method generates a random integer through the nextInt method of the Random class, and then uses this integer as an index to select from the server array A random server.
3. BestConnection (Best Connection)
The best connection strategy chooses the shortest one based on the service provider’s response time. The following is a code analysis of the optimal connection strategy:

public class BestConnectionLoadBalancer extends AbstractLoadBalancer {<!-- -->
    private final List<Server> sortedServers = new ArrayList<>();
    @Override
    protected Server choose(Server[] servers) {<!-- -->
        // Get the response time of all servers
        List<Long> responseTimes = getResponseTimes(servers);
        // Sort servers in descending order by response time
        sortedServers.sort(Comparator.comparingLong(o -> o));
        //Return the server with the shortest response time
        return sortedServers.get(0);
    }
    private List<Long> getResponseTimes(Server[] servers) {<!-- -->
        //The logic of obtaining the server response time is omitted here.
    }
}

In this code, the choose method first obtains the response times of all servers, and then sorts the servers in descending order by response time. Finally, the server with the shortest response time is returned.
4. Sentinel (fence)
The fence strategy is to load balance based on the health status and weight of the service provider. The following is a code analysis of the fence strategy:

public class SentinelLoadBalancer extends AbstractLoadBalancer {<!-- -->
    private final Map<Server, Integer> serverWeights = new HashMap<>();
    @Override
    protected Server choose(Server[] servers) {<!-- -->
        // Get the sum of the weights of all servers
        int totalWeight = getTotalWeights(servers);
        //Initialize fence statistics
        for (Server server : servers) {<!-- -->
            serverWeights.put(server, 1);
        }
        //Select a server from the server list according to the weight ratio
        int randomWeight = new Random().nextInt(totalWeight);
        int currentWeight = 0;
        for (Map.Entry<Server, Integer> entry : serverWeights.entrySet()) {<!-- -->
            currentWeight + = entry.getValue();
            if (randomWeight < currentWeight) {<!-- -->
                return entry.getKey();
            }
        }
        // If there is no suitable server, throw an exception
        throw new IllegalStateException("No available server found");
    }
    private int getTotalWeights(Server[] servers) {<!-- -->
        //The logic of obtaining the sum of server weights is omitted here.
    }
}

In this code, the choose method first obtains the sum of the weights of all servers, then traverses the server list and randomly selects a server based on the weight ratio. If no suitable server is found during the traversal (i.e. the random weight is smaller than the current weight), a reference to the current server is returned. If no suitable server is ultimately found, an exception is thrown.
The advantage of the fence strategy is that load balancing is based on the health status and weight of the service provider, which can ensure that requests are distributed more evenly while avoiding unfair problems that may be caused by traditional polling strategies. The disadvantage is that when the number of servers is small, it may cause servers with more requests to bear more load, while servers with fewer requests have lighter loads. In addition, the implementation of the fence strategy is complex and requires maintaining the weight information of the server.
In practical applications, the weight of the server can be adjusted according to the actual situation to achieve more accurate load balancing. For example, different weights can be set based on factors such as the server’s processing power and response time.

3. Spring Cloud Ribbon in action

Spring Cloud load balancing mainly implements client load balancing through Ribbon. The following will introduce in detail how to use Spring Cloud Ribbon to implement load balancing, including complete code examples.

Add dependencies
In the project’s pom.xml file, add the spring-cloud-starter-ribbon dependency:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-ribbon</artifactId>
    <version>3.2.5</version>
</dependency>

Enable Eureka service registration
Add the @EnableEurekaClient annotation on the project’s main startup class to enable Eureka service registration.

import org.springframework.cloud.client.discovery.EnableDiscoveryClient;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
@EnableDiscoveryClient
public class Application {<!-- -->
    public static void main(String[] args) {<!-- -->
        SpringApplication.run(Application.class, args);
    }
}

Add Ribbon configuration
Add Ribbon configuration in the application.properties or application.yml file, for example:

spring:
  application:
    name: movie-service
  cloud:
    ribbon:
      my-service:
        listOfServers: http://user-service-1.com, http://user-service-2.com
        loadBalancer:
          ribbon:
            sentinel:
              timeout: 1000
              stat:
                enabled: false

Here, the load balancing strategy is set to round-robin, and user-service-1.com and user-service-2.com are specified as service providers.
4. Use @LoadBalanced annotation
Add the @LoadBalanced annotation on the RestTemplate that needs to be load balanced.

import org.springframework.cloud.client.loadbalancer.LoadBalanced;
import org.springframework.web.client.RestTemplate;
public class MovieService {<!-- -->
    @LoadBalanced
    private RestTemplate restTemplate;
    // When using restTemplate to initiate a request, load balancing will be automatically performed
    public String callUserService(String userServiceUrl) {<!-- -->
        return restTemplate.getForObject(userServiceUrl, String.class);
    }
}

Test load balancing
Start the movie service (MovieService) and user service (UserService), and then call the callUserService method in the movie service to observe whether the request address is load balanced.
The above code example shows how to use Spring Cloud Ribbon to implement client-side load balancing. In actual projects, load balancing strategies can also be customized according to needs, such as response time weighting, etc. For more detailed usage of Spring Cloud Ribbon, please refer to the official documentation: http://cloud.spring.io/spring-cloud-static/Camden.SR7/#netflix-ribbon-starter

Complete code: https://github.com/Netflix/ribbon/blob/master/ribbon-examples/src/main/java/com/netflix/ribbon/examples/rx/template/RxMovieTemplateExample.java

4. Spring Cloud Ribbon common problems and solutions

There are some common problems you may encounter when using load balancing ribbons. Here are some of these problems and their solutions:

Problem: The number of service instances discovered by the service registry is inconsistent
Solution: Ensure that the number of service instances in the service registry (such as Eureka, Zookeeper, etc.) is consistent with the number of actual running service instances. Regularly check and delete offline service instances.
Problem: Load balancing policy is invalid
Solution: Check Ribbon’s configuration file to ensure that the load balancing policy is configured correctly. Such as polling, random, best connection, etc. At the same time, ensure that Ribbon integrates correctly with other components such as Spring Cloud, Netflix OSS, etc.
Problem: Service call failed
Solution: Check the service provider’s health status to ensure it is functioning properly. At the same time, check whether the interface address and port of the service provider are correct.
Problem: Ribbon configuration does not take effect
Solution: Make sure the Ribbon’s configuration files (such as application.properties or application.yml) are correct. At the same time, check whether Ribbon related dependencies have been added correctly.
Problem: High service latency
Solution: Adjust Ribbon’s load balancing strategy based on service latency. For example, set the timeout in the optimal connection policy to an appropriate value to reduce the weight of service instances with long response times.
Problem: Service downgrade processing does not take effect
Solution: Check Ribbon’s Sentinel policy configuration to ensure that the downgrade processing logic is correct. For example, set a response time threshold and downgrade service instances that exceed the threshold.
Problem: Load imbalance among multiple instances of the same service
Solution: In the Ribbon configuration file, set different weights for the same service instances to achieve load balancing.
Problem: Communication error between client and server
Solution: Check the network connection between the client and the server to ensure normal communication. For example, check the configuration of network devices such as firewalls and routers to ensure that ports, IP addresses, etc. are set correctly.
Problem: Statistics are missing
Solution: Make sure Ribbon’s statistics are configured correctly, such as turning on statistics, setting statistics timeout, etc. Identify and handle potential problems promptly by viewing Ribbon statistics.
Problem: Resource consumption is too high
Solution: Adjust the Ribbon configuration based on actual business needs and system resource conditions. For example, reduce the number of connections, set a reasonable timeout, etc. to reduce resource consumption.
In short, when using Ribbon, you should pay close attention to its running status and discover and solve potential problems in time. At the same time, adjust the Ribbon configuration according to actual needs to ensure that the load balancing effect meets expectations.

Directory

1. Ribbon development history and underlying principles

2. Load balancing strategy

3. Spring Cloud Ribbon in action

4. Spring Cloud Ribbon common problems and solutions