[Microservice Protection] Sentinel Flow Control Rules – In-depth exploration of Sentinel’s flow control mode, flow control effects, and flow limiting of hotspot parameters

Article directory

Preface
1. Quickly master the use of Sentinel
- 1.1 What is a cluster point link?
- 1.2 Simple usage example of Sentinel
2. Sentinel flow control mode
- 2.1 Direct mode
- 2.2 Association mode
- 2.3 Link mode
3. Flow control effect
- 3.1 Fail fast
- 3.2 Preheating mode
- 3.3 Waiting in line
4. Flow control of hotspot parameters
- 4.1 Hotspot rules
- 4.2 Hotspot rule demonstration

Foreword

The popularity of microservice architecture makes ensuring stability in distributed systems particularly critical. In the previous article, we have discussed the avalanche problems that may occur in microservices and their corresponding solutions. As one of the key tools to ensure system availability, Sentinel emerged as a powerful traffic control component that provides developers with multiple ways to manage and protect microservices.

In this article, I will delve into Sentinel’s core features, including flow control mode, flow control effects, and flow limiting strategies for hotspot resources. Learn more about these concepts and use examples to demonstrate how to use Sentinel in Spring Cloud projects to better deal with various traffic control and throttling needs.

1. Quickly master the use of Sentinel

1.1 What is a cluster point link

Before learning how to use Sentinel, we need to first understand cluster point links. The so-called cluster point link is the calling link within the project, and each interface monitored in the link is a resource. By default, Sentinel monitors each endpoint of Spring MVC, so each endpoint of SpringMVC is a resource in the call link.

Flow control, circuit breaker, etc. are all set for the resources in the cluster point link, so we can click the button behind the corresponding resource to set the rules:

1.2 Simple usage example of Sentinel

Now, we need to limit the flow of the path /order/{orderId}, requiring that its QPS cannot exceed 5 per second. The setting steps are as follows:

In the cluster point link, click to add a flow control rule under the specified link:

Set flow control rules, QPS does not exceed 5:

After adding it, you can view this new rule in the flow control rules:

Test using Jmeter
Set thread properties:

20 threads are set up, the thread startup time is 2s, and the QPS at this time is 10.

Configure HTTP requests:

In this HTTP request, the resource accessed is /order/101.

Start JMeter for testing:

It can be found from the result tree that the number of consecutive successful requests will not exceed 5.

2. Sentinel flow control mode

Sentinel is a powerful flow control component that supports multiple flow control modes to help developers better manage and protect microservices. The three main flow control modes of Sentinel will be introduced below: direct mode, association mode and link mode.

When using Sentinel to add flow limiting rules, click Advanced Options to choose from three flow control modes:

Direct: counts requests for current resources, and directly limits current resources when the threshold is triggered, which is also the default mode;
Association: Statistics on another resource related to the current resource. When the threshold is triggered, the current resource is limited;
Link: Counts the requests to access this resource from the specified link. When the threshold is triggered, the specified link will be flow-limited;

2.1 Direct mode

Direct mode is Sentinel’s most basic flow control mode, which controls traffic by limiting the access frequency of resources. In direct mode, you can configure the allowed QPS (queries per second) limit for each resource. If the actual traffic to a resource exceeds the configured limit, Sentinel will deny or downgrade access requests to that resource.

This mode is suitable for scenarios that require flow control of a specific resource, such as API interfaces, microservices, etc. By configuring direct mode, you can protect critical resources from excessive requests and ensure system stability and availability.

The simple usage example of Sentinel mentioned above is the direct mode, which limits the current request for the specific request resource order/{orderId}.

2.2 Association mode

Correlation mode is a more flexible flow control mode provided by Sentinel. In association mode, you can define associations between multiple resources and then control traffic based on these associations. This enables better adaptation to complex interactions between multiple resources.

Simply put, the function of association mode is to count another resource related to the current resource. When the resource access threshold is triggered, the current resource is limited.

Usage scenarios of association mode:

For example, the user needs to modify the order status when paying, and at the same time the user needs to query the order. Query and modification operations will compete for database locks and cause competition. The business requirement is limited payment and order update business, so when the order business trigger threshold is modified, the query order business flow needs to be limited.

For example, two new interfaces are added below in order-service. query means querying orders, and update means updating orders:

@GetMapping("/query")
public String queryOrder(){<!-- -->
    return "Order query successful!";
}

@GetMapping("/update")
 public String updateOrder(){<!-- -->
     return "Update order successfully!";
 }

Next, set the current limit for the query interface through association mode:

At this time, the two interfaces query and update are associated. When the QPS accessed by the update resource exceeds 5, query interface for current limiting.

Testing with JMeter:

Set the number of threads, QPS is 10 at this time:

Set up HTTP request:

Start JMeter for testing:

At this time, the request for the update interface can be processed normally, but at this time we access the query interface through the browser and find that the interface is restricted by Sentinel:

This is the association model, and we can draw the conclusion from the above example:

Association mode can be used if the following conditions are met:

Two competing resources;
One has higher priority and one has lower priority.

2.3 Link Mode

Link mode: Only make statistics on requests to access this resource from the specified link to determine whether the threshold is exceeded.

For example, there are two request links test1 and test2, both of which can serve the commom resource:

/test1 -> /common
/test2 -> /common

If you only want to count requests from /test2 to /common, you can use link mode for configuration:

Now there is a demand: there is query order and create order business, both of which need to query products. Collect statistics on requests from order inquiry to product inquiry, and set current limits. The steps to set up are as follows:

Add a queryGoods method in OrderService without implementing the business:
```
@SentinelResource("goods") // Sentinel marks methods other than Controller
public void queryGoods(){
    System.err.println("Query product!");
}
```
Note that by default, Sentinel will only monitor the endpoint methods in Controller. If you want to monitor other methods, you can use the @SentinelResource annotation.

In addition, Sentinel will integrate the Controller method with context by default, causing the link mode flow control to fail. You need to modify application.yml and add the following Configuration:
```
spring:
  cloud:
    sentinel:
      web-context-unify: false # Turn off Context integration
```

In OrderController, transform the /order/query endpoint and call the queryGoods method in OrderService:

@GetMapping("/query")
public String queryOrder(){<!-- -->
    // Query products
    orderService.queryGoods();
    // checking order
    System.out.println("query order");
    return "Order query successful!";
}

Add an endpoint of /order/save in OrderController and call the queryGoods method of OrderService:

@GetMapping("/save")
public String saveOrder(){<!-- -->
    // Query products
    orderService.queryGoods();
    //Add new order
    System.out.println("Add new order");
    return "Add order successfully!";
}

Set the current limiting rule for queryGoods. The method of entering queryGoods from /order/query limits the QPS to no more than 2:
Testing using JMeter
Set thread, QPS to 4:

Set HTTP requests:

HTTP requests to the query and save interfaces are set up respectively.
Start JMeter:
It can be found that the query interface is restricted:

There is no current limit for the save interface:

3. Flow control effect

In Sentinel, flow control effects refer to the measures that should be taken when a request reaches the flow control threshold. Sentinel provides a variety of flow control effects, including fast fail, warm-up mode, and queue waiting. Each effect is suitable for different usage scenarios, and the appropriate flow control effect can be selected according to specific needs.

Fail fast: Once the threshold is reached, new requests are immediately rejected with a FlowException exception. Through the previous demonstration, we can find that this is the default processing method.
Preheating mode: Requests that exceed the threshold will also be rejected and an exception will be thrown. But this mode threshold changes dynamically, gradually increasing from a smaller value to the maximum threshold.
Queue waiting: Let all requests be queued for execution in order. The interval between two requests cannot be less than the specified time.

These modes can be set in the advanced options of the Sentinel console. Below are demonstrations of these flow control effects.

3.1 Fail fast

Fast Fail is Sentinel’s default flow control effect. When a request reaches the flow control threshold, new requests will be rejected immediately and a FlowException exception will be thrown. This means that the request will fail immediately and no subsequent logic will continue. This effect is suitable for scenarios with clear limits on system resources, helping to identify and reject excessive requests as early as possible to avoid overloading the system.

Using the fail-fast effect can protect the system from excessive requests and ensure system stability and availability.

3.2 Preheating mode

Preheat mode is also called “warm up” mode. In preheating mode, requests that exceed the flow control threshold will also be rejected and an exception will be thrown, but the difference is that the flow control threshold will gradually increase to the maximum threshold over a period of time.

Preheating mode is a solution for service cold start. The initial value of the request threshold is threshold / coldFactor. After a specified period of time, it will gradually increase to the threshold value. The default value of coldFactor is 3.

For example, set the threshold of QPS to 10 and the warm-up time to 5 seconds. Then the initial threshold is 10 / 3, which is 3, and then gradually changes from 3 after 5 seconds. Grows to 10.

Now there is a requirement: to set a current limit for the resource /order/{orderId}, with a maximum QPS of 10, using the warm up effect, and a warm-up time of 5 seconds. Proceed as follows:

1. Add flow control rules:

2. Test using JMeter:

Set QPS to 10:

Set up HTTP request:

Start JMeter and view the real-time monitoring of the Sentinel console:

It can be found that the value of QPS has increased recently and the number of rejected requests has gradually decreased.

3.3 Waiting in line

Queue waiting The flow control effect allows all requests to be queued for execution in order, ensuring that the interval between requests is not less than the specified time. This means that the request will be queued to be processed until it is the request’s turn to be executed. If requests cannot be executed immediately, they are queued.

The queuing effect is suitable for scenarios that require orderly execution of requests, such as services that need to be processed in the order in which requests arrive, or scenarios that require a fixed time interval between requests.

For example: QPS = 5, which means that a request in the queue is processed every 200ms; timeout = 2000, which means that requests that are expected to wait for more than 2000ms will be rejected and an exception will be thrown.

Now there is a requirement: set a current limit for the resource /order/{orderId}. The maximum QPS is 10. Using the flow control effect of queuing, set the timeout to 5s. The steps are as follows:

1. Add current limiting rules:

2. Test using JMeter:

Set QPS to 15:

Set up HTTP request:

Start JMeter to view Sentinel’s real-time monitoring:

It can be found that in the queuing flow control mode, requests will be queued and executed in an orderly manner, which helps to cut peaks and flatten valleys. This mode is very useful for high-concurrency scenarios and businesses that require requests to be processed in the order of arrival. By queuing requests for execution at time intervals, the system can better cope with sudden increases in traffic, reduce system pressure, and thereby improve system stability and availability.

4. Flow control of hotspot parameters

4.1 Hotspot rules

The previous flow limiting is to count all requests to access a certain resource to determine whether it exceeds the QPS threshold. The hotspot parameter current limiting is to separately count requests with the same parameter value to determine whether the QPS threshold is exceeded.

For example, as shown in the figure below, the same link is accessed, but the resource with id 1 in this link is accessed more times than other resources. This means that the resource is a hot resource, and the QPS of the resource can be set larger:

In the Sentinel console, you can find a hotspot rule through which you can set flow control settings for hotspot resources, for example:

What it means is: To make statistics on parameter 0 (the first parameter) of the hot resource, the number of requests with the same parameter value per second cannot exceed 5.

In addition, in the advanced options of hotspot parameter current limiting, you can set exception configurations for some parameters:

Combined with the previous configuration, the meaning here is to limit the current flow of the long type parameter No. 0. The QPS of the same parameter cannot exceed 5 every 1 second. There are two exceptions:

If the parameter value is 100, the allowed QPS per 1 second is 10;
If the parameter value is 101, the allowed QPS per 1 second is 15.

4.2 Hotspot rules demonstration

Now there is a need to add hotspot parameter current limiting to the resource /order/{orderId}. The rules are as follows:

The default hotspot parameter rule is that the number of requests per 1 second does not exceed 2;
Set an exception for the 102 parameter: the number of requests per 1 second does not exceed 4;
Set an exception for this parameter 103: the number of requests per 1 second does not exceed 10;

1. Add hotspot rules:

Hotspot parameter current limiting is not valid for default SpringMVC resources, so you need to use the @SentinelResource() annotation to specify the hotspot resource name:

2. Test using JMeter:

Set QPS to 5:

Set HTTP requests:

Here we set HTTP requests for three different resources with IDs 101, 102, and 103, corresponding to the different situations of the rules set above.