AuthorAuthor: Li Lai, Senior Software Engineer, Huawei Cloud
1. Full-link traffic label transparent transmission
In the microservice architecture, traffic labels are used to mark and classify traffic, enabling more granular routing, load balancing, flow control and other traffic management capabilities between microservices. Taking HTTP messages as an example, each header can be a traffic label. For example, x-sermant-version: v1 represents a label added through Sermant traffic dyeing, which can be used to identify the version information corresponding to the request traffic.
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: zh-CN,zh;q=0.9,en;q=0.8
Connection: keep-alive
Content-Length: 1000
Content-Type: application/json
Host: api.github.com
Origin: https://github.com
Referer: https://github.com
x-sermant-version: v1 //Custom header, indicating tags added through Sermant traffic dyeing
The core reason why traffic labels can play a role in microservice governance scenarios is that traffic labels can be passed between microservices on the call chain, that is, full-link traffic label transparent transmission. Full-link traffic label transparent transmission refers to transparently transmitting the requested label information (such as x-sermant-version: v1 in the above message) from the starting point of the request to the end point of the request in a distributed system, so as to facilitate Traffic management is performed throughout the request link. Traffic label transparent transmission can play a key role in various service governance scenarios. For example, in a full-link grayscale release scenario, the microservice caller can route the request to grayscale’s new version of the microservice instance based on the traffic label information carried in the request. In traffic control scenarios, traffic label transparent transmission can also be used to implement precise traffic control strategies, such as current limiting, circuit breaking, and downgrading, to prevent the system from crashing under high load.
2. How to realize full-link label transparent transmission
There are currently two mainstream ways to implement full-link label transparent transmission in Java. One is by integrating the label transmission capability in the SDK and working with the API gateway, and the other is through the JavaAgent mounted on the application. to fulfill.
The figure below shows the transparent transmission of traffic tags implemented by Spring Cloud Tencent in the SDK. The figure shows a canary release scenario dyed through the API gateway. Canary release (canary) refers to introducing a part of the actual traffic in the production environment to test a new version to test the performance and performance of the new version. Under the premise of ensuring the overall stable operation of the system, the new version can be discovered as early as possible in the actual environment. The problem. The label transparent transmission method implemented by this SDK generally uses a gateway to add coloring labels, and then transparently transmits the labels in applications using the Spring Cloud Tencent microservice framework. For example, the user center, points center, and activity center in the figure will transparently transmit the gateway Colored canary=true tags.
Figure – Application of Spring Cloud Tencent traffic label transparent transmission in canary release scenario
Note: The image source supports the scene-based capability of grayscale publishing · Issue #631 · polarismesh/polaris · GitHub
The transparent transmission of traffic labels is achieved through JavaAgent. A typical example is Skywalking’s transparent transmission of link trace information. Embed Skywalking’s JavaAgent in the application. By inserting a buried point in the code, it will generate a traceId marking the link at the entry node of the call chain, and then pass the call chain information to each node in the link. Collect application call link information. The focus of Skywalking is link tracking, which is the label transparent transmission of call chain information. The source of the transparent label is to generate a unique traceId at the entrance. As for the general traffic dyeing capability, it is not a Skywaling scenario.
Different traffic label transparent transmissions are implemented in different ways and have their own applicable scenarios. spring-cloud-tencent needs to be used by introducing the SDK. The Skywalking scenario mainly involves link tracking, and does not well support custom label dyeing and transparent transmission. All of the above have their own limitations, and the scope of applicable scenarios is small.
In the field of microservice governance, a full-link label transparent transmission solution with rich custom scenarios and wide application is the cornerstone of the diversified development of traffic management. As a service mesh, Sermant provides a full-link label transparent transmission solution that supports customized traffic dyeing and label transparent transmission.
3. Sermant’s exploration of full-link traffic label transparent transmission
Sermant is an agentless service grid based on Java bytecode enhancement technology. It is not only an out-of-the-box service governance tool, but also an easy-to-use service governance capability development framework. Users only need to add a line -javaagent:/xxx/sermant-agent.jar when starting the Java application to access Sermant’s service management function in a non-invasive manner with one click.
As mentioned above, full-link label transparent transmission includes two parts: coloring and transparent transmission. For these two parts, Sermant provides two plug-ins to independently realize the capabilities of traffic dyeing and label transparent transmission, namely the label routing plug-in (currently the dyeing capability is placed in the routing plug-in, and will eventually be split into independent traffic dyeing plug-ins, combined with Label transparent transmission, providing basic capabilities for a variety of traffic management scenarios) and traffic label transparent transmission plug-ins.
The figure below shows the overall usage of Sermant, also using the canary release scenario mentioned in Chapter 2 to illustrate. Each service instance is started by mounting Sermant through the -javaagent command. Through the dynamic configuration center, the traffic dyeing rules and label transparent transmission rules can be dynamically modified during the running of the service, and pushed to the Sermant of each service instance. When external request traffic is received at the entrance, the Sermant of the entrance service instance will match whether the current traffic conforms to the rules. If it does, it will be dyed, such as the stable version of service A in the figure. The Sermant of each service instance will match the traffic label transparent transmission rules to identify which labels need to be transparently transmitted downstream. In the figure, the V1 version label needs to be transparently transmitted to demonstrate. The entire process realizes the canary publishing capability of microservices.
Figure – Example of Sermant full-link traffic label transparent transmission in canary release scenario
The traffic dyeing capability and traffic label transparent transmission capability in Sermant serve as the foundation, which can provide base capabilities for more complex traffic management scenarios of other plug-ins, making the development of traffic management capabilities easier. Since traffic dyeing and label transparent transmission are independent plug-ins of Sermant, users who perform traffic dyeing based on API gateway can choose not to enable Sermant’s traffic dyeing capability. The dependencies of Sermant’s high-order service governance capabilities based on full-link traffic label transparent transmission are as follows:
Figure – Sermant service governance capability dependencies
3.1 Traffic coloring
The essence of traffic coloring is to use different labels to identify different traffic types. In the current version, Sermant first adapted the traffic dyeing capability to two mainstream microservice frameworks, Dubbo and SpringCloud. Traffic coloring currently resides in Sermant’s label routing plugin and is available as a standalone capability. The main processing flow of traffic dyeing is as follows:
Figure – Sermant’s traffic coloring process
First, in the dynamic configuration center, we can issue service-granular dyeing rules to determine whether the inbound traffic meets specific conditions. If so, the traffic will be dyed, and the dyeing label will be carried in the outbound traffic of the same call chain. . In Sermant, the data model of coloring rules is as follows:
- kind: route.sermant.io/lane # The routing rule is the type of coloring rule
description: lane # Rule description
rules:
- precedence: 1
match:
method: get #http request method, no configuration means matching
path: "/foo" #http request path, if not configured, it means matching
protocol: http # The traffic entry is the http protocol, and the http protocol will only match headers/parameters
headers: #http headers matching, no configuration means matching
ID:
exact: '1'
parameters: #http url parameters match, if not configured, it means matching
name:
exact: 'foo'
route:
- tag-inject:
x-sermant-flag1: gray1
weight: 60
The meaning of the above rules is that we try to match the rule precedence: 1, that is, the traffic entry is a request with http protocol. If the request interface url is /foo, the request method is get, the id in the headers is equal to 1, and the name in the url parameters is equal to foo. , mark 60% of the traffic that meets the conditions with [x-sermant-flag1: gray1] and pass it when calling the downstream. In dyeing rules, we also support issuing multiple matching rules and configuring priorities. For the use of traffic dyeing capability, please refer to the official website usage documentation.
In actual applications, you can customize the configuration rules as needed. For example, we need to color the traffic of the /getPrice interface of http requests. If we want to add all traffic carrying city=Shanghai or city=Hangzhou to subsequent links. [x-sermant-region: east] coloring mark, then the rules can be configured as follows.
- kind: route.sermant.io/lane
description: lane
rules:
- precedence: 1
match:
method: get
path: "/getPrice"
protocol: http
headers:
city:
in: ['Shanghai', 'Hangzhou']
route:
- tag-inject:
x-sermant-region: east
weight: 100
In short, the dyeing rules can be customized and configured, and they support dynamic modification at runtime and can take effect immediately. If dyeing is not required, just delete the dynamic configuration.
3.2 Transparent transmission of traffic tags
After dyeing the traffic through the steps in 3.1, you can use Sermant’s traffic label transparent transmission capability to transparently transmit dyed labels across the entire link. Sermant divides traffic label transparent transmission into two categories: cross-process transparent transmission and intra-process transparent transmission. Cross-process transparent transmission refers to passing traffic markers in different service instance processes, such as the client and server of http requests. In-process transparent transmission refers to the transmission of traffic labels within the process of a service instance, including intra-thread transmission and cross-thread transmission.
As shown in the figure below, according to the type of label transfer that needs to be adapted, we can divide it into two types: interface call (including http protocol and rpc protocol) and message queue (including kafka, RocketMQ, etc.).
Figure – Label transparent transmission process of http/rpc request in Sermant
Figure – Label transparent transmission process of message queue in Sermant
3.2.1 Cross-process transparent transmission
Cross-process traffic label transparent transmission includes http protocol, rpc protocol and message queue. We use the header of the http request, the attachment of the Dubbo request, the metadata of the grpc request, the header of the kafka message, etc. to transparently transmit the traffic label throughout the entire link.
In the Sermant traffic label transparent transmission plug-in, for the client, we use bytecode enhancement to construct aspects at http, rpc sending requests and message queue production messages, and obtain the traffic label in the current process at the aspects, and when sending the request Or inject traffic labels into header, attachment, etc. when producing messages. For the server, at the aspects of http, rpc receiving requests and message queue consuming messages, the traffic labels in the header, attachment, etc. are taken out and continue to be transparently transmitted in the server process. The cross-process label transfer process of http or rpc is as shown in the figure below:
Figure – The process of Sermant transparently transmitting labels across processes (http/rpc)
The message queue’s tag transparent transmission is similar, except that the traffic tag will be sent to the middleware server during production, and the message will be pulled from the server during consumption and the traffic tag will be parsed.
3.2.2 In-process transparent transmission
There are two situations in which traffic labels are transparently transmitted within a process: intra-thread and cross-thread. In the intra-thread scenario, Sermant uses ThreadLocal thread variables to transfer labels according to the conventional practice, which will not be described here.
Cross-thread scenarios are mainly divided into three categories: direct new Thread(), ordinary thread pool and scheduled thread pool. For new Thread(), use JDK’s native InheritableThreadLocal to pass tags in parent and child threads.
For ordinary thread pools, Sermant intercepts its execute and submit methods. When executing a new thread task, it transparently transmits the traffic tag of the parent thread to the Runnable or Callable object, and can continue to pass the traffic tag in the child thread. . The timing thread pool is similar. It passes the parent thread traffic label to the child thread by intercepting the schedule, scheduleAtFixedRate, and scheduleWithFixedDelay methods.
Figure – Sermant’s process of transparently transmitting labels across threads
The above three methods can basically cover all cross-thread scenarios. Based on their enhancement, Sermant can realize cross-thread traffic label transparent transmission.
3.2.3 Dynamic configuration of traffic label transparent transmission
Traffic label transparent transmission also supports dynamic configuration of whether to enable it and which labels to transparently transmit, as shown below:
tag.transmission.config:
# Whether to enable transparent transmission of traffic tags
enabled: true
# Matching rules for the key of traffic labels that need to be transparently transmitted, supporting equal, prefix, and suffix matching
matchRule:
# Exact match
exact: ["id", "name"]
# Prefix matching
prefix: ["x-sermant"]
# Suffix matching
suffix: [""]
Currently, exact matching, prefix matching, and suffix matching are supported for keys that require transparent transmission of labels. After configuring the coloring rules introduced in the previous chapter, configure the coloring label in the configuration of traffic label transparent transmission. This will achieve a complete solution for full-link label transparent transmission from ingress coloring to entire link transmission.
4. Application scenarios of Sermant full-link traffic label transparent transmission
Full-link traffic label transparent transmission is the core foundation of microservice traffic management. It accurately transmits the key key of traffic label to each service instance, allowing traffic to flow according to the planned trajectory. Full-link traffic label transparent transmission can be applied in many service management scenarios, such as full-link grayscale publishing, flow control, etc.
4.1 Full-link grayscale release scenario based on full-link traffic label transparent transmission
In a full-link grayscale release scenario, it is usually necessary to gradually release the new version of the code to the production environment for testing and verification in the production environment. As shown in the figure below, the gray1 version only deploys some services, and the gray2 version deploys all services. Utilizing the ability to transparently transmit full-link traffic labels, traffic can flow along gray links based on the gray version information of service instances. The traffic of the stable version only calls the service instance of the stable version, and the traffic of the grayscale version first calls the current grayscale version instance. If there is no grayscale instance for a service in the middle, the instance of the stable version is called. Regardless of whether the service called at any time is a stable version or a grayscale version, the traffic label continues to be transparently transmitted, and grayscale traffic always preferentially chooses the grayscale version for routing.
Figure – Using Sermant to realize the process of full-link grayscale publishing
In the current version of Sermant, the label routing plug-in includes the capabilities of traffic dyeing and label routing (the functions are independent and will eventually be split into two independent plug-ins). The traffic label transparent transmission plug-in supports label transparent transmission capabilities. The implementation of the full-link grayscale publishing scenario in the picture above only requires mounting these two plug-ins at the same time. The main processes used are:
(1) The label routing plug-in configures the dyeing rules of traffic. For example, grayscale traffic adds x-sermant-version: gray1, x-sermant-version: gray2 tags. (This step is optional and can be dyed in other ways such as API gateway)
(2) The label transparent transmission plug-in configures the label of the x-sermant-version key to be transparently transmitted in the entire link.
(3) The label routing plug-in configures the full-link routing rules for traffic so that gray1 traffic preferentially calls the gray1 version instance and gray2 traffic preferentially calls the gray2 version instance.
4.2 Flow control scenario based on full-link traffic label transparent transmission
In traffic control scenarios, the transparent transmission feature of labels can also be used to perform refined traffic control through traffic labels. Usually we can group traffic by its URL, header, token, request parameters, etc. For example, in scenarios such as Double 11, the capabilities provided by a single API of the service are limited, but the actual peak call demand may be twice the limit QPS. In this scenario, URLs can be used to group traffic and classify different types of traffic. It achieves precise flow control of traffic and meets the needs of most people while ensuring that core business calls are not restricted.
For example, as shown in the figure below, the ingress application dyes and groups different traffic. You can add the VIP label to the core important interfaces that need to be protected, and add the normal label to the non-important interfaces. Service A can implement differentiated traffic limiting strategies based on different group traffic, limiting 20% of the limit QPS to normal and 80% of QPS to VIP traffic. In service B, according to the importance of different grouped traffic, it can be ensured that when the service encounters a non-fatal error, VIP traffic can avoid the final call failure of the traffic by retrying, and can tolerate normal traffic with a higher error rate. Then do not configure the traffic retry policy.
Figure – The process of using Sermant to implement tag-based flow control
The current version of Sermant also includes a flow control plug-in, which supports current limiting, circuit breaker, isolation bin, error injection and retry, circuit breaker indicator collection, system rules, system adaptive flow control capabilities, and supports dynamic configuration rules in the configuration center. Effective in real time. The scenario in the picture above can be implemented by mounting the label routing plug-in (providing dyeing capabilities), the traffic label transparent transmission plug-in, and the flow control plug-in. How to use it:
(1) The label routing plug-in configures traffic coloring rules, such as adding x-sermant-group: vip, x-sermant-group: normal labels to the traffic of different accounts. (This step is optional and can be dyed in other ways such as API gateway)
(2) The label transparent transmission plug-in configures transparent transmission rules. For example, the label of the x-sermant-group key is transparently transmitted in the entire link.
(3) The flow control plug-in configures the flow control rules. For example, the traffic carrying x-sermant-group: vip in the header has corresponding retry policies in service B, and the traffic of x-sermant-group: normal has limits configured in service A. flow strategy.
5. Summary
Sermant integrates the capabilities of traffic dyeing and traffic label transparent transmission in a plug-in form, forming a complete full-link traffic label transparent transmission solution. Traffic labels essentially group traffic. Based on this solution, we can make traffic labels spread throughout the entire traffic topology, including full-link grayscale publishing, current limiting and downgrading, fault diagnosis and recovery, load balancing, traffic monitoring, etc. play an important role in the scene. In addition, Sermant’s plug-in architecture can also support users to use traffic dyeing capabilities or traffic label transparent transmission capabilities independently, and freely combine various plug-in modules in a building-block manner. Sermant’s non-intrusive features also allow users to access the full-link traffic label transparent transmission capability with one click without any code modification.
As a bytecode enhancement framework focusing on the field of service governance, Sermant is committed to providing a high-performance, scalable, easy-to-access, and feature-rich service governance experience, and will take care of performance, functionality, and experience in each version. , everyone is widely welcome to join.
- Sermant Official website: https://sermant.io
- GitHub Warehouse address: https://github.com/huaweicloud/Sermant
- Scan the QR code to join the Sermant community communication group
