In the first column, we discussed why the program crashes under high concurrency conditions. The main reason is that in the case of high concurrency, there are a large number of user requests that require program calculation and processing. The current processing method is to allocate a thread to each user request. When the thread is blocked due to reasons such as accessing the database within the program, the thread It cannot be released to handle other requests, which will cause a pile-up of requests, continue to consume resources, and eventually cause the program to crash.
This is the threading nature of traditional web application runtime. For a highly concurrent application system, there are always many user requests arriving at the system’s Web container at the same time. The web container assigns a thread to each request for processing. During the processing, if the thread encounters operations such as accessing the database or remote service, it will enter a blocking state. At this time, if the database or remote service response is delayed, a program will appear. The internal thread cannot be released, and external requests continue to come in, causing computer resources to be quickly consumed, and eventually the program crashes.
So is there any programming method that does not block threads?
Reactive programming
The answer is reactive programming. Reactive programming is essentially an asynchronous programming solution. Based on technologies such as multi-threading (coroutines), asynchronous method calls, and asynchronous I/O access, it provides a complete set of programming models that match asynchronous calls to achieve The program calls features such as non-blocking and instant response, which means a reactive system is developed to cope with the increasing concurrent processing requirements in the programming field.
People also proposed a reactive manifesto, believing that reactive systems should have the following characteristics:
Instant response, the caller of the application can get the response immediately without waiting for the entire application to complete. In other words, application calls are non-blocking.
Resilience When some functions of the application fail, the application system itself can repair itself to ensure normal operation and response, and there will be no system crash or downtime.
Elastic, the system can respond to application load pressure, can automatically scale to adapt to application load pressure, automatically adjust its own processing capabilities according to the pressure, or adjust access to the system according to its own processing capabilities. Request quantity.
Message-driven, functional modules and services are driven by messages to complete the service process.
The current mainstream reactive programming frameworks include RxJava, Reactor, etc. Their main feature is an asynchronous programming solution based on the Observer Design Pattern, and the programming model adopts functional programming.
Observer pattern and functional programming have their own advantages, but reactive programming does not require the use of observer pattern and functional programming. Flower is a purely message-driven, completely asynchronous, reactive programming framework that supports imperative programming.
Let’s take a look at how Flower implements asynchronous non-blocking calls, and what design principles and patterns are used in the design of the Flower framework.
Basic principles of the reactive programming framework Flower
The threading characteristics of a typical web application developed using the Flower framework are shown in the figure below:
When concurrent users arrive at the application server, the Web container thread does not need to execute the application code. It just changes the user’s HTTP request into a request object, asynchronously hands the request object to the Service of the Flower framework for processing, and returns immediately. Because container threads do not do too much work, very few container threads are needed to satisfy highly concurrent user requests. User requests will not be blocked or cannot be processed due to insufficient container threads. Compared with traditional blocking programming, the Web container thread must complete all request processing operations, and the thread cannot be released until the response result is returned; Using the Flower framework only requires very few container threads to handle more concurrent user requests. And the container thread will not block.
After the user’s request is handed over to the business Service object developed based on the Flower framework, the Services will still be called using asynchronous message communication, and no blocking calls will be made directly. After a Service completes business logic processing and calculation, it will return a processing result, which is asynchronously sent to its next Service in the form of a message.
If calls are made between Services in the traditional programming model, as we discussed in the first article of this column, the called Service method can only block and wait before the called Service returns. Flower’s services use AKKA Actors for message communication. After the caller’s Service sends the call message, it can process its next message without waiting for the callee to return the result. In fact, these services can reuse the same thread to process their own messages. In other words, only a limited number of threads are needed to complete a large amount of service processing and message transmission, and these threads will not block and wait.
We just mentioned that usually the main thread blocking in web applications is thread blocking caused by database access. Flower supports an asynchronous database driver. When a user requests a database, the request is submitted to the asynchronous database driver and returned immediately without blocking the current thread. The asynchronous database accesses the remote database and performs real database operations. After the result is obtained, the result is sent to the asynchronous database driver. Send it to Flower’s Service in an asynchronous callback for further processing. At this time, no thread will be blocked.
In other words, using the system developed by Flower, in a typical Web application, almost no place will be blocked, all threads can be continuously reused, and limited threads can complete a large number of concurrent users requests, thereby greatly improving the system’s throughput capacity and response time. At the same time, because the thread will not be blocked, the application will not crash due to too much concurrency or slow database processing, thus improving Improved system availability.
The Flower framework achieves asynchronous non-blocking. On the one hand, it takes advantage of the asynchronous characteristics of the Web container, mainly the AsyncContext provided after Servlet3.0, to quickly release the container thread; on the other hand, it takes advantage of the asynchronous database driver and asynchronous network communication, mainly It is an asynchronous communication component such as HttpAsyncClient. Within the Flower framework, asynchronous non-blocking calls between core application codes are implemented using Akka’s Actor model.
The asynchronous message-driven implementation of Akka Actor is as follows:
When an Actor communicates with another Actor, the current Actor is the sender of a message. When it wants to communicate with another Actor, it needs to obtain the ActorRef of the other Actor, which is a reference. Through the reference Communicate messages. After ActorRef receives the message, it will put the message into the target Actor’s Mailbox, and then return immediately.
That is to say, when an Actor sends a message to another Actor, the other Actor does not need to actually process the message. It only needs to send the message to the Mailbox of the target Actor. It will not be blocked and can continue to perform its own operations, while the target Actor checks whether there is a message in its Mailbox. If there is a message, the Actor will obtain the message from the Mailbox and process the message asynchronously, and all Actors will share threads, and these threads will not block in any way.
Design method of reactive programming framework Flower
However, there are many inconveniences in programming directly using Actors. The Flower framework encapsulates Actors. Developers only need to write some fine-grained Services. These Services will be packaged in Actors for asynchronous communication.
Flower Service example is as follows:
public class ServiceA implements Service<Message2> {
@Override
public Object process(Message2 message) {
return message.getAge() + 1;
}
}
Each Service needs to implement the process method of the Service interface of the framework. The input parameter of the process method is the return value of the previous Service process method. In this way, you only need to arrange the Service into a process, and the return value of the Service will become one of the Actor. The message is sent to the next Service, thereby realizing asynchronous communication of the Service.
There are two ways to orchestrate Service processes. One way is programming, as follows:
getServiceFlow().buildFlow("ServiceA", "ServiceB");
Indicates that the return value of ServiceA will be sent to ServiceB as a message and become the input value of ServiceB, so that the two Services can cooperate to complete some more complex business logic.
Flower also supports visual service process orchestration. By editing the process definition file as shown in the picture below, you can develop an asynchronous business processing process.
So how is this Flower framework implemented?
The design of the Flower framework is also based on the dependency inversion principle discussed in the previous column. All Service classes implemented by application developers need to be packaged in Actors for asynchronous calls, but Actors will not rely on the Service classes implemented by developers, and developers will not rely on Actor classes. They all rely on a Service interface, which is the framework. provided, as shown in the example above.
The dependency inversion relationship between Actor and Service is shown in the figure below:
Each Actor relies on a Service interface, and a specific Service implementation class, such as MyService, implements this Service interface. When the Actor is instantiated at runtime, this interface is injected into the specific Service implementation class, such as MyService. In Flower, calling the MyService object actually sends a message to the Actor that wraps the MyService object. The Actor receives the message and executes its own onReceive method. In this method, the Actor calls the process method of MyService and sends the Message object received by onReceive. Passed in as an input parameter to the process.
After process processing is completed, an Object object is returned. The Actor will obtain the Actor corresponding to the next Service of MyService in the process according to the arranged process, that is, nextServiceActor, and send the Object object returned by the process as a message to this nextServiceActor. In this way, services are called and executed asynchronously and non-blockingly according to the arranged process.
The implementation effect of the reactive programming framework Flower
The Flower framework has been applied in some projects, and the application effect is relatively significant. On the one hand, Flower can significantly improve the performance of the system. This is a TPS performance comparison of a system developed in C# after reconstruction using Flower. The TPS of the system developed using Flower is almost twice that of the original C# system.
On the other hand, Flower has also greatly improved system availability. The current common Internet application architecture is as follows:
User requests are processed by calling microservices through the gateway server. When a database query connected to a certain microservice is executed slowly, such as service 1 in the figure, then according to the traditional thread blocking model, all threads of service 1 will be blocked. Blocked on this slow query database operation. Similarly, the gateway thread will also be blocked in calling service 1 with a relatively high delay.
The final effect is that all threads of the gateway are blocked, and even user requests that do not call service 1 cannot be processed. Finally, the entire system loses response and the application crashes. Using blocking programming, the actual stress test effect is as follows. When the response of service 1 is delayed and the error rate soars sharply, the error rate of calling normal service 2 through the gateway is also very high.
Using the gateway developed by Flower, the actual stress test results are as follows. In the same situation where service 1 has a delayed response and an extremely high error rate, calling service 2 through the Flower gateway is not affected at all.
Summary
In fact, Flower is not only a reactive web programming framework, but also a reactive microservices framework. In other words, Flower’s Service can be remotely deployed into a Service container, just like the microservice architecture we commonly use today. Flower will provide an independent Flower container for starting some services. After these services are started, they will be registered with the registration center, and the application can process these distributed services to obtain a distributed non-blocking service. Microservice system. The overall architecture is very similar to the mainstream microservice architecture. The main difference is that Flower’s services are asynchronous, and service calls are made through process orchestration instead of through interface dependencies.
You can click here to enter the source code address of the Flower framework. You are welcome to participate in Flower development and apply Flower to your system development.