Talk about the Netty threading model

Hello everyone, I am Yi An!

Netty is a high-performance network application framework, which is widely used. At present, in the field of Java, Netty has basically become the standard configuration of network programs. The Netty framework is rich in functions and very complex. Today we mainly analyze the threading model in the Netty framework, and The threading model directly affects the performance of network programs.

The bottleneck of network programming performance

In the BIO model, all read() operations and write() operations will block the current thread. If the client has established a connection with the server and does not send data for a long time, the read() operation of the server will always be blocked. So Using the BIO model, generally an independent thread will be assigned to each socket, so that the thread will not be blocked in a socket It affects the reading and writing of other sockets. The threading model of BIO is shown in the figure below, each socket corresponds to an independent thread; in order to avoid frequent creation and consumption of threads, a thread pool can be used, but the correspondence between sockets and threads will not change.

BIO’s threading model

The BIO threading model is suitable for scenarios where there are not many socket connections; however, in the current Internet scenario, the server often needs to support 100,000 or even millions of connections, and it is obviously unrealistic to create 100,000 or even millions of threads, so BIO threads The model cannot solve the problem of millions of connections. If you observe carefully, you will find that in Internet scenarios, although there are many connections, the requests on each connection are not frequent, so threads spend most of their time waiting for I/O to be ready. That is to say, the thread is blocked there most of the time, which is a complete waste, if we can solve this problem, there will be no need for so many threads.

Following this line of thought, we can optimize the threading model as shown in the figure below, and use one thread to handle multiple connections, so that the utilization rate of threads will increase, and the number of required threads will also decrease. This idea is very good, but it is impossible to use BIO-related APIs. Why? Because BIO-related socket read and write operations are all blocking, and once the blocking API is called, the calling thread will be blocked until the I/O is ready, and other socket connections cannot be processed.

Ideal thread model diagram

Fortunately, Java also provides a non-blocking (NIO) API, using the non-blocking API, one thread can handle multiple connections. How to achieve it? Now it is generally using the Reactor mode, including the implementation of Netty. Therefore, in order to understand the implementation of Netty, we need to understand the Reactor mode first.

Reactor mode

The following is the class structure diagram of the Reactor mode, where Handle refers to the I/O handle. In Java network programming, it is essentially a network connection. Event Handler is easy to understand. It is an event handler. The handle_event() method handles I/O events, that is, each Event Handler handles an I/O Handle; the get_handle() method can return the I/O Handle. Synchronous Event Demultiplexer can be understood as the I/O multiplexing API provided by the operating system, such as select() in the POSIX standard and epoll() in Linux.

Reactor pattern class structure diagram

The core of the Reactor mode is naturally the class Reactor, where the two methods register_handler() and remove_handler() can register and delete a Event handler; handle_events() method is the core, which is also the engine of Reactor mode. The core logic of this method is as follows: First, through the synchronization The select() method provided by the event multiplexer monitors network events, and when a network event is ready, it traverses the event handler to process the network event. Since network events are continuous, to start the Reactor mode in the main program, you need to call the handle_events() method in the form of while(true){}.

void Reactor::handle_events(){ //provided by synchronous event multiplexer //select() method listens to network events select(handlers); // handle network events for(h in handlers){ h. handle_event(); } } // start the event loop in the main program while (true) { handle_events(); } 

Netty threading model

Although the implementation of Netty refers to the Reactor mode, it does not completely copy it. The core concept in Netty is the event loop (EventLoop) , which is actually Reactor in Reactor mode, Responsible for monitoring network events and calling event handlers for processing. In the 4.x version of Netty, the network connection and EventLoop have a stable many-to-one relationship, while the EventLoop and Java threads have a one-to-one relationship. Stability here means that once the relationship is determined, it will no longer change. That is to say, one network connection will only correspond to one EventLoop, and one EventLoop will only correspond to one Java thread, so One network connection only It will correspond to a Java thread.

What are the benefits of a network connection corresponding to a Java thread? The biggest advantage is that the event processing for a network connection is single-threaded, which avoids various concurrency issues.

The threading model in Netty can refer to the figure below. The core goal is to use one thread to handle multiple network connections.

Threading model in Netty

Another core concept in Netty is EventLoopGroup, as the name suggests, an EventLoopGroup consists of a group of EventLoops. In actual use, two EventLoopGroups are generally created, one is called bossGroup and the other is called workerGroup. Why are there two EventLoopGroups?

This is related to the socket’s mechanism for processing network requests. The socket handles TCP network connection requests in an independent socket. Whenever a TCP connection is successfully established, a new socket will be created, and then the reading and writing of the TCP connection will be Completed by the newly created socket. That is to say, processing TCP connection requests and reading and writing requests is done through two different sockets. When we discussed network requests above, in order to simplify the model, we only discussed read and write requests, but did not discuss connection requests.

In Netty, bossGroup is used to handle connection requests, while workerGroup is used to handle read and write requests. After the bossGroup finishes processing the connection request, it will submit the connection to the workerGroup for processing. There are multiple EventLoops in the workerGroup. Which EventLoop will the new connection be handed over to handle? This requires a load balancing algorithm. Netty currently uses the round-robin algorithm.

Next, we use Netty to re-implement the server side of the following echo program, and feel Netty up close.

Implement the Echo program server with Netty

The following sample code implements the echo program server based on Netty: first creates an event handler (equivalent to the event handler in Reactor mode), then creates bossGroup and workerGroup, and then creates and initializes ServerBootstrap, the code is still very Simple, but there are two places to pay attention to.

First, if the NettybossGroup only listens to one port, then the bossGroup only needs one EventLoop, and more is a waste.

Second, by default, Netty will create “2*CPU cores” EventLoop. Since the network connection has a stable relationship with the EventLoop, the event processor cannot have blocking operations when processing network events, otherwise it will be very It is easy to cause a large area of requests to time out. If it is really unavoidable to use blocking operations, it can be processed asynchronously through the thread pool.

//event handler final EchoServerHandler serverHandler = new EchoServerHandler(); //boss thread group EventLoopGroup bossGroup = new NioEventLoopGroup(1); //worker thread group EventLoopGroup workerGroup = new NioEventLoopGroup(); try { ServerBootstrap b = new ServerBootstrap(); b. group(bossGroup, workerGroup) .channel(NioServerSocketChannel.class) .childHandler(new ChannelInitializer<SocketChannel>() { @Override public void initChannel(SocketChannel ch){ ch.pipeline().addLast(serverHandler); } }); //bind server port ChannelFuture f = b.bind(9090).sync(); f. channel(). closeFuture(). sync(); } finally { // Terminate worker thread group workerGroup. shutdown Gracefully(); // terminate the boss thread group bossGroup. shutdown Gracefully(); } //socket connection handler class EchoServerHandler extends ChannelInboundHandlerAdapter { //handle the read event @Override public void channelRead( ChannelHandlerContext ctx, Object msg){ ctx.write(msg); } // Handle the read completion event @Override public void channelReadComplete( ChannelHandlerContext ctx){ ctx.flush(); } // Handle exception events @Override public void exceptionCaught( ChannelHandlerContext ctx, Throwable cause) { cause. printStackTrace(); ctx. close(); } } 

Summary

Netty is an excellent network programming framework with very good performance. In order to achieve the goal of high performance, Netty has done a lot of optimization, such as optimizing ByteBuffer, supporting zero copy, etc. Its threading model is related to concurrent programming. Netty’s threading model is designed very delicately. Each network connection is associated with a thread. The advantage of this is that for a network connection, read and write operations are performed in a single thread, thus avoiding various problems of concurrent programs. .

This article is published by mdnice multi-platform