In the previous two lectures, I used fork processes and pthread threads to handle multiple concurrencies respectively. These two technologies are simple to use, but their performance will drop rapidly as the number of concurrencies increases, and they cannot meet the needs of extremely high concurrency. As mentioned in Lecture 24, we need to find a better solution at this time. The basic idea of this solution is I/O event distribution.
Revisit event-driven
Event-based programming: GUI, Web
The advantages of event-driven are that it takes up less resources, is highly efficient, and has strong scalability. It is the best choice to support high performance and high concurrency.
If you are familiar with GUI programming, you will know that GUI sets up a series of controls, such as Button, Label, text box, etc. When we design a program based on controls, we usually arrange a function for the click of Button, similar to so:
//Button click event processing
void onButtonClick(){
}
The idea of this design is that an infinite loop event distribution thread runs in the background. Once the user performs some kind of operation on the interface, such as clicking a button or clicking a text box, an event will be generated and placed. In the event queue, this event will have a callback function similar to the previous onButtonClick. The task of the event distribution thread is to find the corresponding event callback function for each event that occurs and execute it. In this way, an event-driven GUI program can work perfectly.
Another similar example is in the field of web programming. Similarly, Web programs will place various interface elements on the Web interface, such as labels, text boxes, buttons, etc. Similar to GUI programs, JavaScript callback functions are designed for the interface elements of interest. When the user operates, the corresponding JavaScript callback The function will be executed to complete a calculation or operation. In this way, an event-driven web program can work perfectly in the browser.
In Lecture 24, we have mentioned that by using I/O distribution technologies such as poll and epoll, a socket-based event driver can be designed to meet the requirements of high performance and high concurrency.
The event-driven model is also called the reactor model (reactor), or the event loop model. There are two core points in this model.
First, it has an infinite loop event distribution thread, also called reactor thread or Event loop thread. Behind this event distribution thread is the use of I/O distribution technologies such as poll and epoll.
Second, all I/O operations can be abstracted into events, and each event must have a callback function to handle it. The connection is successfully established on the acceptor, the send buffer on the connected socket is free for writing, and there is data on the communication pipe that can be read. These are all events. Through event distribution, these events can be detected one by one. And call the corresponding callback function for processing.
Several I/O models and thread model designs
What any network program does can be summarized as follows:
read: Receive data from the socket;
decode: parse the received data;
compute: Calculate and process based on the parsed content;
Encode: Encode the processed results according to the agreed format;
send: Finally, send the result through the socket.
The two processes most related to sockets are read and send. Next, we summarize the several network programming technologies that we have learned that support multi-concurrency, leading to our topic today, using poll to handle all I/O in a single thread.
fork
In Lecture 25, we used fork to create child processes to serve each incoming client connection. This picture explains this design pattern very well. It is conceivable that as the number of clients increases, the number of fork child processes also increases. Even if there is less interaction between clients and servers, like this The child process cannot be destroyed and must always exist. It is very simple to use fork. Its disadvantage is that the processing efficiency is not high and the overhead of forking the child process is too high.
pthread
In Lecture 26, we used pthread_create to create a child thread. Because a thread is a more lightweight execution unit than a process, its efficiency is improved compared to the fork method. However, the overhead of creating a thread each time is still not small. Therefore, the concept of thread pool is introduced, a thread pool is created in advance, and each time a new connection arrives, a thread is selected from the thread pool to serve it. It solves the overhead of thread creation very well. However, this model still does not solve the problem of idle connections occupying resources. If a connection has no data interaction for a certain period of time, the connection will still occupy a certain amount of thread resources until the connection dies.
single reactor thread
As mentioned earlier, the event-driven model is a better model for solving high performance and high concurrency. why?
Because this model meets the needs of mass production. Similar patterns are everywhere in our lives. For example, if you go to a coffee shop to drink coffee, you order a cup of coffee and drink it, and the waiter will not care about you. When you need a refill, you will ask the waiter (trigger event) and the waiter will meet your needs. , you can continue to drink coffee and play on your phone. The service method of the entire counter is an event-driven method.
Here is a diagram that explains the design pattern for this lesson. A reactor thread is also responsible for distributing acceptor events and connected socket I/O events.
single reactor thread + worker threads
However, there is a problem with the above design pattern. Compared with I/O event processing, the application’s business logic processing is more time-consuming, such as XML file parsing, database record search, file data reading and transmission, calculation These tasks are relatively independent, and they will slow down the execution efficiency of the entire reactor mode.
Therefore, it is a wiser choice to place these decode, compute, and enode types of work in another thread pool and decouple them from the reactor thread. Reactor threads are only responsible for processing I/O-related work. Business logic-related work is cut into small tasks one by one and placed in the thread pool to be executed by idle threads. When the result is completed, it is handed over to the reactor thread, which sends the result through the socket.
Sample program
Starting today, we’ll be introduced to a network programming framework tailored for this course. A sample program using this network programming framework is as follows:
#include <lib/acceptor.h>
#include "lib/common.h"
#include "lib/event_loop.h"
#include "lib/tcp_server.h"
char rot13_char(char c) {
if ((c >= 'a' & amp; & amp; c <= 'm') || (c >= 'A' & amp; & amp; c <= 'M'))
return c + 13;
else if ((c >= 'n' & amp; & amp; c <= 'z') || (c >= 'N' & amp; & amp; c <= 'Z'))
return c - 13;
else
return c;
}
//Callback after the connection is established
int onConnectionCompleted(struct tcp_connection *tcpConnection) {
printf("connection completed\
");
return 0;
}
//Callback after data is read into buffer
int onMessage(struct buffer *input, struct tcp_connection *tcpConnection) {
printf("get message from tcp connection %s\
", tcpConnection->name);
printf("%s", input->data);
struct buffer *output = buffer_new();
int size = buffer_readable_size(input);
for (int i = 0; i < size; i + + ) {
buffer_append_char(output, rot13_char(buffer_read_char(input)));
}
tcp_connection_send_buffer(tcpConnection, output);
return 0;
}
//Callback after the data is written through the buffer
int onWriteCompleted(struct tcp_connection *tcpConnection) {
printf("write completed\
");
return 0;
}
//Callback after the connection is closed
int onConnectionClosed(struct tcp_connection *tcpConnection) {
printf("connection closed\
");
return 0;
}
int main(int c, char **v) {
//Main thread event_loop
struct event_loop *eventLoop = event_loop_init();
//Initialize acceptor
struct acceptor *acceptor = acceptor_init(SERV_PORT);
//Initial tcp_server, you can specify the number of threads. If the thread is 0, there will be only one thread, which is responsible for both acceptor and I/O.
struct TCPserver *tcpServer = tcp_server_init(eventLoop, acceptor, onConnectionCompleted, onMessage,
onWriteCompleted, onConnectionClosed, 0);
tcp_server_start(tcpServer);
// main thread for acceptor
event_loop_run(eventLoop);
}
The main function part of this program only has a few lines. Since this is the first time I have encountered it, I will introduce it a little bit.
Line 49 creates an event_loop, which is a reactor object. This event_loop is associated with a thread. Each event_loop executes an infinite loop in the thread to complete the distribution of events.
Line 52 initializes the acceptor to listen on a certain port.
Line 55 creates a TCPServer. You can specify the number of threads when creating it. The thread here is 0, so there is only one thread, which is responsible for both the acceptor connection processing and the I/O processing of the connected socket. The more important thing here is that several callback functions are passed in, which correspond to the completion of connection establishment, data reading, data sending, and connection closing. Through the callback functions, the business program can focus on the development of the business layer.
Line 57 turns on monitoring.
Line 60 runs the event_loop infinite loop, waiting for a connection to be established on the acceptor, data to be read on the new connection, etc.
Sample program results
Run this server program and open two telnet clients. We see the output of the server as follows:
$./poll-server-onethread [msg] set poll as dispatcher [msg] add channel fd == 4, main thread [msg] poll added channel fd==4 [msg] add channel fd == 5, main thread [msg] poll added channel fd==5 [msg] event loop run, main thread [msg] get message channel i==1, fd==5 [msg] activate channel fd == 5, revents=2, main thread [msg] new connection established, socket == 6 connection completed [msg] add channel fd == 6, main thread [msg] poll added channel fd==6 [msg] get message channel i==2, fd==6 [msg] activate channel fd == 6, revents=2, main thread get message from tcp connection connection-6 afadsfaf [msg] get message channel i==2, fd==6 [msg] activate channel fd == 6, revents=2, main thread get message from tcp connection connection-6 afadsfaf fdafasf [msg] get message channel i==1, fd==5 [msg] activate channel fd == 5, revents=2, main thread [msg] new connection established, socket == 7 connection completed [msg] add channel fd == 7, main thread [msg] poll added channel fd==7 [msg] get message channel i==3, fd==7 [msg] activate channel fd == 7, revents=2, main thread get message from tcp connection connection-7 sfasggwqe [msg] get message channel i==3, fd==7 [msg] activate channel fd == 7, revents=2, main thread [msg] poll delete channel fd==7 connection closed [msg] get message channel i==2, fd==6 [msg] activate channel fd == 6, revents=2, main thread [msg] poll delete channel fd==6 connection closed
There is only one main thread working here from beginning to end. It can be seen that a single-threaded reactor can also perform well when handling multiple connections.
Summary
In this lecture, we summarized several different I/O model and thread model designs, and compared their different advantages and disadvantages. Starting from this lecture, we will use the programming framework we wrote to complete business development. This lecture uses poll to handle all I/O events. In the next lecture, we will see how to convert the acceptor’s connection event The I/O events of the connected socket are handled by different threads, and this separation is nothing more than a simple parameter configuration at the application layer.