28. Advanced IO and multiplexing options

Article directory

  • 1. Five IO models
    • (1) Blocking IO:
    • (2) Non-blocking IO:
    • (3) Signal driver IO:
    • (4) IO multiplexing:
    • (5) Asynchronous IO:
  • 2. Important concepts of advanced IO
    • (1) Synchronous communication vs asynchronous communication (synchronous communication/ asynchronous communication)
    • (2) Blocking vs non-blocking
    • (3) Other advanced IO
    • (4) Non-blocking IO
      • 1.fcntl
      • 2. Implement the function SetNoBlock
  • 3. Selection of I/O multiplexing
    • (1) First acquaintance with select
    • (2) Select function prototype
      • 1. Parameter explanation:
      • 2. Parameter timeout value:
      • 3. About the fd_set structure
      • 4. About the timeval structure
    • (3) Understand the select execution process
    • (4) Socket readiness conditions
    • (5) Characteristics of select
    • (6) Disadvantages of select
    • (7) Select usage examples

1. Five IO models

(1) Blocking IO:

Before the kernel prepares the data, the system call will wait. All sockets are blocked by default.

(2) Non-blocking IO:

Non-blocking IO often requires programmers to repeatedly try to read and write file descriptors in a loop. This process is called polling. This is a huge waste of CPU and is generally only used in specific scenarios.

(3) Signal driver IO:

When the kernel prepares the data, it uses the SIGIO signal to notify the application to perform IO operations.

(4) IO multiplexing:

Although it looks similar to blocking IO from the flow chart, the core is actually that IO multiplexing can wait for the ready status of multiple file descriptors at the same time.

(5) Asynchronous IO:

The kernel notifies the application when the data copy is completed (and the signal driver tells the application when it can start copying data).

Any IO process contains two steps. The first is waiting, and the second is copying. And in actual application scenarios, the time consumed by waiting is often much higher than the time spent copying. Make IO more efficient , the core method is to minimize the waiting time.

2. Important concepts of advanced IO

(1) Synchronous communication vs asynchronous communication (synchronous communication/ asynchronous communication)

Synchronization and asynchronous focus on the message communication mechanism.

  • The so-called synchronization means that when a call is issued, the call will not return until the result is obtained. But once the call returns, the return value will be obtained; in other words, the caller actively waits for the result of the call;
  • Asynchronous is the opposite. After the call is issued, the call returns directly, so no result is returned; in other words, when an asynchronous procedure call is issued, the caller will not get the result immediately; instead, after the call is issued, the result is The caller notifies the caller through status, notification, or handles the call through a callback function.

In addition, we recall that when we talked about multi-process and multi-threading, we also mentioned synchronization and mutual exclusion. Synchronous communication here and synchronization between processes are completely unrelated concepts.

  • Process/thread synchronization is also the direct restriction relationship between processes/threads
  • It is a constraint relationship created by two or more threads established to complete a certain task. This thread needs to coordinate their work order at certain positions while waiting and transmitting information. Especially when accessing critical resources.

(2) Blocking vs non-blocking

Blocking and non-blocking focus on the state of the program while waiting for the call result (message, return value).

  • A blocking call means that the current thread will be suspended before the call result is returned. The calling thread will only return after getting the result.
  • A non-blocking call means that the call will not block the current thread until the result cannot be obtained immediately.

(3) Other advanced IO

Non-blocking IO, record locks, system V stream mechanism, I/O multiplexing (also called I/O multiplexing), readv and writev functions, and storage mapped IO (mmap), these are collectively called advanced IO.

(4) Non-blocking IO

1.fcntl

A file descriptor blocks IO by default.
prototype:

#include <unistd.h>
#include <fcntl.h>
int fcntl(int fd, int cmd, ... /* arg */);
The value of the cmd passed in is different, and the parameters appended later are also different.
The fcntl function has 5 functions:
Duplicate an existing descriptor (cmd=F_DUPFD).
Get/set the file descriptor flag (cmd=F_GETFD or F_SETFD).
Get/set file status flag (cmd=F_GETFL or F_SETFL).
Get/set asynchronous I/O ownership (cmd=F_GETOWN or F_SETOWN).
Get/set record lock (cmd=F_GETLK, F_SETLK or F_SETLKW).
We only use the third function here, to get/set the file status flag, to set a file descriptor to non-blocking

2. Implement function SetNoBlock

Based on fcntl, we implement a SetNoBlock function to set the file descriptor to non-blocking.

void SetNoBlock(int fd) {<!-- -->
 int fl = fcntl(fd, F_GETFL);
 if (fl < 0) {<!-- -->
 perror("fcntl");
 return;
 }
 fcntl(fd, F_SETFL, fl | O_NONBLOCK);
}

Use F_GETFL to get the attributes of the current file descriptor (this is a bitmap).
Then use F_SETFL to set the file descriptor back. While setting it back, add an O_NONBLOCK parameter.

  • Reading standard input in polling mode
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
void SetNoBlock(int fd) {<!-- -->
int fl = fcntl(fd, F_GETFL);
if (fl < 0) {<!-- -->
perror("fcntl");
return;
}
fcntl(fd, F_SETFL, fl | O_NONBLOCK);
}
int main() {<!-- -->
SetNoBlock(0);
while (1) {<!-- -->
char buf[1024] = {<!-- -->0};
ssize_t read_size = read(0, buf, sizeof(buf) - 1);
if (read_size < 0) {<!-- -->
perror("read");
sleep(1);
continue;
}
printf("input:%s\\
", buf);
}
return 0;
}

3. I/O multiplexing selection

(1) First acquaintance with select

The system provides the select function to implement the multiplexed input/output model:

  • The select system call is used to allow our program to monitor the status changes of multiple file descriptors;
  • The program will stop at select and wait until one or more of the monitored file descriptors changes state;

(2) select function prototype

The function prototype of 

select is as follows: #include <sys/select.h>
int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout);

1. Parameter explanation:

  • The parameter nfds is the largest file descriptor value that needs to be monitored + 1;
  • rdset, wrset, and exset respectively correspond to the set of readable file descriptors, the set of writable file descriptors, and the set of abnormal file descriptors that need to be detected;
  • The parameter timeout is the structure timeval, used to set the waiting time of select()

2. Parameter timeout value:

  • NULL: It means that select() has no timeout, and select will be blocked until an event occurs on a certain file descriptor;
  • 0: Only detect the status of the descriptor set and then return immediately without waiting for external events to occur.
  • Specific time value: If no event occurs within the specified time period, select will time out and return.

3. About the fd_set structure



In fact, this structure is an integer array, more strictly speaking, a “bitmap”. The corresponding bits in the bitmap are used to represent the file descriptor to be monitored.
Provides a set of interfaces for operating fd_set to more conveniently operate bitmaps.

void FD_CLR(int fd, fd_set *set); // Used to clear the bits related to fd in the description phrase set
int FD_ISSET(int fd, fd_set *set); // Used to test whether the bit related to fd in the description phrase set is true
void FD_SET(int fd, fd_set *set); // Used to set the bits related to fd in the description phrase set
void FD_ZERO(fd_set *set); // Used to clear all bits of the description phrase set

4. About timeval structure

The timeval structure is used to describe the length of a period of time. If no event occurs in the descriptor that needs to be monitored within this time, the function returns and the return value is 0.

Function return value:

  • If the execution is successful, the number of file descriptor status changes will be returned.
  • If 0 is returned, it means that the timeout time has passed before the descriptor status changes, and no return is made.
  • When an error occurs, -1 is returned and the cause of the error is stored in errno. At this time, the values of the parameters readfds, writefds, exceptfds and timeout become unpredictable.

Error values may be:

EBADF file descriptor is invalid or the file has been closed
EINTR This call was interrupted by a signal
EINVAL parameter n is a negative value.
ENOMEM Insufficient core memory

(3) Understanding the select execution process

The key to understanding the select model is to understand fd_set. For the convenience of explanation, the length of fd_set is 1 byte. Each bit in fd_set can correspond to a file descriptor fd. Then a 1-byte long fd_set can correspond to a maximum of 8 fds.

* (1) Execute fd_set set; FD_ZERO( & amp;set); then the bit representation of set is 0000,0000.
*(2) If fd=5, execute FD_SET(fd, & amp;set); and set becomes 0001,0000 (the fifth position is 1)
*(3) If fd=2 and fd=1 are added, the set becomes 0001,0011
*(4) Execute select(6, & set,0,0,0) to block and wait
*(5) If readable events occur on both fd=1 and fd=2, select returns and set changes to 0000,0011. Note: fd=5 without events is cleared.

(4) socket readiness conditions

Ready to read:

  • In the socket kernel, the number of bytes in the receive buffer is greater than or equal to the low water mark SO_RCVLOWAT. At this time, the file can be read without blocking
  • descriptor, and the return value is greater than 0;
  • In socket TCP communication, the peer closes the connection. If the socket is read at this time, 0 will be returned; there is a new connection request on the monitored socket;
  • There is an unhandled error on the socket;
    Ready to write:
  • In the socket kernel, the number of available bytes in the send buffer (the size of the free location in the send buffer) is greater than or equal to the low water mark
    SO_SNDLOWAT, you can write without blocking at this time, and the return value is greater than 0;
  • The write operation of the socket is closed (close or shutdown). Writing to a socket whose write operation is closed will trigger the SIGPIPE signal;
  • After the socket connects successfully or fails using non-blocking connect;
  • There is an unread error on the socket;

(5) Characteristics of select

  • The number of monitorable file descriptors depends on the value of sizeof(fd_set). On my server, sizeof(fd_set)=512. Each bit represents a file descriptor, so the maximum file descriptor supported on my server is 512* 8=4096.
  • When adding fd to the select monitoring set, a data structure array must be used to save the fd placed in the select monitoring set.
    One is for selecting and returning, array is used as source data and fd_set for FD_ISSET judgment.
    Second, after the select returns, the previously added fd but no event occurred will be cleared. Each time before starting the select, the fd must be obtained from the array and added one by one (FD_ZERO first). While scanning the array, the maximum fd value maxfd is obtained. The first parameter for select.

(6) Select Disadvantages

  • Every time you call select, you need to manually set the fd collection, which is also very inconvenient from the perspective of interface usage.
  • Every time select is called, the fd collection needs to be copied from user mode to kernel mode. This overhead will be very large when there are many fds.
  • At the same time, each time you call select, you need to traverse all the passed FDs in the kernel. This overhead is also very large when there are many FDs.
    The number of file descriptors supported by select is too small.

(7) Select usage examples

#pragma once
#include <vector>
#include <unordered_map>
#include <functional>
#include <sys/select.h>
#include "tcp_socket.hpp"
//Necessary debugging functions
inline void PrintFdSet(fd_set* fds, int max_fd) {<!-- -->
printf("select fds: ");
for (int i = 0; i < max_fd + 1; + + i) {<!-- -->
if (!FD_ISSET(i, fds)) {<!-- -->
continue;
}
printf("%d ", i);
}
printf("\\
");
}
typedef std::function<void (const std::string & amp; req, std::string* resp)> Handler;
// Encapsulate Select into a class. Although this class saves many TcpSocket object pointers, it does not manage memory
class Selector {<!-- -->
public:
Selector() {<!-- -->
// [Attention!] Don’t forget to initialize!!
max_fd_ = 0;
FD_ZERO( & amp;read_fds_);
}
bool Add(const TcpSocket & amp; sock) {<!-- -->
int fd = sock.GetFd();
printf("[Selector::Add] %d\\
", fd);
if (fd_map_.find(fd) != fd_map_.end()) {<!-- -->
printf("Add failed! fd has in Selector!\\
");
return false;
}
fd_map_[fd] = sock;
FD_SET(fd, & amp;read_fds_);
if (fd > max_fd_) {<!-- -->
max_fd_ = fd;
}
return true;
}
bool Del(const TcpSocket & amp; sock) {<!-- -->
int fd = sock.GetFd();
printf("[Selector::Del] %d\\
", fd);
if (fd_map_.find(fd) == fd_map_.end()) {<!-- -->
printf("Del failed! fd has not in Selector!\\
");
return false;
}
fd_map_.erase(fd);
FD_CLR(fd, & amp;read_fds_);
// Re-find the largest file descriptor. It is faster to search from right to left.
for (int i = max_fd_; i >= 0; --i) {<!-- -->
if (!FD_ISSET(i, & amp;read_fds_)) {<!-- -->
continue;
}
max_fd_ = i;
break;
}
return true;
}
 // Return the read-ready file descriptor set
bool Wait(std::vector<TcpSocket>* output) {<!-- -->
output->clear();
// [Note] A temporary variable must be created here, otherwise the original result will be overwritten.
fd_set tmp = read_fds_;
//DEBUG
PrintFdSet( & amp;tmp, max_fd_);
int nfds = select(max_fd_ + 1, & amp;tmp, NULL, NULL, NULL);
if (nfds < 0) {<!-- -->
perror("select");
return false;
 }
 // [Note!] The loop condition here must be i < max_fd_ + 1
for (int i = 0; i < max_fd_ + 1; + + i) {<!-- -->
if (!FD_ISSET(i, & amp;tmp)) {<!-- -->
continue;
}
output->push_back(fd_map_[i]);
}
return true;
 }
private:
fd_set read_fds_;
int max_fd_;
// Mapping relationship between file descriptors and socket objects
std::unordered_map<int, TcpSocket> fd_map_;
};
class TcpSelectServer {<!-- -->
public:
 TcpSelectServer(const std::string & amp; ip, uint16_t port) : ip_(ip), port_(port) {<!-- -->
 
 }
 bool Start(Handler handler) const {<!-- -->
 // 1. Create socket
TcpSocket listen_sock;
bool ret = listen_sock.Socket();
if (!ret) {<!-- -->
return false;
}
 // 2. Bind port number
ret = listen_sock.Bind(ip_, port_);
if (!ret) {<!-- -->
return false;
}
 // 3. Monitor
ret = listen_sock.Listen(5);
if (!ret) {<!-- -->
return false;
 }
 // 4. Create Selector object
Selector selector;
selector.Add(listen_sock);
 // 5. Enter the event loop
for (;;) {<!-- -->
std::vector<TcpSocket> output;
bool ret = selector.Wait( & amp;output);
if (!ret) {<!-- -->
continue;
}
 // 6. Determine subsequent processing logic based on the difference in ready file descriptors
for (size_t i = 0; i < output.size(); + + i) {<!-- -->
if (output[i].GetFd() == listen_sock.GetFd()) {<!-- -->
// If the ready file descriptor is listen_sock, execute accept and add it to select
TcpSocket new_sock;
listen_sock.Accept( & amp;new_sock, NULL, NULL);
selector.Add(new_sock);
} else {<!-- -->
// If the ready file descriptor is new_sock, process the request once
std::string req, resp;
bool ret = output[i].Recv( & amp;req);
if (!ret) {<!-- -->
selector.Del(output[i]);
// [Note!] The socket needs to be closed
output[i].Close();
continue;
}
// Call the business function to calculate the response
handler(req, & amp;resp);
//Write the results back to the client
output[i].Send(resp);
}
} // end for
} // end for (;;)
return true;
}
private:
 std::string ip_;
 uint16_t port_;
};