Every time I implement the tcp server side, I always think: When dealing with the details of the received client message, I always fall into a little misunderstanding.
In addition to analyzing the various business codes of the predecessors of the company, they will always be slightly biased. Here is a simple processing idea of relevant streams after tcp reception.
0: Summary
When handling tcp receive:
1: tcp is reliable, the kernel allocates a send buffer and a receive buffer for each tcp client connection.
2: Based on the first point:
====” For each connection, the data can be received reliably and in order (that is, put into the corresponding receive buffer).
====”The buffer of each connection is independent, and there will be no string packet phenomenon.
====” is stored in the buffer, it is in the form of a stream, the boundaries of multiple packets cannot be recognized, and the business layer needs to be adapted.
3: Based on the second point:
====”We need to adapt at the business layer to identify a complete package. Generally, there are two schemes: (Length + Data), ( specific header or tail identifier), my understanding of other related specific protocols is probably the same
====”As the server side, we need to process multiple receive buffers (epoll/select management) at the same time, and at the same time consider how to read a complete package (how long the data is read each time, you can definitely read one complete package?)
4: Based on the third point: how to ensure that the event is triggered once and the content of the buffer can be read correctly.
===” For example, if it is in epoll ET mode, you should read all the data in a loop (there may be sticky packets, which need to be processed)
===” If it is epoll LT or select mode, the code is relatively simple, but if the bottom layer of the tcp kernel has been unpacked and sent over multiple times, there may be a half-packet phenomenon, and how is this length defined during recv?
===” for the above: (The best I can think of is: read a fixed length, while reading is put into the application layer buffer (we should maintain an application layer buffer per connection), and then the buffer does unpacking)
5: Based on the fourth point, what if some code is not temporarily stored in the application layer buffer? (Only consider the mode of Length + data)
If you see some code, it is a specific business, that is, every time an io event is triggered, receive a specific length first, and then read the actual data, which can determine a complete package for business processing.
I will think about whether there will be any flaws in the processing, and the easiest thing to think about is
? 1: One event triggers processing of a packet, will there be data unprocessed, there is still data in the receive buffer of the kernel, such as epoll’s ET mode scene, but it looks like select and epoll’s LT Has little effect
? 2: If the business layer does not do related processing, the possible scenario is tcp bottom layer unpacking, so read the specific self-fetched length first, and then read the actual length data /strong>, maybe there is a problem? The next package never arrived.
===” However, in the final thought, if the business layer has done processing to ensure that the bottom layer of tcp will not be unpacked, it should be feasible if we do not use the application layer buffer.
===” That is, the business layer has guaranteed that each reception is a complete package of a specific format (length + data), each time a specific byte header is received first, and after parsing, the actual length data is received and then the complete package is processed.
===” Considering the event-triggering feature to ensure complete processing, when we process epoll ET, we should read all the packets in one cycle, and epoll LT and select modes seem to have little impact and can be read correctly.
1: When tcp is processing related recv
In actual business, we usually cooperate with select or epoll to manage related connections on the server side. When receiving messages from clients, there are some concerns:
1.1: tcp is reliable streaming, and in fact the server side maintains a receive buffer and a send buffer for each client.
First: The reception of the underlying stream of tcp can be guaranteed. (has its own buffer and is reliable, sequential)
For each client’s connection, it can be guaranteed to be received reliably in order and put into its corresponding buffer.
===” I have always fallen into a misunderstanding. If you rely on the underlying unpacking logic of tcp, you may receive other packets in the middle of receiving multiple packets. This is actually a ideological misunderstanding, it is impossible to string packets /strong>.
===”The server has a sending buffer and a receiving buffer for each connection of tcp. For a connection plus the reliable transmission of tcp, the reception of the underlying tcp can be guaranteed.
Second: A custom protocol is required to handle the buffer stream correctly.
===> Although tcp is reliable, it is received in the form of a stream, and it is impossible to know the boundaries of multiple packets.
===” In order to correctly identify each complete packet, go to correct processing (identify a complete packet (tcp recv is to take data from the buffer, first: The buffer may have multiple packets ( Sticky package) Second: maybe recv has read half of the package, or tcp bottom layer unpacking, the second package has not come yet))
===” So we need to make specific restrictions on the flow at the business layer to ensure that a packet can be recognized: such as length + data, such as plus a specific identified protocol header or tail strong>
Third: On the basis of a specific protocol, how to ensure that multiple buffers are correctly read.
===” If it is the ET edge trigger mode of epoll, you have to use the while loop to read multiple times
===”If it is the LT horizontal trigger mode of epoll, or select, it can be triggered next time, but it can also be read and processed in a loop to improve efficiency.
Fourth: how to ensure the integrity of the package, a complete package to do the corresponding business processing.
===”According to the application layer protocol, the data can be read first, put into the cache, and then the relevant data in the cache can be parsed according to the protocol to process the complete package. (Correspondingly, the application layer buffer should also be a connection corresponding to a buffer)
? ? There is no problem using the buffer method, but for a specific protocol, such as the length + data method, is it okay to read specific bytes first, parse the actual length, and then receive specific actual data?
===”Personal understanding is that it is possible to guarantee a certain business layer.
===”If tcp sends an oversized packet, it will be unpacked. After this scenario event is triggered, read the actual length according to the specific own byte, and there will be a problem if the next packet is delayed.
===” But if tcp our business layer guarantees that we will not return to tcp for unpacking, and we control the size of the packet, I think it is actually feasible.
2: Pay attention to details: network byte order, structure alignment
If you implement tcp sending and receiving related protocol design in the code, you need to pay attention to some details:
1: If the protocol uses the structure method, pay attention to the size of the structure byte alignment, which will affect the parsing of the receiving end.
2: Generally larger than 2byte bytes, and finally perform the relevant host byte order and network byte order conversion according to specific functions
#include <arpa/inet.h> uint32_t htonl(uint32_t hostlong); //32-bit host byte order to network byte order Host to Network Long 4 bytes uint16_t htons(uint16_t hostlong); //The 16-bit host byte order is converted to the network byte order Host to Network Short 2 bytes uint32_t ntohl(uint32_t hostlong) //32-bit network byte order to host byte order Network to Host Long uint16_t ntohs(uint16_t hostlong) //16-bit network byte order to host byte order Network to Host Short
3: select: Invalid argument during code testing
The stack memory definition structure variable is best to be cleared.
When a simple demo is used for testing, the environment operation cannot be started, and the error select: Invalid argument is reported.
After Baidu combined the test, it was found that the last parameter struct timeval tv was selected; the problem of setting
===”1: Referring to Baidu, there is a similar problem because the tv.tv_usec value is set too large.
===”2: However, I did not set this value too large in my code, but did not clear it during initialization.
struct timeval tv;
memset( & amp;tv, 0, sizeof(struct timeval)); //Note that the initialized memory is cleaned up here
fd_set rset;
int maxfd = m_listenfd + 1;
while(m_running)
{<!-- -->
tv.tv_sec = 30;
//tv.tv_usec = 0; //Because of the stack memory used, if the memory here is relatively large, it will cause select Invalid argument
rset = m_allset;
ret = select(maxfd, & amp;rset, (fd_set *)0,(fd_set *)0, (struct timeval *) & amp;tv);
...
}
===” In addition, I heard colleagues say that the tcp buffer overflow problem, my personal understanding is that the tcp receive buffer will not have the so-called overflow problem, and the send buffer should have similar problems because of the sending frequency.