How many TCP connections can a server support at most?

1. The maximum number of files that can be opened by a server

1. Limit parameters

We know that everything is a file in Linux, so how many files can a server open at most? The maximum number of files that can be opened on Linux is affected by three parameters, namely:

  • fs.file-max (system-level parameter): This parameter describes the maximum number of files that the entire system can open. But the root user will not be limited by this parameter (for example: the number of file descriptors opened by the entire system has reached fs.file-max, and the root user can still use ps, kill, etc. command or open another file descriptor)
  • soft nofile (process level parameter): Limit the maximum number of files that can be opened on a single process. It can only be configured once on Linux, and different values cannot be configured for different users
  • fs.nr_open (process level parameter): Limit the maximum number of files that can be opened on a single process. Different values can be configured for different users

There is also a coupling relationship between these three parameters, so you need to pay attention to the following three points when configuring the values:

  1. If you want to increase the soft nofile, then the hard nofile parameter value also needs to be adjusted together. If the hard nofile parameter value is set low, then no matter how high the soft nofile parameter value is set, it is useless, and the actual effective value will be based on the lowest of the two.
  2. If the hard nofile is increased, then fs.nr_open also needs to be adjusted together (fs.nr_open parameter value must be greater than hard nofile parameter value). If you accidentally set the value of hard nofile larger than fs.nr_open, the consequences will be serious. This will cause the user to be unable to log in. If it is set to *, then all users will not be able to log in
  3. If the fs.nr_open is increased, but the fs.nr_openfs.nr_openecho "xxx" > ../fs/nr_open command is used to modify /code> value, then there may be no problem just after changing it, but as soon as the machine restarts, the fs.nr_open value set by the echo command before will become invalid, and the user still cannot log in. So it is not recommended to use echo to modify kernel parameters! ! !

2. Example of adjusting the maximum number of files that the server can open

Assuming that you want the process to open 1 million file descriptors, here is a suggestion by modifying the conf file. If there are similar needs in future work, it can be used as a reference.

  • vim /etc/sysctl.conf
fs.file-max=1100000 // Set the system level to 1.1 million, leave more buffer
fs.nr_open=1100000 // The process level is also set to 1.1 million, because it must be larger than the hard nofile
  • Make the above configuration effective sysctl -p

  • vim /etc/security/limits.conf

// User process level is set to 100
soft nofile 1000000
hard nofile 1000000

2. How many connections can a server support at most

We know that a TCP connection is fundamentally a set of [socket kernel objects] maintained in memory by the client and server (this also corresponds to the TCP quadruple: source IP, source port, target IP, target port), As long as they can find each other, it is considered a connection. So how many connections can a server establish at most?

  • Since a TCP connection can essentially be understood as a pair of socket kernel objects on the client-server side, theoretically it should be [2^32 (ip number) * 2^16 (port number)] connections (approximately equal to two hundred multi-trillion)
  • But in fact, due to the influence of other software and hardware, it is impossible for one of our servers to establish so many connections (mainly limited by CPU and memory).

If only the connections in the ESTABLISH state are counted (these connections are only established, but do not send or receive data or process related business logic), then how many connections can a server establish at most? Take a server with 4GB memory as an example!

  • In this case, the number of connections that can be established mainly depends on the [memory size] (because if it is) the idle connection in ESTABLISH state will not consume CPU (although there is TCP keep-alive packet transmission, But this effect is very small and can be ignored)
  • We know that a connection in ESTABLISH state consumes about [3.3KB of memory], then through calculation, we know that a server with 4GB of memory, [can establish a TCP connection of 100w+] (Of course, this is only the case where all connections are only established but do not send and process data. If there is data exchange and processing in the real scene (data receiving and sending need to apply for memory, data processing requires CPU), then it will It consumes more memory and takes up more CPU, and the concurrency cannot reach 100w+)

The above discussion is the ideal situation of establishing a connection. In reality, if there is frequent data sending and receiving and processing (such as: compression, encryption, etc.), then it is considered good for a server to support 1000 connections, so a server How many connections can be supported must be analyzed in combination with specific scenarios, and cannot be calculated solely by theoretical values. It doesn't make much practical sense to talk about concurrency without business logic.

The bulk of the server's overhead is often not the connection itself, but the data sent and received on each connection, as well as the request business logic processing! ! !

3. How many connections can a client machine initiate at most

We know that every time the client establishes a connection with the server, it will consume a port on the client side. The port range of a machine is [0 ~ 65535], so does it mean that a client machine can establish up to 65535 connections with a server machine (there are many reserved ports among the 65535 ports, and the available ports may only be 64000 or so)?

It can be seen from the quaternion characteristics of the TCP connection that as long as a certain element in the quaternion is different, it is considered to be a different TCP connection. So it needs to be discussed on a case-by-case basis:

[Situation 1], if a client has only one IP, the server also has only one IP and only one program is started, and in the case of listening to one port, the client and the server can be at most The number of established connections is 65535.

Because the source IP is fixed, the target IP and port are fixed, the only variable in the quadruple is [source port], and the available range of [source port] is [0 ~ 65535], so a client machine can create a maximum of 65535 connect.

[Situation 2 ], if a client has multiple IPs (assuming that the client has n IPs), and the server only has one IP and only one program is started to listen to one port, one The maximum number of connections that a client machine can establish is: n * 65535.

Because the target IP and port are fixed, and there are n source IPs, the variable in the quadruple is [source port] + [source IP], and the available range of [source port] is [0 ~ 65535], so the maximum value of one IP If 65535 connections can be established, then n IPs can establish a maximum of n * 65535 connections.

With the current technology, it is very easy to assign multiple IPs to a client, you only need to contact your network administrator to do it.

[Situation 3], if a client has only one IP, and the server has only one IP, but the server starts multiple programs, and each program listens to a port (for example, the server starts m programs, listening to m different ports), the maximum number of connections that can be established by a client machine is: 65535 * m.

The source IP is fixed, the target IP is fixed, the number of target ports is m, and the source port can be changed, and the source port range is [0 ~ 65535], so the maximum number of TCP connections that can be established by a client machine is 65535 * m indivual.

The rest of the situation is analogous, but the available port range of the client is generally less than 65535, which is limited by the kernel parameter net.ipv4.ip_local_port_range, if you want to modify the port range that the client can use, you can modify this kernel parameter value.

Therefore, not only a server can receive 100w + TCP connections, but a client can still send 100w + connections.

4. Other

  • The length of the full connection queue of the socket in the three-way handshake is controlled by the parameter net.core.somaxconn, and the default size is 128. When two machines are very close to each other, but the concurrency of establishing a connection is very high, It may cause the semi-connection queue or the full connection queue to overflow, which will cause the server to discard the handshake packet. Then it causes the client to timeout and retransmit the handshake packet (it will be retransmitted at least 1s later), resulting in the establishment of the three-way handshake connection taking too long. We can adjust the parameter net.core.somaxconn to increase the length of the connection queue, thereby reducing the impact of packet loss

  • Sometimes we terminate a certain process by means of ctrl + c, but when restarting the process, we find that the error port is occupied is reported. This problem is due to the [operating system I haven't had time to recycle the port, just wait a while and restart the application]

  • When the client program establishes a connection with the server, if the client does not call the bind method to pass in the specified port, then the client will randomly select a port to establish a connection when establishing a connection with the server. Once our client program calls the bind method to pass in the specified port, the client will use the port specified in our bind to establish a connection with the server. Therefore, it is not recommended that the client call the bind method. The bind function will change the strategy of the kernel to select the port.

    public static void main(String[] args) throws IOException {
        SocketChannel sc = SocketChannel. open();
      // The client can also call the bind method
        sc.bind(new InetSocketAddress("localhost", 9999));
        sc.connect(new InetSocketAddress("localhost", 8080));
        System.out.println("waiting..........");
    }
  • Everything in Linux is a file, of course, including the socket mentioned in the previous TCP connection. When a process opens a socket, several kernel objects need to be created. To put it bluntly, opening a file object consumes memory, so the Linux system is based on security considerations (for example: a user process maliciously opens countless file descriptors, you must not crash the system), and limit the number of file descriptors that can be opened in multiple places.

  • The kernel manages all sockets that have been connected through [hash table], so that when a request arrives, it can be quickly found through [TCP quadruple] The corresponding socket object in the kernel

In the epoll model, all sockets managed by the epoll object are managed through a red-black tree, and the red-black tree structure is used to balance the efficiency of fast deletion, insertion, and search of sockets

5. Related practical issues

In network development, many people have never fully understood a basic problem, that is, how many TCP connections can a machine support at most. However, because the client and the server use ports differently, it is easier to understand this problem apart.

Note that what is mentioned here is that both the client and the server are just roles, not a specific machine. For example, for an application developed by ourselves, when he responds to a client request, he is the server. When he requests data from MySQL, he becomes a client again.

What is wrong with "too many open files" and how to solve it

You may have encountered the error of too many open files online, so do you understand the principle of this error? If you were asked to fix this error, what should you do?

  • Because every time you open a file (including socket), you need to consume a certain amount of memory resources. In order to prevent individual processes from opening too many files uncontrollably and causing the entire server to crash, Linux has a limit on the number of open file descriptors. If your process hits a kernel limit, the "too many open files" error will be generated.

  • You can modify the maximum number of file descriptors that a process can open by modifying the values of the three parameters fs.file-max, soft nofile, and fs.nr_open.

Need to pay attention to the coupling relationship between these three parameters!

How many connections can a server machine support?

Because the maximum number is to be considered here, the data sending and receiving and processing on the connection are not considered first, only the empty connection in the ESTABLISH state is considered. So how many TCP connections can a server machine support? What factors will this number of connections be affected by?

  • In the case of not considering the sending, receiving and processing of data on the connection, but only considering the empty connection in the ESTABLISH state, the maximum number of TCP connections that can be supported on a server is basically determined by the size of the memory.

  • The quadruple uniquely determines a connection, but the server can receive requests from any client, so the number calculated according to this theory is too large to be practically meaningful. In addition, the file descriptor limit is actually a limit added by the kernel to prevent certain applications from opening [file handles] without restriction. This limit can be increased by modifying only a few kernel parameters.

  • A socket consumes about 3kb of memory, so the memory that really restricts the maximum concurrency of the server machine is the memory. Taking a server with 4GB memory as an example, the number of TCP connections that can be supported is about 100w +.

How many connections can a client machine support?

Unlike the server, the client needs to consume a port every time it establishes a connection. In the TCP protocol, the port is a 2-byte integer, so the range can only be 0~65535. So can a customer order only support a maximum of 65535 connections? Is there a way to break through this limitation, and if so, what are the ways?

  • Each time a client establishes a connection, it consumes a port. Judging by the numbers, it seems that there is only a maximum of 65535 connections established. But in fact we have two ways to break the 65535 limit.

Method 1, configure multiple IPs for clients Method 2, connect to different servers respectively

  • So there is no problem for a client to initiate millions of connections.

How many machines are needed to make a long connection push product to support 100 million users

Suppose you are a system architect, and now the boss gives you a requirement to make a product like Umeng upush. To maintain a long-term connection with the client on the server machine, the connection is idle in most cases, and the push is at most two or three times a day. The total user scale is expected to be 100 million. So now please evaluate how many servers are needed to support these 100 million long connections.

syntaxbug.com © 2021 All Rights Reserved.