The philosophy of Linux is that everything is a file, and files are manipulated through file descriptors. This article sorts out the operation of the dup / dup2 / sendfile / splice / tee function on the file descriptor.
Directory
1. dup
2.dup2
3. sendfile
4.splice
5. tee
1.dup
#include
int dup(int fd);
Duplicate an existing file descriptor, dup will return a new descriptor, this description must be the minimum value among the currently available file descriptors. We know that the general 0, 1, and 2 descriptors are occupied by standard input, output, and error respectively, so if the standard output 1 is closed in the program, the dup function is called, and the returned descriptor is 1. After the function returns, the return values of fd and dup point to the same file, but the file descriptors are different.
The following code demonstrates copying standard output to a normal file:
#include <assert.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <fcntl.h>
int main(int argc, char* argv[]) {
if (argc != 2) {
printf("usage: %s <file>\
", argv[0]);
return 1;
}
int filefd = open(argv[1], O_CREAT | O_WRONLY | O_TRUNC, 0666);
assert(filefd > 0);
close(STDOUT_FILENO);
printf("=== %d\
", dup(filefd));
write(filefd, "hello world\
", 12);
close(filefd);
printf("dup function, hello");
return 0;
}
operation result:
2.dup2
#include
int dup2(int fd, int fd2);
Copy an existing file descriptor, specify the value of the new descriptor with fd2, if fd2 itself is already open, it will be closed first. If fd is equal to fd2, returns fd2 without closing it. After the function returns, fd and fd2 point to the same file, but the file descriptors are different.
The following code demonstrates copying standard output to a normal file:
#include <assert.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <fcntl.h>
int main(int argc, char* argv[]) {
if (argc != 2) {
printf("usage: %s <file>\
", argv[0]);
return 1;
}
int filefd = open(argv[1], O_CREAT | O_WRONLY | O_TRUNC, 0666);
assert(filefd > 0);
write(filefd, "hello liudehua\
", 15);
printf("=== %d\
", dup2(filefd, STDOUT_FILENO));
write(filefd, "hello world\
", 12);
close(filefd);
return 0;
}
The content of the file is as follows:
Use strace to trace:
3. sendfile
#include
ssize_t sendfile(int out_fd, int in_fd, off_t* offset, size_t count);
Function: Transfer data between two file descriptors (completely operated in the kernel), thereby avoiding data copying between the kernel buffer and the user buffer, which is very efficient, which is the legendary zero copy.
parameter:
in_fd: The parameter is the file descriptor of the content to be read
out_fd: The parameter is the file descriptor of the content to be written
offset: The position where the parameter execution starts to read the file stream, if it is empty, the default starting position of the read file stream will be used
count: The parameter specifies the number of bytes transferred between the file descriptors in_fd and out_fd
Return value: returns the number of bytes transferred on success, returns -1 and sets errno on failure
Important note: The man manual of this function clearly states that in_fd must be a file descriptor that supports the mmap function, that is, it must point to a real file, not a socket or a pipe.
And out_fd must be a socket. So sendfile is almost exclusively designed for transferring files over the network
The following test code writes a simple tcp server. An example of sending a file to a client via sendfile.
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <assert.h>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/sendfile.h>
int main( int argc, char* argv[] ) {
if( argc <= 3 ) {
printf("usage: %s ip port filename\
", basename(argv[0]));
return 1;
}
const char* ip = argv[1];
int port = atoi(argv[2]);
const char* file_name = argv[3];
int filefd = open(file_name, O_RDONLY);
assert(filefd > 0);
struct stat stat_buf;
fstat(filefd, &stat_buf);
struct sockaddr_in address;
bzero( & amp; address, sizeof(address));
address.sin_family = AF_INET;
inet_pton(AF_INET, ip, & address. sin_addr);
address.sin_port = htons(port);
int sock = socket(PF_INET, SOCK_STREAM, 0);
assert(sock >= 0);
int ret = bind(sock, (struct sockaddr*) & address, sizeof(address));
assert(ret != -1);
ret = listen(sock, 5);
assert(ret != -1);
struct sockaddr_in client;
socklen_t client_addrlength = sizeof(client);
int connfd = accept(sock, (struct sockaddr*) &client, &client_addrlength);
if (connfd < 0) {
printf( "errno is: %d\
", errno);
}
else {
off_t len = 0;
while (len < stat_buf.st_size) {
int ret = sendfile(connfd, filefd, &len, stat_buf.st_size - len);
//printf("ret value %d \
", ret);
if (-1 == ret) {
if (EAGAIN == errno) {
//printf("no data\
");
perror("sendfile");
}
else {
printf("client quit \
");
break;
}
}
}
close(connfd);
}
close(sock);
return 0;
}
Using nc test, the results are as follows:
It can be seen that the contents of the file are correctly received by the nc program.
4.splice
#include
ssize_t splice(int fd_in, loff_t* off_in, int fd_out, loff_t* off_out, size_t len, unsigned int flags);
Function: The splice function is used to move data between two file descriptors, and it is also a zero copy
parameter:
fd_in/off_in: fd_in is the file descriptor of the data to be input, if fd_in is a pipeline file descriptor, then the off_in parameter must be set to NULL.
If fd_in is not a pipeline file (such as a socket), then off_in indicates where to start reading data from the input data stream. At this time, if off_in is set to NULL, it means reading from the current offset position of the input data stream;
If off_in is not NULL, it will point out the specific offset position.
The fd_out/off_out parameters have the same meaning as fd_in/off_in, but are used to output data streams.
len: the length of the mobile data
flag: Controls how data is moved, it can be set to a bitwise OR of some of the values in the table below.
return value:
When using the splice function, at least one of fd_in and fd_out must be a pipe file descriptor. The splice function returns the number of moved bytes on success. It may return 0, indicating that no data needs to be moved. This occurs when data is read from a pipe that has not been written to any data. When the spice function fails, return -1 and set errno. The common errno is as shown below 
The following test code writes a simple tcp server, using anonymous pipes combined with splice to echo information to the client.
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <assert.h>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <fcntl.h>
int main(int argc, char* argv[]) {
if (argc <= 2) {
printf("usage: %s ip port\
", basename(argv[0]));
return 1;
}
const char* ip = argv[1];
int port = atoi(argv[2]);
struct sockaddr_in address;
bzero( & amp; address, sizeof(address));
address.sin_family = AF_INET;
inet_pton(AF_INET, ip, & address. sin_addr);
address.sin_port = htons(port);
int sock = socket(PF_INET, SOCK_STREAM, 0);
assert(sock >= 0);
int ret = bind(sock, (struct sockaddr*) & address, sizeof(address));
assert(ret != -1);
ret = listen(sock, 5);
assert(ret != -1);
struct sockaddr_in client;
socklen_t client_addrlength = sizeof(client);
int connfd = accept(sock, (struct sockaddr*) &client, &client_addrlength);
if (connfd < 0) {
printf("errno is: %d\
", errno);
}
else {
//Create an anonymous queue, pipefd[0] read, pipefd[1] write
int pipefd[2];
assert(ret != -1);
ret = pipe(pipefd);
// connfd -> pipefd[1]
ret = splice(connfd, NULL, pipefd[1], NULL, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
assert(ret != -1);
// pipefd[0] -> connfd
ret = splice(pipefd[0], NULL, connfd, NULL, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
assert(ret != -1);
close(pipefd[0]);
close(pipefd[1]);
close(connfd);
}
close(sock);
return 0;
}
The result of the operation is as follows:
You can see that the content entered by the client is correctly echoed.
5. tee
#include
ssize_t tee(int fd_in, int fd_out, size_t len, unsigned int flags);
Function: Copy data between two pipeline file descriptors, which is also a zero-copy operation. It does not consume data, so the data on the source file descriptor is still available for subsequent read operations.
parameter:
fd_in: The file descriptor of the data to be input, which must be a pipeline file.
fd_out: The file descriptor of the data to be output, which must be a pipeline file.
len: the length of the assigned data (number of bytes)
flags modifier flags, share namespace with splice(2)/vmsplice(2):
1) SPLICE_F_MOVE currently has no effect on tee.
2) SPLICE_F_NONBLOCK non-blocking I/O operation, the actual effect will also be affected by the blocking state of the file descriptor itself.
3) SPLICE_F_MORE currently has no effect on tee.
4) SPLICE_F_GIFT has no effect on tee.
return value:
On success, returns the amount of data (in bytes) copied between the two file descriptors. Returning 0 means that no data has been copied, and EOF may be encountered. On failure, -1 is returned and errno is set.
The following code shows receiving data from the standard input and writing the data to a file through a pipeline. You can see that tee(pipefd_stdout[0] is called twice, and the data in pipefd_stdout[0] always exists.
#include <assert.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <fcntl.h>
int main(int argc, char* argv[]) {
if (argc != 2) {
printf("usage: %s <file>\
", argv[0]);
return 1;
}
int filefd = open(argv[1], O_CREAT | O_WRONLY | O_TRUNC, 0666);
assert(filefd > 0);
int pipefd_stdout[2];
int ret = pipe(pipefd_stdout);
assert(ret != -1);
int pipefd_file[2];
ret = pipe(pipefd_file);
assert(ret != -1);
// standard input STDIN_FILENO -> pipefd_stdout[1]
ret = splice(STDIN_FILENO, NULL, pipefd_stdout[1], NULL, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
assert(ret != -1);
// pipefd_stdout[0] -> pipefd_file[1]
ret = tee(pipefd_stdout[0], pipefd_file[1], 32768, SPLICE_F_NONBLOCK);
assert(ret != -1);
ret = tee(pipefd_stdout[0], pipefd_file[1], 32768, SPLICE_F_NONBLOCK);
assert(ret != -1);
// pipefd_file[0] -> filefd
ret = splice(pipefd_file[0], NULL, filefd, NULL, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
assert(ret != -1);
close(filefd);
close(pipefd_stdout[0]);
close(pipefd_stdout[1]);
close(pipefd_file[0]);
close(pipefd_file[1]);
return 0;
}
The result of the operation is as follows:
The difference between tee and splice
Tee is similar to splice and is used for data copying between two fds. The difference is that:
1) Requirements for parameter fd
Splice requires that at least one of the two fds must be a pipeline file;
tee requires both fds to be pipe files.
2) Consumption of fd data
splice is data movement between two fd, splice will consume fd data;
tee is data copy between two fd, tee will not consume fd data.
3) flags parameter
Before Linux2.6.21, SPLICE_F_MOVE has effect on splice, but it has no effect after that. Both SPLICE_F_NONBLOCK and SPLICE_F_MORE have effects on splice;
Only SPLICE_F_NONBLOCK has an effect on tee;
The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge