Table of Contents
1. Introduction to Rsync
1. What is rsync?
2. Backup method
3. rsync synchronization method
4. Commonly used rsync commands
5. Two expression methods of configuration source
2. rsync experiment
1. Local copy
?Edit?Edit
2. Off-site replication
2.1 rsync server configuration
2.2 rsync client configuration
2.2.1 Normal synchronization
2.2.2 Password-free synchronization
2.2.3 Usage of –delete option
3. Initiator (client) configuration rsync + inotify
3.1 Server configuration
3.2 Initiator configuration
3.3 Client verification
1. Introduction to Rsync
1. What is rsync
- An open source fast backup tool that can mirror and synchronize the entire directory tree between different hosts and support incremental backup
- Compression performed before transfer makes it ideal for off-site backups, mirror servers, etc.
- The client responsible forinitiating an rsync synchronization operation is called the initiator, and the server responsible forresponding to the rsync synchronization operation from the client is called the synchronization source. During the sync process, the sync source is responsible for providing the original location of the file to which the initiating end should have read access.
- Official website: https://rsync.samba.org/
2. Backup method
1. Initiator: The client responsible for rsync synchronization operation is called the initiator. It notifies the server that I want to back up your data.
2. Backup source: The server responsible for responding to the rsync synchronization operation from the client is called the backup source, and the server that needs to be backed up
3. Server: Run the rsyncd service. Generally speaking, the server needs to be backed up.
4. Client: Store backup data
The principle of rsync copy
- First, compare the source file and the file in the target location to find the differences.
- Consistency synchronization is performed based on the differences between the files in the target location and the source files.
3. rsync synchronization method
①Local copy: First, compare the source file and the destination file to find the differences, and then synchronize the differences between the source file and the destination location consistently.
②Download synchronization: The client and server synchronize content, which can be understood as the client downloading content from the server.
③Upstream synchronization: The server and the client synchronize content, which can be understood as the client uploading content to the server.
4. Commonly used rsync commands
Basic format: rsync [options] original location target location
Common options:
- -v: Display detailed (verbose) information about the synchronization process.
- -z: Compress when transferring files.
- -a: Archive mode, retains file permissions, attributes and other information, equivalent to the combination option “-rlptgoD”.
- –delete: Delete files that exist in the target location but not in the original location.
- -r: Recursive mode, including all files in the directory and subdirectories.
- -l: Symbolic link files are still copied as symbolic link files.
- -p: Preserves the file’s permission flags.
- -t: Keep the timestamp of the file.
- -g: Preserve the file’s group tag (only for superusers).
- -o: Keep the file’s owner mark (for superuser use only).
- -H: Keep hard link files.
- -A: Preserve ACL attribute information.
- -D: Preserve device files and other special files.
- –checksum: Determine whether to skip files based on checksum (rather than file size or modification time).
5. Two expression methods of configuration source
#Format 1: #Username@host address::shared module name rsync -avz [email protected]::wwwroot /opt/ #Format 2: #rsync://username@host address/shared module name rsync -avz rsync://[email protected]/wwwroot /opt/
2. rsync experiment
1. Local copy
rsync -avz /abc /opt #Copy the entire abc directory rsync -avz /abc/ /opt #Copy the files in the abc directory
2. Off-site replication
2.1 rsync server configuration
Experimental environment:
192.168.247.20 Server
192.168.247.80 Client
# Prerequisite: First turn off the firewall and enhanced functions systemctl stop firewalld setenforce 0 rpm -q rsync #General systems have rsync installed by default #Create the /etc/rsyncd.conf configuration file vim /etc/rsyncd.conf #Add the following configuration items uid=root gid=root use chroot = yes #imprisoned in the source directory address = 192.168.10.18 #Listening address port 873 #Listening port tcp/udp 873, which can be viewed through cat /etc/services | grep rsync log file = /var/log/rsyncd.log #Log file location pid file = /var/run/rsyncd.pid #The file location where the process ID is stored hosts allow = 192.168.10.0/24 #Client address allowed to be accessed dont compress = *.gz *.bz2 *.tgz *.zip *.rar *.z #File types that are no longer compressed during synchronization [wwwroot] #Shared module name path = /var/www/html #The actual path of the source directory comment = Document Root of www.kgc.com read only = yes #Whether it is read-only auth users = backuper #Authorized accounts, multiple accounts separated by spaces secrets file = /etc/rsyncd_users.db #Data file to store account information #If you use anonymous mode, just remove the "auth users" and "secrets file" configuration items. #Create data files for backup accounts vim /etc/rsyncd_users.db backuper:123456 #No need to create a system user with the same name chmod 600 /etc/rsyncd_users.db #Start rsync service program rsync --daemon #Start the rsync service and run it as an independent listening service (daemon process)
Create shared files, also known as source files
mkdir -p /var/www/html chmod + r /var/www/html cd /var/www/html echo "yudeqiang">ydq.txt echo "zhouxingchi">zxc.txt
2.2 rsync client configuration
2.2.1 Normal synchronization
2.2.2 Password-free synchronization
vim /etc/server.pass 123 chmod 600 /etc/server.pass rsync -avz --password-file=/etc/server.pass [email protected]::wwwroot /abc
2.2.3 Usage of –delete option
rsync -avz -delete --password-file=/etc/server.pass [email protected]::wwwroot /abc
3. Initiator (client) configuration rsync + inotify
Combining the inotify mechanism with the rsync tool can achieve triggered backup (real-time synchronization), that is, as long as the document in the original location changes, the incremental backup operation will be started immediately; otherwise, it will be in a silent waiting state. In this way, problems such as delays and excessive cycles that exist when backing up on a fixed cycle are avoided.
Because the inotify notification mechanism is provided by the Linux kernel,it is mainly used for local monitoring and is more suitable for upstream synchronization when applied in triggered backup.
3.1 Server Configuration
1. Modify the rsync source server configuration file
vim /etc/rsyncd.conf ... read only = no #Turn off read-only, upstream synchronization needs to be writeable kill $(cat /var/run/rsyncd.pid) rsync --daemon netstat -anpt | grep rsync chmod 777 /var/www/html/
2. Adjust inotify kernel parameters
#max_queue_events (monitoring event queue, default value is 16384), #max_user_instances (maximum number of monitoring instances, default value is 128), #max_user_watches (maximum number of monitored files per instance, default value is 8192). When the number of directories and files to be monitored is large or changes frequently, it is recommended to increase the values of these three parameters. #cat /proc/sys/fs/inotify/max_queued_events #cat /proc/sys/fs/inotify/max_user_instances #cat /proc/sys/fs/inotify/max_user_watches vim /etc/sysctl.conf fs.inotify.max_queued_events = 16384 fs.inotify.max_user_instances = 1024 fs.inotify.max_user_watches = 1048576 sysctl -p
3.2 Initiator Configuration
Install inotif installation package
Write a trigger sync script
The purpose of the script is to provide real-time monitoring and real-time synchronization
vim /opt/inotify.sh #!/bin/bash INOTIFY_CMD="inotifywait -mrq -e create,delete,move,modify,attrib /abc" RSYNC_CMD="rsync -azH --delete --password-file=/etc/server.pass /abc/ [email protected]::wwwroot" $INOTIFY_CMD | while read DIRECTORY EVENT FILE do if [ $(pgrep rsync | wc -l) -le 0 ] ; then $RSYNC_CMD fi done #Script explanation First define a variable to monitor changes in information such as creation, deletion, movement, modification, attributes, etc. in this directory. Define another variable for rsync synchronization to synchronize the information in the directory in /abc to the specified directory in the rsync server Write another while loop, first execute the monitoring variable, and then execute the while loop. The content of the loop is to read the directory event file, and if the rsync process synchronization operation does not exist, execute the variables for the synchronization operation.
3.3 Client Authentication
