Introduction to Rsync
rsync (Remote Sync) is an open source fast backup tool that can mirror and synchronize the entire directory tree between different hosts, support incremental backup, and maintain links and permissions.
In a remote synchronization task,the client responsible for initiating rsync synchronization operations is called the initiator, and the server responsible for responding to rsync synchronization operations from the client is called the synchronization source. During the synchronization process, the synchronization source is responsible for providing the original location of the file, and the initiating end should have read permissions to this location.
Rsync is a data mirroring backup tool under Linux systems. The fast incremental backup tool Remote Sync can be used to synchronize remotely and synchronize between different hosts. It can realize full backup and incremental backup, maintain links and permissions, and adopt optimized Synchronization algorithm, compression is performed before transmission, so it is very suitable for applications such as centralized backup or off-site backup. At the same time, Rsync supports local replication or synchronization with other SSH and Rsync hosts.
Rsync backup method
- Initiator: The client responsible for rsync synchronization operation is called the initiator. It notifies the server that I want to back up your data.
- Backup source: The server responsible for responding to rsync synchronization operations from the client is called the backup source, and the server that needs to be backed up
- Server: Run the rsyncd service. Generally speaking, the server needs to be backed up.
- Client: store backup data
Rsync synchronization method
- Full backup: Each backup backs up all files or directories from the backup source to the destination.
- Differential backup: Backs up data that has changed since the last full backup (he targets the last full backup, and does not clear the archive attributes during the backup process)
- Incremental backup: Back up data that has changed since the last backup (it doesn’t care what type of backup it is, just back up the changed data, and it will clear the archive attributes)
Rsync synchronization source
rsync synchronization source: refers to the remote server for backup operations, also called backup source.
For example: Server A synchronizes the data of Server B, and Server B is the backup source – downlink synchronization
The opposite is – upstream synchronization
Rsync common command options
Options | Function |
---|---|
-a | Regulation mode, recursive and retaining object attributes, equivalent to -rlptgoD |
-v | Display synchronization process information |
-z | Compress during transfer |
-H | Keep hard link files |
-A | Keep ACL attribute information |
–delete | Delete the target location Yes, but there are files that are not in the synchronization source |
–checksum | Determine whether to skip the file based on the checksum of the object |
Two expression methods of configuration source
Format 1
Username@host address::shared module name rsync -avz [email protected]::wwwroot /opt/
Format 2
rsync://username@host address/shared module name rsync -avz rsync://[email protected]/wwwroot /opt/
Rsync Experiment
Local copy
Configuring server and client
Install Rsync
systemctl stop firewall systemctl disbale firewalld setenforce 0 yum -y install rsync
Create the /etc/rsyncd.conf configuration file
uid = root gid=root use chroot = yes #imprisoned in the source directory address = 192.168.10.18 #Listening address port 873 #Listening port tcp/udp 873, can be viewed through cat /etc/services | grep rsync log file = /var/log/rsyncd.log #Log file location pid file = /var/run/rsyncd.pid #The file location where the process ID is stored hosts allow = 192.168.10.0/24 #Client address allowed to be accessed dont compress = *.gz *.bz2 *.tgz *.zip *.rar *.z #File types that are no longer compressed during synchronization [wwwroot] #Shared module name path = /var/www/html #The actual path of the source directory comment = Document Root of www.kgc.com read only = yes #Whether it is read-only auth users = backuper #Authorized accounts, multiple accounts separated by spaces secrets file = /etc/rsyncd_users.db #Data file to store account information
Create data files for backup accounts
Ensure that all users have read permissions to the source directory /var/www/html
Normal synchronization completed
Password-free synchronization
Add a specified password file in the middle for password-free login rsync -avz --password-file=/etc/server.passbackupera192.168,154.10: :wwwroot /opt
Usage of –delete option
rsync -avz -delete --password-file=/etc/server.pass [email protected]::wwwroot /abc
Introduction to inotify
Can monitor file system changes and respond to notifications
vim /etc/sysctl.conf (kernel parameter file) max_queue_events #Monitor event queue size max_user_instances #Maximum number of monitoring instances max_user_watches #Maximum number of monitored files per instance #Command tool inotifywait #for continuous monitoring and real-time output results inotifywatch #Used for short-term monitoring, the results will be output after the task is completed
Common command options
Options | Function |
---|---|
-m | Continuous monitoring |
-r | Recursively monitor all child objects |
-q | Simplified output information |
-e | Specify which event types to monitor |
modify | Modify |
create | Create |
move | Move |
delete | Delete |
rsync + inotify implementation configuration
Server side configuration
Modify configuration file
vim /etc/rsyncd.conf read only=no kill (cat /var/run/rsyncd.pid) rsync --daemon netstat -natp |grep rsync
Modify the kernel file (an optimization operation)
Client configuration
Modify configuration file
Write a trigger synchronization script
vim /opt/inotify.sh #!/bin/bash INOTIFY_CMD="inotifywait -mrq -e create,delete,move,modify,attrib /abc" RSYNC_CMD="rsync -azH --delete --password-file=/etc/server.pass /abc/ [email protected]::wwwroot" $INOTIFY_CMD | while read DIRECTORY EVENT FILE do if [ $(pgrep rsync | wc -l) -le 0 ] ; then $RSYNC_CMD fi done #Script explanation First define a variable to monitor changes in information such as creation, deletion, movement, modification, attributes, etc. in this directory. Define another variable for rsync synchronization to synchronize the information in the directory in /abc to the specified directory in the rsync server Write another while loop, first execute the monitoring variable, and then execute the while loop. The content of the loop is to read the directory event file, and if the rsync process synchronization operation does not exist, execute the variables for the synchronization operation.
Client-side verification
Summary
Advantages
- Can be real-time
Disadvantages
When bandwidth is particularly abundant, the synchronization rate of rsync is slower than scp.
To synchronize millions of data, parameters must be written strictly as required, otherwise synchronization may fail.