NFS Network File System
- Function: Users can use files on remote systems in the network just like accessing their own local file system
- Principle: User process–>RPC service (portman)–>tcp/ip protocol stack–>After remote host RPC service–>Remote NFS service–>Remote host local disk data–>NFS service –>tcp/ip protocol stack–>Local data
- Advantages: It is easy to handle problems such as sission and data cannot be found during data search when the front-end server agent is offloaded, and at the same time, even if any web server is down, data access will not be affected.
- Disadvantages: Single point of failure. All data is on the same physical host. Once the physical host goes down, the data stored in the entire project will be lost and inaccessible. The solution: do real-time incremental backup of data based on inotify + rsync to deal with this problem.
NFS build (based on the startup file /etc/fatab or automatically mount autofs)
- Packages: nfs-utils, rpcbind, tcp_wrappers
- The linux kernel needs to support nfs.ko
- Log: /var/lib/nfs/
- Configuration file: /etc/exports ,/etc/exports.d/*.exports
- Start the service: systemctl start nfs-server
File format:
- Shared file host: 192.168.47.117
host: Host 1 (opt1, opt2) Host 2 (opt1, opt2) Single host: # ipv4, ipv6, FQDN IP networks: # Both mask formats are supported 172.18.0.0/255.255.0.0 172.18.0.0/16 wildcards: # Host name wildcard, such as *.xiehegang.online, IP is not allowed netgroups: # Host group of NIS domain, @group_name anonymous: # means using * to wildcard all clients #------------------------------------------------ -------------------------------------------------- -------------------------- option: (ro,sync,root_squash,no_all_squash) ro,rw # read-only and read-write async # Asynchronous, no writing to disk immediately after data changes, high performance sync # (default after 1.0.0) Synchronization, data is written to the shared storage disk immediately upon request root_squash # (Default) remote root mapping is nfsnobody, UID is 65534, CentOS8 is nobody, earlier version is 4294967294 (nfsnobody) no_root_squash #Remote root is mapped to root user all_squash # All remote users (including root) become nfsnobody, CentOS8 is nobody no_all_squash # (Default) Keep UID and GID of shared files anonuid and anongid # Indicate that anonymous users are mapped to specific user UID and group GID instead of nfsnobody, which can be used with all_squash #------------------------------------------------ -------------------------------------------------- -------------------------- # example /data/app1/ 192.168.47.106(ro,sync,no_all_squash) /data/app2 192.168.47.106(rw,no_rootsquash,all_squash)
- Mount host 192.168.47.106
# Check network sharing status showmount -e 192.168.47.117 #Create a mount point and mount it mkdir /data/webdata/ -pv; mount 192.168.47.117:/data/app1 /data/webdata # Complete the mounting of the file and create the file, which can be seen on another host. However, the App2 mount point cannot create the file because the configuration file of 47.117 is set to ro #------------------------------------------------ -------------------------------------------------- -------------------------- # Implement automatic mounting under /etc/fatab to prevent mount loss caused by power outage. **Add** the following content 192.168.47.117:/data/app1 /data/webdata nfs _netdev 0 0 #------------------------------------------------ -------------------------------------------------- -------------------------- # If necessary, you can also set up real-time mounting just like accessing ls/msic/cd to access a CD, that is: access and mount (aotofs) # Install autofs yum install -y autofs # Edit configuration file vi /etc/auto.misc and append content nfs -fstype=nfs 192.168.47.117:/data/app1 # Start autofs service systemctl start autofs #Access directory: cd /msic/app1
Incremental backup based on inotify + rsync
- Backup NFS host: 192.168.47.129
- linux kernel version: greater than 2.6.13
# Modify kernel parameters: vim /etc/sysctl.conf fs.inotify.max_queued_events=66666 fs.inotify.max_user_watches=100000 # Reapply sysctl-p # Check the modification status cat /proc/sys/fs/inotify/*
inotify-tools tool
- inotify-tools installation tool: yum install inotify-tools (yum -y install epel-release if the epel source is not installed)
- inotifywait command options:
-m, --monitor #Always keep event monitoring -d, --daemon # Execute in daemon mode, similar to -m, used with -o -r, --recursive # Recursively monitor changes in directory data information -q, --quiet # Output a small amount of event information --exclude #Specify to exclude files or directories, using extended regular expression matching patterns. --excludei # Similar to exclude, not case sensitive -o, --outfile # Print events to a file, which is equivalent to standard correct output. Note: use absolute paths. -s, --syslogOutput # Send errors to syslog equivalent to standard error output --timefmt #Specify time output format --format #Specified output format; that is, the actual monitoring output content -e #Specify to monitor the specified event. If omitted, it means all events will be monitored. # --timefmt format------------------------------------------------ -------------------------------------------------- -------------------------- %Y # Year information, including century information %y # Year information, excluding century information %m # Display month, range 01-12 %d # The day of the month, the range is 01-31 %H # Hour information, using 24-hour format, range 00-23 %M # minutes, range 00-59 Example: --timefmt "%Y-%m-%d %H:%M" # --format format------------------------------------------------ -------------------------------------------------- -------------------------- %T # Output the time format information defined in the time format, specify the time information through the --timefmt option syntax format %w # When the event occurs, monitor the name information of the file or directory %f # When an event occurs, the file or directory information that triggered the event in the monitoring directory will be displayed, otherwise it will be empty. %e # Display event information that occurred. Different events are separated by commas by default. %Xe # Display event information that occurred. Different event specifications are separated by X. Example: --format "%T %w%f event: %;e" # -e format------------------------------------------------- -------------------------------------------------- ------------------- create #File or directory creation delete # The file or directory is deleted modify # The file or directory content is written attrib # File or directory attribute changes close_write # The file or directory is closed after the write mode is opened. close_nowrite # The file or directory is closed after opening in read-only mode. close # The file or directory is closed, regardless of read or write mode open # The file or directory is opened moved_to # The file or directory is moved to the monitored directory moved_from #The file or directory is moved from the monitored directory move #Events will be triggered regardless of whether a file or directory is moved to or out of the monitoring directory. access # File or directory content is read delete_self #The file or directory is deleted, and the directory itself is deleted unmount # Unmount Example: -e create,delete,moved_to,close_write, attrib
inotify complete usage example
# Monitor one-time events inotifywait/data #Continuous foreground monitoring inotifywait -mrq /data #Continuous background monitoring and logging inotifywait -o /root/inotify.log -drq /data --timefmt "%Y-%m-%d %H:%M" --format "%T %w%f event: %e" #Continuously monitor specific events in the foreground inotifywait -mrq /data --timefmt "%F %H:%M" --format "%T %w%f event: %;e" -e create,delete,moved_to,close_write,attrib
rsync realize incremental backup
- Package rsync
- Service file:/usr/lib/systemd/system/rsyncd.service
- Configuration file: /etc/rsyncd.conf
- Port: 873/tcp
- Format: rsync [OPTION…] SRC… [DEST]
rsync has three working modes: 1. Implement synchronization on the local file system. The command line syntax format is the format of the "Local" section mentioned above. 2. The local host uses the remote shell to communicate with the remote host. The command line syntax format is that of the "Access via remote shell" section above. Mode. 3. The local host connects to the rsync daemon on the remote host through a network socket. The command line syntax format is the above "Access via rsync daemon" section format.
This example uses independent mode to run rsync to achieve synchronization
- Run rsync as a standalone service (192.168.47.117)
# Installation yum -y install rsync #Create the configuration file of the rsync server vi /etc/rsyncd.conf uid=root gid=root use chroot=no max connections = 0 ignore errors exclude = lost + found/ log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid lock file = /var/run/rsyncd.lock reverse lookup = no hosts allow = 192.168.47.0/24 [backup] path = /backup/ comment=backup read only=no authusers=rsyncuser secrets file = /etc/rsync.pass # Server-side preparation directory cd /; mkdir /backup # Generate verification file on the server side echo "rsyncuser:justin" > /etc/rsync.pass chmod 600 /etc/rsync.pass #Start service /usr/bin/rsync --daemon ,rsync --daemon
- Backup server 192.168.47.129
#Client configuration password file echo "justin" > /etc/rsync.pass chmod 600 /etc/rsync.pass #Client test synchronization data rsync -avz --delete --password-file=/etc/rsync.pass /data/app1 [email protected]::/backup
Create a script file to implement real-time pull based on inotify + rsync
vim /etc/profile.d/bf.sh #!/bin/bash SRC='/data/app' DEST='[email protected]::backup' inotifywait -mrq --timefmt '%Y-%m-%d %H:%M' --format '%T %w %f' -e create,delete,moved_to,close_write,attrib ${SRC} |while read DATE TIME DIR FILE;do FILEPATH=${DIR}${FILE} rsync -az --delete --password-file=/etc/rsync.pass $SRC $DEST & amp; & amp; echo "At ${TIME} on ${DATE}, file $FILEPATH was backed up up via rsync" >> /var/log/changelist.log done ./etc/profile.d/bf.sh
Final effect
The actual location of the file /data/webdata of server 47.106 is /data/app1 of 47.117,
Server 47.129 backs up the data of 47.117 /data/app1 to the local /backup folder
Of course, this is my own experiment, and there may be errors in the process. If you find any, please comment, thank you ~