Redis persistence files are divided into two types:
dump.rdb and appendonly.aof
1. Redis persistence RDB
Redis provides 2 different forms of persistence methods
RDB (Redis DataBase)
AOF (Append Of File)
1. What is RDB
Write a snapshot of the data set in memory to disk within a specified time interval, which is also known as Snapshot in jargon. When it is restored, the snapshot file is read directly into memory.
2. How backups are performed
Redis will create (fork) a child process separately for persistence. It will first write the data to a temporary file. After the persistence process is completed, this temporary file will be used to replace the last persisted file. During the entire process, the main process does not perform any IO operations, which ensures extremely high performance. If large-scale data recovery is required and the integrity of data recovery is not very sensitive, the RDB method is more efficient than the AOF method. of high efficiency. The disadvantage of RDB is that the data after the last persistence may be lost.
3.Fork
The function of Fork is to copy a process that is the same as the current process. All the data (variables, environment variables, program counter, etc.) of the new process have the same values as the original process, but it is a completely new process and serves as a child process of the original process.
4. RDB persistence process
5. dump.rdb file
Configure the file name in redis.conf, the default is dump.rdb
Wherever the file is started, the file is located
6. Configuration location
The saving path of the rdb file can also be modified. The default is the directory where the command line is located when Redis is started.
redis.conf:
7. How to trigger RDB snapshot; retention policy
7.1 Default snapshot configuration interval in configuration file
7.2 Command save bgsave
save: When saving, just save, ignore other things, and block everything. Save manually. Not recommended.
bgsave: Redis will perform snapshot operations asynchronously in the background, and the snapshot can also respond to client requests.
You can use the lastsave command to obtain the time of the last successful snapshot execution.
7.3 flushall command
Executing the flushall command will also generate the dump.rdb file, but it is empty and meaningless.
7.4 SNAPSHOTTING
7.5. Save
Format: save seconds number of write operations
RDB is a compressed Snapshot of the entire memory. The data structure of RDB can be configured with compound snapshot trigger conditions.
The default is 10,000 changes in 1 minute, 10 changes in 5 minutes, or 1 change in 15 minutes.
Disable
Do not set the save command, or pass an empty string to save
7.6. stop-writes-on-bgsave-error
When Redis cannot write to the disk (the disk is full), directly turn off the writing operation of Redis. Recommend yes.
7.7 rdbcompression compressed files
For snapshots stored on disk, you can set whether to compress them for storage. If so, redis will perform compression.
If you don’t want to consume CPU for compression, you can set this feature to off. Recommend yes.
7.8. rdbchecksum checks data integrity
After storing the snapshot, you can also let redis perform data verification. If the data has been damaged, there is no need to perform persistence operations. Doing so will increase performance consumption by about 10%. If you want to get the maximum performance improvement, you can Turn off this feature
Recommend yes.
7.9. RDB backup
First query the directory of the rdb file
Create backup file mkdir dump-bak.rdb
Copy the backup files to the working directory cp dump-bak.rdb dump.rdb
Close Redis
Then remove dump.rdb mv -rf dump.rdb
rdb recovery
Enter myredis and delete the original dump.rdb
Then copy dump-bak.rdb cp dump-bak.rdb dump.rdb
Start Redis and the backup data will be loaded directly.
Open client
7.10. Advantages
Suitable for large-scale data recovery
It is more suitable to use if the requirements for data integrity and consistency are not high.
Save disk space
Fast recovery
7.11 Disadvantages
When forking, the data in the memory is cloned, and roughly 2 times the expansion needs to be considered.
Although Redis uses copy-on-write technology when forking, it still consumes performance if the data is large.
A backup is made at a certain interval during the backup cycle, so if Redis goes down unexpectedly, all modifications after the last snapshot will be lost.
Two Redis persistence AOF
2.1. AOF (Append Only File)
2.1.1. What is
Record each write operation in the form of log (incremental save), record all write instructions executed by Redis (read operations are not recorded), Only files can be appended but not rewritten. When redis starts, it will read the file and reconstruct the data. In other words, when redis restarts, it will execute the write instructions from front to back according to the contents of the log file to complete the data. Return to work
2.1.2. AOF persistence process (memorize)
(1) The client’s requested write command will be appended to the AOF buffer;
(2) The AOF buffer synchronizes operations to the AOF file on the disk according to the AOF persistence policy [always, everysec, no];
(3) When the AOF file size exceeds the rewrite policy or is manually rewritten, the AOF file will be rewritten to compress the AOF file capacity;
(4) When the Redis service is restarted, the write operations in the AOF file will be reloaded to achieve data recovery;
2.1.3. AOF is not enabled by default ——–》 Change it to enabled status
The file name can be configured in redis.conf, the default is appendonly.aof
The saving path of the AOF file is the same as the path of the RDB.
aof saved policy (if the computer performance is good, change it to) Always open
Cannot select no
2.1.4. AOF and RDB are enabled at the same time, who does redis listen to?
AOF and RDB are enabled at the same time, and the system takes the AOF data by default (data will not be lost)
2.1.5. AOF Repair/Restore
If you encounter AOF file corruption, use
/usr/redis/bin/redis-check-aof –fix The location of the file /appendonly.aof Perform recovery
Back up corrupted AOF files
Recovery: restart redis and reload
./redis-check-aof –fix ./myredis/appendonly.aof (location of aof file)
Rewrite file size
The first time: 64m rewrite ->50m 50 + 50*100%=100
Second time: 100m
Third time: 200m
Trigger mechanism, when to rewrite
Redis will record the AOF size when it was last rewritten. The default configuration is to trigger when the AOF file size is twice the size after the last rewrite and the file is larger than 64M.
Although rewriting can save a lot of disk space and reduce recovery time. However, each rewrite still carries a certain burden, so Redis must meet certain conditions before it can be rewritten.
auto-aof-rewrite-percentage: Set the baseline value for rewriting, and start rewriting when the file reaches 100% (triggered when the file is twice the size of the original rewritten file)
->40m 80
auto-aof-rewrite-min-size: Set the baseline value for rewriting, the minimum file is 64MB. Rewriting starts when this value is reached.
3. Rewriting process (back)
(1) bgrewriteaof triggers rewriting and determines whether bgsave or bgrewriteaof is currently running. If so, wait for the command to end before continuing execution.
(2) The main process forks out the child process to perform the rewrite operation to ensure that the main process will not be blocked.
(3) The child process traverses the data in the redis memory to the temporary file. The client’s write request is written to the aof_buf buffer and the aof_rewrite_buf rewrite buffer at the same time to ensure that the original AOF file is complete and the new data modification actions during the generation of the new AOF file will not be lost. .
(4) 1). After the child process finishes writing the new AOF file, it sends a signal to the main process, and the parent process updates the statistical information. 2). The main process writes the data in aof_rewrite_buf to the new AOF file.
(5) Use the new AOF file to overwrite the old AOF file to complete AOF rewriting.
3.1. Advantages
3.2. Disadvantages
- Takes up more disk space than RDB.
- Restoring backups is slower.
- If each read and write is synchronized, there will be a certain performance pressure.
- There are individual bugs that prevent recovery.
3.3. Which one is better?
It is officially recommended to enable both.
If the data is not sensitive, you can choose to use RDB alone.
It is not recommended to use AOF alone because bugs may occur.
If you are just doing pure memory caching, you don’t need to use it at all.
Three master-slave replication
3.1. What is
After the host data is updated, it is automatically synchronized to the master/slaver mechanism of the standby machine according to the configuration and policy. Master is mainly for writing, and Slave is mainly for reading
3.2. What can you do
Read and write separation, performance expansion (master write slave read)
Disaster recovery and rapid recovery
3.3 Master-slave replication
One master, two servants
6380 main
6381 from
6382 from
Copy multiple redis.conf files include (write absolute path)
enable daemonize yes
Pid file name pidfile
Specify port port
Log file name
dump.rdbnamedbfilename
Appendonly changed to no
Note: Password cannot be set
Host configuration file redis.conf
include /usr/redis/bin/redis.conf port 6380 dbfilename dump_6380.rdb appendfilename "appendonly_6380.aof" pidfile /var/run/redis_6380.pid daemonize yes
View the operation status of three servers
Connect client: ./redis-cli -p 6380 (6381/6382)
Check the running status: info replication
All three are hosts
With slaves
The host does not need to be configured
6380 master 81 82 slave
slaveof
Become a slave server for an instance
1. Execute on 6381 and 6382: slaveof 127.0.0.1 6380
host data
Slave data
2. Write on the host and read data on the slave
3. If the host hangs up, just restart it and everything will be as before.
4. The slave needs to be reset after restarting: slaveof 127.0.0.1 6379
(If the host machine is down, just restart and everything will be restored. If the slave machine is down, reset the corresponding host)
Copying Principle
- After the slave starts and successfully connects to the master, it will send a sync command.
- The master receives the command to start the background save process and collects all received commands for modifying the data set. After the background process is completed, the master will transfer the entire data file to the slave to complete a complete synchronization.
- Full copy: After receiving the database file data, the slave service saves it and loads it into memory.
- Incremental replication: Master continues to pass all new collected modification commands to slave in order to complete synchronization.
- But as long as the master is reconnected, a full synchronization (full replication) will be automatically performed.
Passing on the Fire
The previous slave can be the master of the next slave, and the slave can also receive connection and synchronization requests from other slaves. Then the slave acts as the next master in the chain, which can effectively reduce the write pressure of the master and reduce risks through decentralization.
Use slaveof
Midway change direction: the previous data will be cleared and the latest copy will be re-created.
The risk is that once a slave goes down, subsequent slaves cannot be backed up.
The host is down, and the slave is still a slave and cannot write data.
Focus on customers
When a master goes down, the following slave can be promoted to master immediately without any modifications to the following slave.
Use slaveof no one to change from slave to master.
Manual version
Sentinel mode (sentinel)
Automatic version that is mainly anti-customer can monitor whether the host is faulty in the background. If it fails, it will automatically switch from the slave database to the main database based on the number of votes.
Adjusted to one master and two servants mode, 6379 brings 6380 and 6381
Create a new sentinel.conf file in the customized /myredis directory. The name must not be wrong.
80
Configure Sentinel and fill in the content
sentinel monitor mymaster 127.0.0.1 6380 1
Among them, mymaster is the server name of the monitoring object, and 1 is the minimum number of sentinels that agree to migrate.
Start Sentinel
Execute ./redis-sentinel sentinel.conf
sentinel.conf after startup
If the test is successful, shut down the host 6380 and generate a new host from the slave election
(You can see the Sentinel window log in about 10 seconds, switching to a new host)
Which slave has the chance to be elected as the master? According to priority: replica-priority
- replica-priority: 100 The smaller the value, the higher the priority.
- Offset: The data obtained from the host is relatively complete
- Runid: 40 small uuid automatically generated by redis
The original host will become a slave after restarting.
Leadership (Master)
Reserve cadre employees
The default priority in redis.conf is replica-priority 100. The smaller the value, the higher the priority.
The offset refers to the most complete data obtained from the original host.
After each redis instance is started, a 40-digit runid will be randomly generated.
The knowledge points of the article match the official knowledge files, and you can further learn related knowledge. MySQL entry-level skills treeOperation and maintenance and architectureFrom master-slave replication to chain replication 76766 people are learning the system