1. RDB for redis persistence
Redis provides 2 different forms of persistence.
RDB (Redis DataBase)
AOF (Append Of File)
1.What is RDB
Write a snapshot of the data set in the memory to the disk within a specified time interval, which is also called a Snapshot in the jargon. When it is restored, the snapshot file is read directly into the memory.
2. How backup is performed
Redis will create (fork) a child process separately for persistence. It will first write the data to a temporary file. After the persistence process is completed, this temporary file will be used to replace the last persisted file. During the entire process, the main process does not perform any IO operations, which ensures extremely high performance. If large-scale data recovery is required and the integrity of data recovery is not very sensitive, the RDB method is more efficient than the AOF method. of high efficiency. The disadvantage of RDB is that data after the last persistence may be lost.
3.Fork
The function of Fork is to copy a process that is the same as the current process. All the data (variables, environment variables, program counter, etc.) of the new process have the same values as the original process, but it is a completely new process and serves as a child process of the original process.
The function of Fork is to copy a process that is the same as the current process. All the data (variables, environment variables, program counter, etc.) of the new process have the same values as the original process, but it is a completely new process and serves as a child process of the original process.
Generally, the parent process and the child process will share the same physical memory. Only when the content of each segment of the process space changes, the content of the parent process will be copied. to the child process.
4.RDB persistence process
5.dump.rdb file
Configure the file name in redis.comf, the default is dump.rdb
6. Configuration location
The saving path of the rdb file can also be modified. The default is the directory where the command line is located when Redis is started.
redis.conf:
dir “/myredis/
7. How to trigger RDB snapshot; retention strategy
Default snapshot configuration interval in 7.1 configuration file
7.2 Command save bgsave
save: When saving, just save, ignore other things, and block everything. Save manually. Not recommended.
bgsave: Redis will perform snapshot operations asynchronously in the background, and the snapshot can also respond to client requests.
You can use the lastsave command to obtain the time of the last successful snapshot execution.
7.3 flushall command
Executing the flushall command will also generate the dump.rdb file, but it is empty and meaningless.
7.4 stop writes on bgsave error
When Redis cannot write to the disk (the disk is full), directly turn off the writing operation of Redis. Recommend yes
7.5rdbcompression compressed file
For snapshots stored on disk, you can set whether to compress them for storage. If so, redis will perform compression.
If you don’t want to consume CPU for compression, you can set this feature to off. Recommend yes
7.6rdbchecksum checks data integrity
After storing the snapshot, you can also let redis perform data verification. If the data has been damaged, there is no need to perform persistence operations. Doing so will increase performance consumption by about 10%. If you want to get the maximum performance improvement, you can Turn off this feature
Recommend yes.
7.7 Advantages
Suitable for large-scale data recovery
It is more suitable to use if the requirements for data integrity and consistency are not high.
Save disk space
Fast recovery
7.8 Disadvantages
When forking, the data in the memory is cloned, and roughly 2 times the expansion needs to be considered.
Although Redis uses copy-on-write technology when forking, it still consumes performance if the data is large.
A backup is made at a certain interval during the backup cycle, so if Redis goes down unexpectedly, all modifications after the last snapshot will be lost.
2. Redis persistence AOF
2.1 AOF
2.1.1 What is AOF
Record each write operation (incremental save) in the form of log, and record all write instructions executed by Redis (Read operations do not Record), Only files can be appended but files cannot be rewritten. When redis starts, it will read the file and reconstruct the data. In other words, When redis is restarted, the write instructions will be executed from front to back according to the contents of the log file to complete the data recovery work.
2.1.2 AOF persistence process
(1) The client’s requested write command will be appended to the AOF buffer;
(2) The AOF buffer synchronizes operations to the AOF file on the disk according to the AOF persistence policy [always, everysec, no];
(3) When the AOF file size exceeds the rewrite policy or is manually rewritten, the AOF file will be rewritten to compress the AOF file capacity;
(4) When the Redis service is restarted, the write operations in the AOF file will be reloaded to achieve data recovery.
2.1.3 AOF is not enabled by default
The file name can be configured in redis.conf, the default is appendonly.aof
The saving path of the AOF file is the same as the path of the RDB.
2.1.4 AOF and RDB are enabled at the same time, who does redis listen to
AOF and RDB are enabled at the same time, and the system takes the AOF data by default (data will not be lost)
2.1.5 AOF startup/repair/recovery
Although the backup mechanism and performance of AOF are different from RDB, the backup and recovery operations are the same as RDB. They copy the backup file. When recovery is needed, copy it to the Redis working directory and load it when the system is started.
normal recovery
Modify the default appendonly no to yes
Copy the aof file with data and save it to the corresponding directory
Recovery: restart redis and reload
Exception recovery
Modify the default appendonly no to yes
If you encounter AOF file corruption, use
/usr/redis/bin/redis-check-aof –fix File location/appendonly.aofRestore
Back up corrupted AOF files
Recovery: restart redis and reload
2.1.6 AOF synchronization frequency setting
appendfsync always
Always synchronized, every Redis write will be recorded in the log immediately; performance is poor but data integrity is better
appendfsync everysec
Synchronize every second and log once every second. If there is a downtime, the data of this second may be lost.
appendfsync no
Redis does not actively synchronize and leaves the synchronization timing to the operating system.
2.1.7 Rewrite compression
What is 1:
AOF uses file appending, and the file will become larger and larger. To avoid this situation, a new rewriting mechanism is added. When the size of the AOF file exceeds the set threshold, Redis will start the content compression of the AOF file. Only Keep the minimum set of instructions that can recover data. You can use the command bgrewriteaof
2 Principles of rewriting, how to implement rewriting
When the AOF file continues to grow and becomes too large, a new process will be forked to rewrite the file (the temporary file is also written first and then renamed). Rewriting after the redis4.0 version refers to taking a snapshot of the rdb. The secondary form is attached to the new AOF header as existing historical data, replacing the original running account operation.
no-appendfsync-on-rewrite:
If no-appendfsync-on-rewrite=yes, the AOF file is not written and only the cache is written. User requests will not be blocked, but if there is a downtime during this period, the cache data during this period will be lost. (Reduce data security, improve performance)
If no-appendfsync-on-rewrite=no, the data will still be flushed to the disk, but it may be blocked when encountering a rewrite operation. (Data is safe, but performance is reduced)
Trigger mechanism, when to rewrite
Redis will record the AOF size when it was last rewritten. The default configuration is to trigger when the AOF file size is twice the size after the last rewrite and the file is larger than 64M.
Although rewriting can save a lot of disk space and reduce recovery time. However, each rewrite still carries a certain burden, so Redis must meet certain conditions before it can be rewritten.
auto-aof-rewrite-percentage: Set the baseline value for rewriting, and start rewriting when the file reaches 100% (triggered when the file is twice the size of the original rewritten file)
->40m 80
auto-aof-rewrite-min-size: Set the baseline value for rewriting, the minimum file is 64MB. Rewriting starts when this value is reached.
For example: rewriting starts when the file reaches 70MB, and when it drops to 50MB, when will it start rewriting next time? 100MB
When the system is loaded or when the last rewrite is completed, Redis will record the AOF size at this time and set it to base_size.
If the current size of Redis’s AOF >= base_size + base_size*100% (default) and the current size >= 64mb (default), Redis will rewrite the AOF.
3. Rewriting process (back)
(1) bgrewriteaof triggers rewriting and determines whether bgsave or bgrewriteaof is currently running. If so, wait for the command to end before continuing execution.
(2) The main process forks out the child process to perform the rewrite operation to ensure that the main process will not be blocked.
(3) The child process traverses the data in the redis memory to the temporary file. The client’s write request is written to the aof_buf buffer and the aof_rewrite_buf rewrite buffer at the same time to ensure that the original AOF file is complete and the new data modification actions during the generation of the new AOF file will not be lost. .
(4) 1). After the child process finishes writing the new AOF file, it sends a signal to the main process, and the parent process updates the statistical information. 2). The main process writes the data in aof_rewrite_buf to the new AOF file.
(5) Use the new AOF file to overwrite the old AOF file to complete AOF rewriting.
2.2 Advantages
The backup mechanism is more robust and the probability of data loss is lower.
Readable log text, robust by operating AOF, can handle misoperations
2.3 Disadvantages
- Take up more disk space than RDB.
- Restoring backups is slower.
- If every read and write is synchronized, there will be a certain performance pressure.
There are some bugs that make recovery impossible.
2.4 Which one is better
It is officially recommended to enable both.
If the data is not sensitive, you can choose to use RDB alone.
It is not recommended to use AOF alone because bugs may occur.
If you are just doing pure memory caching, you don’t need to use it at all.
2.5 official recommendation
The RDB persistence method can snapshot and store your data at specified time intervals.
AOF persistence mode records each write operation to the server. When the server restarts, these commands will be re-executed to restore the original data. The AOF command uses the redis protocol to append and save each write operation to the end of the file.
Redis can also rewrite AOF files in the background to prevent the size of AOF files from being too large.
Caching only: If you only want your data to exist when the server is running, you can also not use any persistence method.
Enable two persistence methods at the same time
In this case, when redis restarts, the AOF file will be loaded first to restore the original data, because usually the data set saved by the AOF file is more complete than the data set saved by the RDB file.
RDB data is not real-time, and when the server is restarted when both are used at the same time, only AOF files will be found. So should we just use AOF?
It is not recommended, because RDB is more suitable for backing up the database (the AOF is constantly changing and it is difficult to back up), restarting quickly, and there will be no potential bugs in the AOF, so it is reserved as a precautionary measure.
3. Master-slave replication
3.1 What is master-slave replication
After the host data is updated, it is automatically synchronized to the master/slaver mechanism of the standby machine according to the configuration and policy. Master is mainly for writing, and Slave is mainly for reading
What can 3.2 do
Read and write separation, performance expansion (master write slave read)
Disaster recovery and rapid recovery
3.3 Master-slave replication
One master, two servants
Copy multiple redis.conf files include (write absolute path)
enable daemonize yes
Pid file name pidfile
Specify port port
Log file name
dump.rdbnamedbfilename
Appendonly changed to no
Note: Password cannot be set
Copy the following into a new file
redis_6381.conf include /usr/java/myredis/redis.conf pidfile /var/run/redis_6381.pid port 6381 dbfilename dump6381.rdb
Start redis server
/usr/java/redis/bin/redis-server /usr/java/myredis/redis6379.conf /usr/java/redis/bin/redis-server /usr/java/myredis/redis6380.conf /usr/java/redis/bin/redis-server /usr/java/myredis/redis6381.conf
View the operation status of three servers
Connect client: ./redis-cli -p 6379
Check the running status: info replication
Become a slave server for an instance
1. Execute on 6380 and 6381: slaveof 127.0.0.1 6379
2. Write on the host and read data on the slave
3. If the host hangs up, just restart it and everything will be as before.
4. The slave needs to be reset after restarting: slaveof 127.0.0.1 6379
Copying principle
- Slave will send a sync command after it starts and successfully connects to the master.
- Master receives the command to start the background save process, and at the same time collects all received commands for modifying the data set. After the background process is executed, master The entire data file will be transferred to the slave to complete a full synchronization
- Full copy: After receiving the database file data, the slave service saves it and loads it into memory.
- Incremental replication: Master continues to pass all new collected modification commands to slave in turn to complete synchronization
- But as long as the master is reconnected, a full synchronization (full copy) will be automatically performed
Passing on the fire
The previous slave can be the master of the next slave, and the slave can also receive connection and synchronization requests from other slaves. Then the slave acts as the next master in the chain, which can effectively reduce the write pressure of the master and reduce risks through decentralization.
Use slaveof
Midway change direction: the previous data will be cleared and the latest copy will be re-created.
The risk is that once a slave goes down, subsequent slaves cannot be backed up.
The host is down, and the slave is still a slave and cannot write data.
Focus on customers
When a master goes down, the following slave can be promoted to master immediately without any modifications to the following slave.
Use slaveof no one to change from slave to master.
Sentinel Mode
An anti-customer-oriented automatic version can monitor whether the host is faulty in the background. If it fails, it will automatically switch from the slave database to the main database based on the number of votes.
Adjusted to one master and two servants mode, 6379 brings 6380 and 6381
Create a new sentinel.conf file in the customized /myredis directory. The name must not be wrong.
Configure Sentinel. Fill in the content
Write a sentine.conf and add the following code to the file
sentinel monitor mymaster 127.0.0.1 6379 1
Among them, mymaster is the server name of the monitoring object, and 1 is the number of at least several sentinels that agree to migrate.
Start Sentinel
.Then start Sentinel
./redis-sentinel sentinel.conf
Test later
Turn off your host and Sentinel will automatically switch to the slave.
The knowledge points of the article match the official knowledge files, and you can further learn related knowledge. Java Skill TreeHomepageOverview 138563 people are learning the system