Table of contents
1. Caching concept
1.1 System cache
1.1.1buffer and cache
1.2 Cache storage location and hierarchical structure
1.2.1 DNS cache
1.2.2 Application layer caching
1.2.3 Data layer cache
1.2.4 Hardware cache
2. Relational databases and non-relational databases
2.1 What is a relational database?
2.2 What is a non-relational database
2.3 The background of non-relational databases
2.4 Differences between relational databases and non-relational databases
2.5 Summary
3. Introduction to Redis
4. Advantages of Redis
5. Single thread
6. Redis installation and deployment
1. The concept of caching. The purpose of caching is to adjust the speed of two or more different substances with inconsistent speeds, and to accelerate the slower one in the middle. For example, the first-level and second-level caches of the CPU save the CPU’s recent frequently used data. The accessed data and memory are used to store the data that the CPU frequently accesses the hard disk, and the hard disk also has caches of different sizes, and even the raid card of the physical server has caches, all for the purpose of accelerating the CPU’s access to hard disk data, because the CPU The speed is too fast. The hard disk often cannot meet the needs of the CPU in a short time. Therefore, the CPU cache, memory, Raid card cache and hard disk cache meet the data needs of the CPU to a certain extent, that is, the CPU caches data from the cache. Reading data can greatly improve the efficiency of the CPU.
1.1 System cache 1.1.1buffer and cachebuffer: Buffer is also called write buffer. It is generally used for write operations. Data can be written to memory first and then to disk. Buffer is generally used for write buffering to solve the problem of inconsistent speeds of different media. , first temporarily write the data to the nearest place to improve the writing speed. The CPU will first write the data to the disk buffer of the memory, and then it will consider that the data has been written to completion, and then the kernel will check it at a later time. Writing to disk, so a sudden power outage on the server will cause some data in the memory to be lost.
Cache: Cache is also called read cache. It is generally used for read operations. The CPU reads files from the memory. If the memory does not exist, it first reads from the hard disk to the memory and then to the CPU. Data that needs to be read frequently is placed in its own recent cache. area, it can be read quickly the next time it is read.
1.2 Cache storage location and hierarchical structure In the Internet application field, it is mentioned that cache is king
User layer: browser DNS cache, application DNS cache, operating system DNS cache client
Proxy layer: CDN, reverse proxy cache
Web layer: Web server cache
Application layer: page staticization
Data layer: distributed cache, database
System layer: operating system cache
Physical layer: disk cache, Raid Cache
1.2.1 DNS cache
The default DNS cache of the browser is 60 seconds, that is, no DNS resolution will be performed when accessing the same domain name within 60 seconds.
1.2.2 Application layer cache Web services such as Nginx and PHP can set up application cache to speed up response to user requests. In addition, some interpreted languages, such as PHP/Python/Java, cannot be run directly and need to be compiled into bytecode first, but bytecode The code needs to be interpreted into machine code by the interpreter before it can be executed. Therefore, the bytecode is also a kind of cache. Sometimes the bytecode is not updated after the program code goes online. Therefore, generally before launching a new version, you need to clear the application cache and then launch the new version.
In addition, dynamic page staticization technology can be used to speed up access. For example, dynamic pages that access database data can be programmed to generate static page files in HTML in advance. Product introductions on e-commerce websites, non-real-time data of review information, etc. can all be implemented using this technology. .
1.2.3 Data layer cache
-
Distributed cache service
Redis
Memcached
-
database
MySQL query cache
innodb cache, MYISAM cache
1.2.4 Hardware Cache
-
CPU cache (L1 data cache and L1 instruction cache), L2 cache, L3 cache
-
Disk Cache: Disk Cache
-
Disk array cache: Raid Cache, which can use batteries to prevent data loss during power outages
2. Relational databases and non-relational databases2.1 What is a relational database? A structured database, created on the basis of the relational model (two-dimensional tabular model)
Generally oriented towards recording
SQL statement (standard data query language)
It is a language based on a relational database, used to retrieve and operate data in a relational database. Including: Oracle, MySQL, SQL Server, Microsoft Access, DB2, etc.
2.2 What is a non-relational database
-
NoSQL (NoSQL=NotOnlySQL), which means “not just SQL”, is the general term for non-relational databases.
-
Databases other than mainstream relational databases are considered non-relational.
-
Mainstream NoSQL databases include Redis, MongBD, Hbase, Memcached, etc.
2.3 The background of non-relational databases
-
High performance – high concurrent reading and writing requirements for the database
-
Huge Storage–Requirements for efficient storage and access of massive data
-
High Scalability & amp; & amp; High Availability–Requirements for high scalability and high availability of databases
2.4 Differences between relational databases and non-relational databases
2.5 Summary
-
Relational database: instance->database->table (table)->record row (row), data field (column)
-
Non-relational database: Instance->Database->Collection–>Key-value pair (key-value) Non-relational database does not require manual creation of databases and collections (tables).
3. Introduction to RedisOfficial website: Redis
Redis is an open source NoSOL database written in C language. The Redis server program is a single-process model.
Redis runs based on memory and supports persistence (supports storage on disk). It adopts key-value (key-value pair) storage form and is an indispensable part of the current distributed architecture.
The Redis service can start multiple Redis processes at the same time on one server. The actual processing speed of Redis completely depends on the execution efficiency of the main process.
If only one Redis process is running on the server, when multiple clients access it at the same time, the server’s processing capability will decrease to a certain extent;
If multiple Redis processes are opened on the same server, Redis will put a lot of pressure on the server’s CPU while improving concurrent processing capabilities. That is, in an actual production environment, you need to decide how many Redis processes to start based on actual needs. (It is generally recommended to open 2 for backup and anti-high concurrency)
If you have higher requirements for high concurrency, you may consider starting multiple processes on the same server. If CPU resources are tight, use a single process.
4. Advantages of Redis It has extremely high data reading and writing speed: the data reading speed can reach up to 110,000 times/s, and the data writing speed can reach up to 81,000 times/s. ,
Supports rich data types: supports key-value, Strings, Lists, Hashes (hash values), Sets, OrderedSets and other data type operations. pS: string string (can be integer, floating point and character type, collectively referred to as elements) list list: (implementation of queue, elements are not unique, first-in-first-out principle) set set: (different elements) hash hash hash Value: (hash key must be unique) set /ordered sets set/ordered set
Supports data persistence: the data in the memory can be saved on the disk and can be loaded again for use when restarting.
Atomic: All Redis operations are atomic.
Supports data backup: data backup in master-salve mode.
5. Single thread
Before Redis version 6.0, user requests were processed in a single-threaded manner.
Why is single thread so fast?
-
pure memory
-
non-blocking
-
Avoid thread switching and race consumption
Six. Redis installation and deployment
6.1 Deployment steps
#Close firewall
systemctl stop firewalld
setenforce 0
(2)#Install gcc gcc-c++ compiler
yum install -y gcc gcc-c++ make
(3)#Switch to the /opt directory, upload the downloaded installation package and decompress it
cd /opt/
tar zxvf redis-5.0.7.tar.gz
4)#Enter the directory and compile and install
cd /opt/redis-5.0.7/
make
make PREFIX=/usr/local/redis install
#Since the Makefile file is directly provided in the Redis source code package, after decompressing the software package, you do not need to execute ./configure for configuration first. You can directly execute the make and make install commands for installation. 5) #Execute the install_server.sh script
5)#Execute install_server.sh script
cd /opt/redis-5.0.7/utils
./install_server.sh #Press Enter all the way and the guide will let you enter the path.
#The path needs to be entered manually
Please select the redis executable path [] /usr/local/redis/bin/ redis-server
Selected config:Port: 6379 #The default listening port is 6379Config file: /etc/redis/6379.conf #Configuration file path Log file: /var/log/redis_6379.log #Log file path Data dir: /var/lib/ redis/6379 #Data file path Executable: /usr/local/redis/bin/redis-server #Executable file path Cli Executable: /usr/local/redis/bin/redis-cli #Client command tool 6)#Optimization Path and check whether the port is open #Put the redis executable program file into the directory of the path environment variable to facilitate system identification ln -s /usr/local/redis/bin/* /usr/local/bin/#When install_server.sh After the script is run, the Redis service has been started. The default listening port is 6379netstat -natp | grep redis7) #Modify the configuration file vim /etc/redis/6379.confbind 127.0.0.1 192.168.200.50 #70 line, add the listening host address port 6379 #93 line, Redis default listening port daemonize yes #137 line, enable the daemon process pidfile /var/run/redis_6379.pid #159 line, specify the PID file loglevel notice #167 line, log level logfile /var/log/ redis_6379.log #172 line, specify log text 8) #Restart redis to view the listening address/etc/init.d/redis_6379 restart #Restart ss -antp|grep redis9)##Redis service control/etc/init.d/redis_6379 stop #Stop/etc/init.d/redis_6379 start #Start/etc/init.d/redis_6379 restart #Restart/etc/init.d/redis_6379 status #Status
1)# Turn off the firewall and SElinux
2)#Install gcc gcc-c++ compiler
3)#Switch to the /opt directory, upload the downloaded installation package and decompress it
4)#Enter the directory and compile and install
5)#Execute install_server.sh script
6)#Optimize the path and check whether the port is open
7)#Modify configuration file
8) #Restart redis to view the monitored address
6.2Redis command tool
rdb and aof are two forms of persistence function in redis service. redis-cli is often used to log in to the redis database.
6.3 redis-cli command line tool (remote login)
6.4 redis-benchmark testing tool
redis-benchmark is the official Redis performance testing tool that can effectively test the performance of Redis services.
#Syntax redis-benchmark [option] [option value]
redis-benchmark -h 192.168.217.103 -p 6379 -c 100 -n 100000 #Send 100 concurrent connections and 100000 requests to the Redis server with IP address 192.168.224.101 and port 6379 to test performance
redis-benchmark -h 192.168.217.103 -p 6379 -q -d 100 #Test the performance of accessing data packets with a size of 100 bytes
6.5 Common Redis commands
6.5set/get stores/gets data
set stores data, the command format is set key value get gets data, the command format is get key
6.6keys value
The
keys command can obtain a list of key values that conform to the rules. It can usually be used in combination with options such as * and ?. keys * #seeall keys v* #Look at all those starting with v keys v? #Look at the ones starting with v and followed by only one byte keys v? ? #Look at the words starting with v and followed by only two bytes
6.7del delete key
The
del command can delete the specified key of the current database. k1 #deletek1
6.8exists determines whether the value exists
The
exists command can determine whether the key value exists exists v1 #Check if v1 exists (integer) 1 #exist exists k1 #Check if k1 exists (integer) 0 #does not exist
6.9type Get the type of value
The
type command can obtain the value type corresponding to the key. type v1 #View v1 type
6.10rename rename (overwrite)
The
rename command renames an existing key. (cover) Command format: rename source key target key
6.11renamenx rename (not overwrite)
The purpose of the
renamenx command is to rename an existing key and detect whether the new name exists. If the target key exists, the rename will not be performed. (not covered) Command format: renamenx source key target key
6.12dbsize Check the number of keys in the library
6.13 Set password
Use the config set requirepass password command to set the password Use the config get requirepass command to view the password (once a password is set, it must be verified first, otherwise all operations will be unavailable) config set requirepass 123456 #Change password to 123456 auth 123456 #Verify password
6.14move mobile data
Redis's multiple databases are relatively independent to a certain extent. For example, the data of k1 stored in database 0, It cannot be viewed on other 1-15 databases. move key name database number move v1 1 #Move v1 to 1 library select 1 #Switch to library 1 keys *
Example 1: Send 100 concurrent connections and 100,000 requests to the Redis server with IP address 192.168.59.118 and port 6379 to test performance
redis-benchmark -h 192.168.59.118 -p 6379 -c 100 -n 100000
redis command tool
sed (create) get (get) key (view)
* So v* starts with all v? starts with v and follows one character
exists determines whether the value exists. Returns a result of 1 for existence and 0 for non-existence. Type View type
del delete rename (overwrite) renamenx (do not overwrite)
dbsize counts how many values auth login password
config set requirepass 123456 set 123456 password redis -cli-a XXX (password)
select Set the serial number (0-15). By default, there are only 16 libraries. Move Remove. Remove. flushdb. Clear all data in the current library. flushall. Clear all data. Use with caution.