Cache middleware distributed redis

1. Introduction to redis

Nosql non-relational database key => value key-value pair

Redis is the abbreviation of Remote Dictionary Server (remote data service)

A memory cache database developed by Italian antirez (Salvatore Sanfilippo). The software is written in C language, and its data model is key-value

It supports rich data structures, such as “”String list (doubly linked list) hash (hash) set (set) sorted set (zset ordered set) big

Persistence (save data to disk), ensuring data security

Business usage scenarios:

①[Sort set] ranking application, take top n operations, such as sina Weibo hot topics

②[List] Obtain the latest N data or the latest data of a certain category

③[string] counter application

④[Set]Jsns(social network site) get mutual friends

⑤[Set] Anti-attack system (ip judgment) black and white list, etc.

Compared with memcached

①Redis not only supports simple k/v type data, but also provides storage of data structures such as list, set, zset, and hash. ②Redis supports master-slave (master-slave) mode applications.

③Redis supports data persistence, which can save the data in the memory to the disk, and can be loaded again for use when restarting. ④The maximum limit of Redis single value storage string is 512MB, and memcached can only save 1MB of data

⑤redis is single-core, memcached is multi-core

Since redis can only use a single core, and memcached can use multiple cores, in comparison, on average, redis on each core has higher performance than memcached when storing small data. However, in the data above 100K, the performance of memcached is higher than that of redis. Although redis has recently optimized the performance of storing large data, it is still a bit inferior to memcached. The conclusion is that no matter which one you use, the number of requests per second should not be the bottleneck.

You need to keep an eye on memory usage. For simple data storage like key-vlaue, the memory usage of memcached is higher. If the hash structure is used, the memory usage of redis will be higher. Of course, this all depends on the specific application scenario.

2. Redis source code compilation and installation and client test start

Official website: https://redis.io/

github: https://github.com/antirez/redis

  1. Source code compilation and installation

① Upload the software to the directory

redis-4.3.0.tgz php extension server01 and server03 installation

redis-5.0.5.tar.gz redis software server08 installation

② Unzip, compile and install

tar xvf redis-5.0.5.tar.gz
cd redis-5.8.5
make PREFIX=/usr/local/redis install //This configuration directory

Use the file /usr/local/redis/bin

redis-cli command line client

redis-server server

③Test start

cd /usr/local/redis/bin

./redis-server //Start the server

2. Background startup configuration

① Copy the configuration file to the software directory

②Modify and edit the configuration file

vim /usr/local/redis/bin/redis.conf

/daemonize daemon process

③Start redis-server and add redis.conf parameters

cd /usr/local/redis/bin
./redis ./redis-server //Start redis background service

3. The command line client is simple to use

redis-cli -h

Client Help Parameters

3. Data structure type operation

  1. key

Keys containing spaces and newlines like “my key” and “mykey\
” are not allowed.

Key should not be too long. It takes up memory and the query is slow.

Key should not be too short.

Like u:1000:pwd:123456 is not as readable as user:1000:password:123456

2, String

string is the most basic type of redis

The string of redis can contain any data. The maximum limit of a single value of a jpg image base64 or serialized object is 512MB

If only string type is used, redis can be regarded as memcached with persistence feature

In a real business environment, it is only necessary to obtain whether the key has data and whether the key exists

3, List

key value(value1,value2,value3)

The list type is actually a doubly linked list. Add and delete elements from the head or tail of the linked list through push and pop operations, which makes the list can be used as a stack or a queue

In and out at the same end First in last out Stack Candied haws, the one inserted first can be eaten last

– end other – end out first in first out queue queue, first come first served

Header (left left)

Tail (right right)

Requirement: Display the last logged-in users

Design implementation:

①Log in a user, store the user name or id in the list

②Save from the head (left part), take from the head (left part)

User: xiaoming

xiaohua

xiaobaitu

rpop deletes the first element on the right of the list, and returns the column of the first logged-in user

xiaobaitu xiaohua xiaoming

lpop deletes the first element on the left of the list, and returns the stack of the last logged-in user

xiaobaitu xiaohua xiaoming

4, set

The set of redis is an unordered collection of string type. Duplicate elements are not allowed in the set

The set element can contain a maximum of (2 to the 32nd power-1) elements.

Regarding the set collection type, in addition to the basic addition and deletion operations, other commonly used operations include collection union, intersection, and difference. Through these operations, the friend recommendation function in sns can be easily implemented.

Tip: mysql connected table Venn diagram

https://www.cnblogs.com/sunjie9606/p/4167190.html

Requirements: Realize the storage of the circle of friends and the calculation of common friends

design:

key value

xiaomingFR xiaohong xiaoqiang xiaogang xiaohei xiaobai

xiaohongFR xiaoming xiaolv xiaolan xiaobai xiaohei

l set type operation

Add a string element to the set corresponding to the key, return 1 if successful, return 0 if the set element is already in the set,

The set corresponding to the key does not exist and returns an error

srem key member [menber]

Remove the given element from the set corresponding to the key, and return 1 on success

move p1 p2 menber

Remove the menber from the set corresponding to p1 and add it to the set corresponding to p

card key

Returns the number of elements in the set

sismenber key menber

Determine whether the menber is in the set

sinter key1 key2- –keyN

Returns the intersection of all given keys

sunion key1 key2 —.kegH

Returns the union of all given keys

sdiff keg1 key2—keyN

Returns the difference of all given keys

smenbers key

Returns all elements corresponding to the key in unordered order

5, zset

zadd key score menber

Add an element to the collection, update the corresponding score if the element exists in the collection

zren keg menber

Delete the specified element, 1 means success, if the element does not exist, return 0

zincrby key incr menber

Increase the score value of the corresponding menber according to the incr amplitude (positive or negative) and return the score value

zrank key menber

Returns the rank (subscript) of the specified element in the collection, the elements in the collection are sorted by score from small to large

zrevrank key nenber

As above, the elements in the collection are sorted by score from largest to largest

zrange key start end

Similar to the lrange operation, the elements of the specified range are removed from the collection. Returns ordered results

zrev range key start end

Same as above, the returned results are in reverse order of score

zcard key

Returns the number of elements in the collection

zscore key element

return score

zrenrangebyrank key min max

Deletes elements in the given range from the collection

6, Hash

Using redis as a cache, you can use a hash structure, which is more efficient in compression and use. Compared with string Hash to store data and relational database (mysql), the structure of a piece of stored data is very similar Key: value (field: value)

hash type

hset key field value

Set the hash field to the specified value, if the key does not exist, create it first

hget key field

Get the specified hash fieid

hmget key filed1—FieldN

Get all the specified hash filed

hnset key filed1 value1 —

filedN valueN

Set multiple fields of hash at the same time

hincrby key field integer

Add the specified hash filed to the given value

hexists key field

Test whether the specified fie1d exists

hdel key field

Delete the specified hash field

hlen key

Returns the number of fields of the specified hash

hkeys key

Return all fields of hash

hvals key

Return all values with the same hash

hgetall key

Return all filed and value of hash

devops: (username: devops, phone: 15313131313, email: [email protected])

Hash data type: storage and value are all through key, and the data stored in relational data string has nothing to do with it, it is independent

17301296261: (username: 17301296261, phone: 17301296261, email: [email protected])

Fourth, data persistence operation

Data persistence (data is not lost after service or software restart)

If the data only exists in the memory, it will definitely be lost. To achieve persistence, the data needs to be stored in the disk (hdd ssd)

1, snapshotting (snapshot)

snap to take pictures in advance

By default snapshotting is enabled and there is a backup frequency

By looking at the configuration file you can see

set num1 1 always store 10 keys to trigger the backup frequency of snapshotting

Manual data backup

Save directly after storing the data

127.0.0.1:6379> save

Data recovery needs to stop the redis service first killall redis-server shutdown redis-cli

If you need to restore data, just move the backup file (dump.rdb) to the redis installation directory and start the service

server08 bin]# cp /root/dump.rdb

Restart the service and the data will be restored

2, append only file

aof backup frequency

# appendfsync always //Forcibly write to disk every time a write command is received, the slowest, but

It guarantees complete persistence and is not recommended

appendfsync everysec // Forced to write to disk once per second, in terms of performance and persistence

A good compromise, recommended (keep a backup)

#appendfsync no //Completely depends on os, the performance is the best, and the persistence is not guaranteed

①Open aof

After enabling aof, the data in the previous redis will be lost

appendfilename “appendonly.aof” //data storage location

Summarize:

Snapshotting is used for general data persistence, high efficiency, and convenient data migration

aof is suitable for backup and data real-time backup requirements

6. Case use

1. Master-slave mode

1. Master configuration

shell > vim /usr/local/redis/bin/redis.conf

Enable monitoring, the network card for network interaction with other servers in the network, default eth0

The ip of bind refers to the ip that other hosts need to communicate with this host (the ip of this machine’s external communication)

master:192.168.17.107

Restart the server and check if it starts successfully

./redis-cli -h 192.168.17.107
slave remote connection master

2. slave configuration

vim /usr/local/redis/bin/redis.conf

yy copy the line where the cursor is located, p paste it to the line where the cursor is located n look down N look up/find content

Before redis5.0 version is to modify slaveof

Restart the server and check if it starts successfully

The slave server does not allow write operations, the configuration file is set

2. Security configuration

Configure the remote connection password for the master

Tip: If the password restriction is enabled, you need to fill in the master password in the slave configuration to build the master-slave

3. PHP extension installation

To view phpinfo, you need to write a php file with the following content

<?php
    phpinfo();
?>
The phpinfo() function outputs information about the PHP configuration.
  1. session stored in redis

In the same way as the previous way of storing sessions in memcached, storing sessions in redis can also realize session sharing and single sign-on (sso) operations.

2. Nginx + lua + redis implement access attack blacklist WAF

https://www.cnblogs.com/huligong1234/p/4163832.html

①The installation of openresty is consistent with the previous installation

Calculate the number of visits to the same ip on the load balancing server through the lua script, and add it to the blacklist if there are too many consecutive visits

The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge Cloud native entry skill treeHomepageOverview 10869 people are studying systematically