[Nginx] cache integration

Article directory

  • The concept of caching
  • Nginx web cache service
  • Related instructions for Nginx cache settings
  • Nginx cache setting case
  • Clearing of Nginx cache
    • Method 1: Delete the corresponding cache directory
    • Method 2: Use a third-party extension module
  • Nginx sets resources not to cache

The concept of caching

Cache is the buffer for data exchange (called: Cache). When users want to obtain data, they will first query and obtain data from the cache. If there is any in the cache, it will be returned to the user directly. Send a request to re-query the data from the server, return the data to the user and put the data into the cache at the same time, the next time the user will directly get the data from the cache.

Caching is actually useful in many scenarios, such as:

Scenario Function
Operating system disk cache Reduce disk mechanical operations
Database cache Reduce file system IO operations
Application caching Reduce queries to the database
Web server caching Reduce the number of requests to the application server
Browser cache Reduce the number of interactions with the background

Advantages of caching

? 1. Reduce data transmission, save network traffic, speed up response, and improve user experience;

2. Reduce server pressure;

3. Provide high availability of the server;

Disadvantages of caching

? 1. Data inconsistency

2. Increased cost


As a web cache server, Nginx is between the client and the application server. When a user accesses a URL through a browser, the web cache server will go to the application server to obtain the content to be displayed to the user, and cache the content on its own server. , when the next request comes, if the same URL is accessed, the web cache server will directly return the previously cached content to the client instead of sending another request to the application server. Web caching reduces the load on application servers and databases, reduces network delays, improves the response speed of user access, and enhances user experience.

Nginx web cache service

Nginx has provided caching function since version 0.7.48. Nginx is a web cache service based on Proxy Store. Its principle is to use URL and related combinations as Key, and use MD5 algorithm to hash the Key to obtain the corresponding hash directory path on the hard disk, so as to save the cached content in this in the directory. It can support any URL connection, and also supports non-200 status codes such as 404/301/302. Nginx can not only support setting the expiration time for the specified URL or status code, but also use the purge command to manually clear the cache of the specified URL.

What is Proxy Store?

Proxy Store is an advanced caching function provided by Nginx. It works as follows:

  1. Proxy Store stores the cache key (key) and cache value (value) separately. Keys are stored in memory and values are stored on disk. This enables storing more and larger cache values while maintaining high performance.
  2. Keys are defined by variables (such as

    the s

    c

    h

    e

    m

    e

    ,

    scheme,

    scheme, host, etc.) and cache key template (proxy_store_key) generation. like:
    proxy_store_key”

    the s

    c

    h

    e

    m

    e

    scheme

    schemeproxy_host$uri”;
    This template will generate keys like: “httpswww.example.com/foo” upon request.

  3. The value (value) is stored on disk in the cache path (proxy_store_path), and the filename is also generated from the key. The key in the example above would generate the filename “httpswww.example.com/foo”.
  4. When a request comes in, Nginx first generates a key according to the template and variables, searches for the key in memory, and if it finds it (cache hit), it directly reads the file with the corresponding cache value file name from the disk to respond to the client.
  5. If not found (cache miss), the request is forwarded to the backend server, and the response is written to disk, and the key is recorded in memory at the same time, and the implement session is completed.
  6. Compared with the basic cache, Proxy Store can store a larger size (several GB) cache, but the search speed will be slightly slower (requires IO to read the disk). So Proxy Store is suitable for requests with large response bodies.
  7. Proxy Store can work with basic cache. Nginx will check the basic cache first, and then check the Proxy Store if it misses. This allows for high performance while maximizing cache utilization.

So overall, by storing keys and values separately, Proxy Store not only realizes larger-capacity cache storage, but also achieves higher performance, which is a good supplement to Nginx cache function.

Instructions related to Nginx cache settings

Nginx’s web cache service is mainly completed by using the relevant instruction set of the ngx_http_proxy_module module. Next, we will introduce the commonly used instructions.

proxy_cache_path

This designation is used to set the storage path of the cache file

Syntax proxy_cache_path path [levels=number]
keys_zone=zone_name:zone_size [inactive=time][max_size=size];
default –
Location http

path: cache path address, such as:

/usr/local/proxy_cache

levels: Specifies the directory corresponding to the cache space, up to 3 levels can be set, and the value of each level is 1|2, such as:

levels=1:2 The cache space has two levels of directories, the first is 1 letter, the second is 2 letters
for example:
The value of itheima[key] encrypted by MD5 is 43c8233266edce38c2c9af0694e2107d
levels=1:2 The final storage path is /usr/local/proxy_cache/d/07
levels=2:1:2 The final storage path is /usr/local/proxy_cache/7d/0/21
levels=2:2:2 The final storage path is /usr/local/proxy_cache/7d/10/e2

keys_zone: Used to set the name and specify the size of this cache area, such as:

keys_zone=itcast:200m The name of the cache area is itcast, the size is 200M, and 1M can store about 8000 keys

inactive: The specified cached data will be deleted if it is not accessed within the specified time, such as:

inactive=1d cached data will be deleted if it is not accessed within 1 day

max_size: Set the maximum cache space. If the cache space is full, the resource with the longest cache time will be overwritten by default, such as:

max_size=20g

Configuration example:

http{
proxy_cache_path /usr/local/proxy_cache keys_zone=itcast:200m levels=1:2:1 inactive=1d max_size=20g;
}

proxy_cache

This command is used to turn on or off the proxy cache, if it is turned on, which cache area to use for caching.

Syntax proxy_cache zone_name|off;
default proxy_cache off;
location http, server, location

zone_name: specifies the name of the cache area to be used

proxy_cache_key

This command is used to set the key value of the web cache, and Nginx will store the cache according to the MD5 hash of the key value.

Syntax proxy_cache_key key;
default proxy_cache_key $scheme$proxy_host$request_uri;
location http, server, location

proxy_cache_valid

This instruction is used to set different cache time for URLs with different return status codes

Syntax proxy_cache_valid [code …] time;
default –
Location http, server, location

like:

proxy_cache_valid 200 302 10m;
proxy_cache_valid 404 1m;
Set a 10-minute cache for 200 and 302 response URLs, and a 1-minute cache for 404 response URLs
proxy_cache_valid any 1m;
Set a 1-minute cache for all URLs with response status codes

proxy_cache_min_uses

This instruction is used to set how many times the resource is accessed before being cached

Syntax proxy_cache_min_uses number;
default proxy_cache_min_uses 1;
location http, server, location

proxy_cache_methods

This directive is used to set which HTTP methods are cached

Syntax proxy_cache_methods GET|HEAD|POST;
default proxy_cache_methods GET HEAD;
location http, server, location

By default, the GET and HEAD methods of HTTP are cached, and the POST method is not cached.

Nginx cache setting case

Needs Analysis

Steps to achieve

1. Environment preparation

Environment preparation for the application server

(1) Add a js directory under tomcat’s webapps on the 192.168.200.146 server, and add a jquery.js file in the js directory

(2) start tomcat

(3) Access test

http://192.168.200.146:8080/js/jquery.js

Environment preparation for Nginx

(1) Complete Nginx reverse proxy configuration

http{
upstream backend{
server 192.168.200.146:8080;
}
server {
listen 8080;
        server_name localhost;
        location / {
        proxy_pass http://backend/js/;
        }
}
}

(2) Complete Nginx cache configuration

4. Add cache configuration

http{
proxy_cache_path /usr/local/proxy_cache levels=2:1 keys_zone=itcast:200m inactive=1d max_size=20g;
upstream backend{
server 192.168.200.146:8080;
}
server {
listen 8080;
        server_name localhost;
        location / {
        proxy_cache itcast;
            proxy_cache_key itheima;
            proxy_cache_min_uses 5;
            proxy_cache_valid 200 5d;
            proxy_cache_valid 404 30s;
            proxy_cache_valid any 1m;
            add_header nginx-cache "$upstream_cache_status";
        proxy_pass http://backend/js/;
        }
}
}
</code><img class="look-more-preCode contentImg-no-view" src="//i2.wp.com/csdnimg.cn/release/blogv2/dist/pc/img/newCodeMoreBlack. png" alt="" title="">

Clearing of Nginx cache

Method 1: Delete the corresponding cache directory

rm -rf /usr/local/proxy_cache/…

Method 2: Use a third-party extension module

ngx_cache_purge

(1) Download the resource package corresponding to the ngx_cache_purge module and upload it to the server.

ngx_cache_purge-2.3.tar.gz

(2) Decompress the resource file

tar -zxf ngx_cache_purge-2.3.tar.gz

(3) Modify the folder name to facilitate later configuration

mv ngx_cache_purge-2.3 purge

(4) Query the configuration parameters of Nginx

nginx -V

(5) Enter the installation directory of Nginx and use ./configure to configure parameters

./configure --add-module=/root/nginx/module/purge

(6) Compile with make

make

(7) Back up the nginx secondary executable file in the nginx installation directory

mv /usr/local/nginx/sbin/nginx /usr/local/nginx/sbin/nginxold

(8) Copy nginx in the compiled objs to the sbin directory of nginx

cp objs/nginx /usr/local/nginx/sbin

(9) Use make to upgrade

make upgrade

(10) Configure the following in the nginx configuration file

server{
location ~/purge(/.*) {
proxy_cache_purge itcast itheima;
}
}

Nginx sets resources not to cache

Earlier we have completed the use of Nginx as a web cache server. But we have to think about a problem that not all data is suitable for caching. For example, for some data that changes frequently. If it is cached, it is easy to show that the data accessed by the user is not the real data of the server. Therefore, we need to filter these resources during the caching process and not cache them.

Nginx also provides this feature setting, which requires the use of the following two instructions

proxy_no_cache

This instruction is used to define the conditions for not caching data.

Syntax proxy_no_cache string …;
default –
location http, server, location

Configuration example

proxy_no_cache $cookie_nocache $arg_nocache $arg_comment;

proxy_cache_bypass

This instruction is used to set the condition not to fetch data from the cache.

Syntax proxy_cache_bypass string …;
default –
location http, server, location

Configuration example

proxy_cache_bypass $cookie_nocache $arg_nocache $arg_comment;

The above two instructions have a specified condition, which can be multiple, and at least one of the multiple conditions is not empty and not equal to “0”, then the condition is satisfied and established. The configuration example given above is obtained from the official website, and three variables are used in it, namely $cookie_nocache, $arg_nocache, $arg_comment

The meanings of these three parameters are:

$cookie_nocache
Refers to the value corresponding to the name of the key in the cookie of the current request as nocache
$arg_nocache and $arg_comment
Refers to the attribute value corresponding to the attribute name nocache and comment in the parameters of the current request

Under the case demonstration:

log_format params $cookie_nocache | $arg_nocache | $arg_comment;
server {
listen 8081;
server_name localhost;
location /{
access_log logs/access_params.log params;
add_header Set-Cookie 'nocache=999';
root html;
index index.html;
}
}

Case Implementation

Set the configuration scheme that does not cache resources

server{
listen 8080;
server_name localhost;
location / {
if ($request_uri ~ /.*\.js$){
           set $nocache 1;
        }
proxy_no_cache $nocache $cookie_nocache $arg_nocache $arg_comment;
        proxy_cache_bypass $nocache $cookie_nocache $arg_nocache $arg_comment;
}
}