Introduction
Scene
Recently, the underlying API needs to be connected to the cluster, so nginx is used for forwarding. However, as time goes by, the nginx log becomes larger and larger, and the disk space cannot withstand it. Therefore, the log needs to be divided, because nginx does not support it natively. Logs are stored and divided by day. It is also introduced on the Internet to realize storage by day by configuring nginx.conf, but it did not take effect after a simple local test, and it feels not very flexible, so I did not continue to dig deeper.
Now there are two ways to achieve daily storage of nginx logs or other more customized requirements:
Custom script
The core of the custom script is to use the kill -USR1 `cat /var/run/nginx.pid` command.
The general meaning of this command is: if there is a PID file, then this command will send a USR1 signal to the Nginx main process.
In Nginx, the USR1 signal is used to reopen the log file to achieve log file cutting and renaming. By sending USR1 signal, Nginx master
The process reopens the log file and starts writing new log data, while renaming the old log file as a backup.
Then we can mv or compress the old nginx log files that need to be processed, and then execute the kill -USR1 `cat /var/run/nginx.pid` command.
However, although this method is flexible and customizable enough, the disadvantage is that the operation gap may cause some logs to fail to be written and be lost.
Using logrotate
This article briefly introduces the more powerful and flexible logrotate method.
A brief introduction to logrotate
logrotate is a log file management tool. It is used to split the log file, delete the old log file, and create a new log file, which acts as a “dump”. Can save disk space. Using the logrotate command, you can easily manage the log files generated by the system. Each log file can be set to be processed daily, weekly or monthly, and can be processed immediately if the file is too large.
The following are some important concepts and functions of logrotate:
- Configuration file: The configuration file of logrotate is located in the /etc/logrotate.conf and /etc/logrotate.d/ directories. Among them, logrotate.conf is the main configuration file, and the /etc/logrotate.d/ directory contains individual configuration files of other applications, and each configuration file corresponds to a log rotation rule of an application.
- Log rotation rules: Each log file can define a rotation rule in the configuration file. These rules include the path of the log files, the rotation period (such as daily, weekly, monthly, etc.), and the number of log files to keep.
- Predefined options: logrotate provides some predefined options, such as daily (rotate every day), weekly (rotate every week), monthly (rotate every month), etc. You can use these options in the configuration file to define the log file rotation period.
- Rotation method: logrotate supports different rotation methods, including copytruncate (copy and truncate), rotate (rotate, that is, rename and create a new file), etc. The copytruncate method will copy the current log content to a new file, and then truncate the current log file, so that the application can continue to write to the new file.
- postrotate and prerotate: before or after rotation, logrotate can run user-defined scripts. This is useful if you need to perform additional operations during the rotation, such as restarting services or compacting old log files.
- Configuration check: logrotate provides a -d option that can be used to check the syntax and effect of configuration files to help you confirm whether the rotation rules work as expected.
- Manual rotation: Although logrotate runs automatically on a regular basis, you can also manually trigger log rotation by running the logrotate -f /path/to/config command.
First of all, we need a mirror, which contains nginx, cron, logrotate and other software. For the creation and download of the mirror, please refer to: https://blog.csdn.net/weixin_43702146/article/details/131958486
In-container logrotate installation (Debian)
The Linux system installs the logrotate tool by default. Logrotate runs based on cron. Its script is /etc/cron.daily/logrotate, and the log rotation is automatically completed by the system.
When actually running, logrotate will call the configuration file /etc/logrotate.conf. You can place a custom configuration file in the /etc/logrotate.d directory to override the default value of logrotate
The docker container does not have the logrotate tool by default. For the installation steps, please refer to: https://blog.csdn.net/weixin_43702146/article/details/131958486
How to use logrotate and explain the parameters
-
grammar:
logrotate [options] [configuration file] [configuration parameters] -
options:
-d or –debug: debug mode, display the execution process of instructions in detail, and test whether there are errors in the configuration file
-f or –force: force dump file
-sor –state= : use the specified state file
-v or –version: display the dump process
-m or –mail=command: After compressing the log, send the log to the specified mailbox.
-usage: display the basic usage of the command -
configuration file:
Specify the configuration file of the logrotate command, which can be defaulted, and the default value is /etc/logrotate.conf -
configuration parameters
| Command | Meaning |
|---|---|
| daily | Rotate once a day. By default, the rotation cycle is once a week |
| weekly | Specify the dump cycle as weekly |
| monthly | Specify the dump cycle as monthly |
| Command | Meaning |
|---|---|
| rotate [count] | The number of old log files to keep. For example, rotate 5 means to keep the latest 5 old log files. |
| Command | Meaning |
|---|---|
| dateext | Use the current date as the naming format |
| dateformat .%s | is used with dateext, followed by the next line to define the file name after the file is cut, must When used with dateext, only the four parameters %Y %m %d %s are supported |
| Command | Meaning |
|---|---|
| compress | After the log is compressed and dumped by gzip |
| delaycompress | During the rotation, the old log file will not be compressed immediately, but the compression operation will be postponed once. Usually used with the compress option to ensure that the application does not fail to write new log data while the compaction operation is in progress. |
| nocompress | Do not do gzip compression |
| Command | Meaning |
|---|---|
| create | Specify the attributes of creating a new file during rotation. The default is the same as the previous permissions. You can also customize permissions and groups, such as create 640 nobody nobody |
| nocreate | do not create new log files |
| Command | Meaning |
|---|---|
| size [size] | Trigger rotation when the log file size reaches the specified size. You can use k for KB, M for MB, and G for GB. For example, size 100M means that the log file will be rotated when the size reaches 100MB. |
| maxsize [size] | Set the maximum size of the log file. Once the log file size exceeds this limit, rotation will be forced. Unlike the size option, maxsize does not trigger periodic rotation, it only takes effect when the log file size exceeds the limit |
| minsize [size] | Set the minimum size of log files. If the log file size is smaller than this limit, it will not be rotated |
| Command | Meaning |
|---|---|
| ifempty | Rotate even if the log file is empty, this is the default option of logrotate |
| notifempty | Rotate only when the log file is not empty. If the log file is empty, logrotate will not perform the rotation operation |
| missingok | If the log file does not exist, it will not report an error. logrotate will continue to execute other rotation rules |
| Command | Meaning |
|---|---|
| postrotate [command] endscript | After the rotation, the commands specified in postrotate and endscript will be executed, usually used to restart the service or perform other additional operations |
| prerotate [command] endscript | Before the rotation, the commands specified in prerotate and endscript will be executed, and can also be used to perform some preprocessing operations |
| sharedscripts | When executing postrotate and prerotate, only run the shared script once, instead of executing each log file separately |
| dateyesterday | Execute postrotate or When prerotate, use yesterday’s date |
| Command | Meaning |
|---|---|
| copytruncate | On rotation, copy the current log file to a new file, then truncate the current log file. This allows the application to continue writing to the current file without being affected by changes in the log file name |
| nocopytruncate | Do not use copy and truncate. Old log files will be renamed, which may cause the application to no longer be able to write to the log |
| Command | Meaning |
|---|---|
| errors [address] | Send the error message when saving to the specified email address |
| mail [address] | Send the dumped log file to the specified email address E-mail address |
| nomail | Do not send log files when dumping |
| Command | Meaning |
|---|---|
| olddir [directory] | The dumped log file must be placed in the specified directory, which must be in the same file system as the current log file |
| noolddir | After dumping The log file and the current log file are placed in the same directory |
Test cutting with logrotate
Edit configuration file
Edit the file /etc/logrotate.d/nginx, the following is a brief configuration stored by day
/var/log/nginx/*.log {<!-- -->
# Rotate every day
daily
# ignore errors
missing ok
notifempty
# keep the last 7
rotate 7
# disable gzip compression
delaycompress
# create new file
create
# The suffix of the rotated log file is date
dateext
# Execute script after rotation
shared scripts
# Send USR1 signal to nginx, USR1 signal is used to reopen the log file to achieve log file cutting and renaming. By sending the USR1` signal, the Nginx master process will reopen the log file, and then start writing new log data, while renaming the old log file as a backup.
postrotate
if [ -f /var/run/nginx.pid ]; then
kill -USR1 `cat /var/run/nginx.pid`
the fi
endscript
}
Modify state file
Note that when manually executing logrotate /etc/logrotate.conf, the log file will not be cut after execution.
The working principle of logrotate is roughly: each cutting operation, or the first cutting, will record the time points of all log files in the status file /var/lib/logrotate/status.
In the next execution, if the judgment time minus the recorded time has exceeded a certain period of time (configured time), then the cutting will be performed.
So if you want to cut immediately, you need to modify the status file yourself, change the time to 1 day ago, and then execute.
Modify the status file vim /var/lib/logrotate/status:
logrotate state -- version 2 "/var/log/nginx/error.log" 2023-8-7-22:35:37 "/var/log/nginx/access.log" 2023-8-7-22:35:37
Execute log rotation interactively logrotate -vf /etc/logrotate.d/nginx:
root@75e7c81b599d:/# logrotate -vf /etc/logrotate.d/nginx reading config file /etc/logrotate.d/nginx Creating stub state file: /var/lib/logrotate/status Reading state from file: /var/lib/logrotate/status Allocating hash table for state file, size 64 entries Handling 1 logs rotating pattern: /var/log/nginx/*.log forced from command line (7 rotations) empty log files are not rotated, old logs are removed considering log /var/log/nginx/access.log Creating a new state Now: 2023-08-07 22:35 Last rotated at 2023-08-07 22:00 log needs rotating considering log /var/log/nginx/error.log Creating a new state Now: 2023-08-07 22:35 Last rotated at 2023-08-07 22:00 log needs rotating rotating log /var/log/nginx/access.log, log->rotateCount is 7 dateext suffix '-20230807' glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]' rotating log /var/log/nginx/error.log, log->rotateCount is 7 dateext suffix '-20230807' glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]' renaming /var/log/nginx/access.log to /var/log/nginx/access.log-20230807 creating new /var/log/nginx/access.log mode = 0644 uid = 101 gid = 0 renaming /var/log/nginx/error.log to /var/log/nginx/error.log-20230807 creating new /var/log/nginx/error.log mode = 0644 uid = 101 gid = 0 running postrotate script
It is found that the log in the specified directory has become:
-rw-r--r-- 1 101 root 40745 Aug 7 22:41 access.log -rw-r--r-- 1 root root 3173 Aug 7 14:32 access.log-20230806 -rw-r--r-- 1 101 root 61823 Aug 7 22:31 access.log-20230807 -rw-r--r-- 1 101 root 357 Aug 7 22:37 error.log -rw-r--r-- 1 root root 2635 Aug 7 14:49 error.log-20230806 -rw-r--r-- 1 101 root 15466 Aug 7 22:35 error.log-20230807
Cooperate with timed tasks to complete the daily rotation
Start scheduled task
service cron start
crontab syntax
crontab [-u user] file or crontab [-u user] [-e | -l | -r]
Parameter Description:
- -u user refers to setting the schedule of the specified user. The premise is that you must have its authority (for example, root) to be able to specify the schedule of others. If you don’t use -u user, it means setting your own schedule.
- -e executes a text editor to set the schedule
- -l list the current schedule
- -r delete the current schedule (use with caution)
Set log rotation at 0:00 every day
crontab -e
Enter the following in the pop-up editor:
59 23 * * * /usr/sbin/logrotate -f /etc/logrotate.d/nginx
crontab -l View current scheduled tasks:
root@75e7c81b599d:/# crontab -l # Edit this file to introduce tasks to be run by cron. # # Each task to run has to be defined through a single line # indicating with different fields when the task will be run # and what command to run for the task # # To define the time you can provide concrete values for # minute (m), hour (h), day of month (dom), month (mon), # and day of week (dow) or use '*' in these fields (for 'any'). # # Notice that tasks will be started based on the cron's system # daemon's notion of time and timezones. # # Output of the crontab jobs (including errors) is sent through # email to the user the crontab file belongs to (unless redirected). # # For example, you can run a backup of all your user accounts # at 5 a.m every week with: # 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/ # # For more information see the manual pages of crontab(5) and cron(8) # # m h dom mon dow command 59 23 * * * /usr/sbin/logrotate -f /etc/logrotate.d/nginx
Modify the time zone and restart the service
echo 'Asia/Shanghai' >/etc/timezone
service cron restart