How to use logrotate for cutting log file technology
By the way, condemn yourself, the technical documentation is always not updated, laziness is really terrible.
logrotate is a built-in command of centos. Other linux operating systems may need to be installed by themselves for log cutting and periodic deletion. Simply put, it is to divide a log file into multiple parts according to time or size, and automatically delete old logs. Commonly used log cutting methods: by time and by log size.
Problem: The mysql.err log file is too large today, and it needs to be divided into files. It is required to divide it every day and keep the logs for seven days.
First go to /etc/logrotate.d and create a file named mysqlerr
/data/mysql/mysql.err {<!-- -->
missingok #When the rotated log does not exist, continue the rotation of the next log without reporting an error
notifempty #No rotation when the log is empty
copytruncate #When rotating, copy the complete log file and clear the original log file (equivalent to echo > logfile), and the newly written log will continue to be written to the cleared log file. But the logs written in the process of log copy and truncate may be lost. When the copytruncate option is specified, the create option will be disabled
rotate 7 #keep up to 7 files after cutting
compres #Use compression, the default compression method is gzip
delaycompress #Delayed compression, the time when the actual compression takes effect occurs at the next log cut
dateext #The suffix name of the log after cutting is in the format of "YYYYMMDD"
}
Do it manually
/usr/sbin/logrotate -vf /etc/logrotate.d/mysqlerr
The file is cut successfully, and the configuration of crontab is performed
0 0 * * * /usr/sbin/logrotate -vf /etc/logrotate.d/mysqlerr &>/dev/null
So far, it’s over.
Learn more about it
logrotate – log rotation
Log splitting on the Linux operating system can be achieved through logrotate.
Compared with other log cutting software, using logrotate has the following advantages:
logrotate is an open source log cutting software that comes with the Linux operating system, so you don’t need to install
logrotate itself has been integrated into the scheduled task of the operating system, so you don’t need to configure the scheduled task
logrotate itself supports log compression
Logrotate configuration file analysis
The global configuration file for logrotate is /etc/logrotate.conf.
[root@server ~]# cat /etc/logrotate.conf
# see "man logrotate" for details
# rotate log files weekly
weekly
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create
# use date as a suffix of the rotated file
dateext
# uncomment this if you want your log files compressed
#compress
# RPM packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own wtmp and btmp -- we'll rotate them here
/var/log/wtmp {
every month
create 0664 root utmp
min size 1M
rotate 1
}
/var/log/btmp {
missing ok
every month
create 0600 root utmp
rotate 1
}
# system-specific logs may be also be configured here.
weekly: indicates rotation once a week, common usages include daily, weekly, monthly, yearly
rotate: number of log copies to keep
create: Create a new log file to accept newly generated logs during log rotation
dateext: use the date as the file suffix when cutting the log
compress: Whether to compress the log after rotation
include /etc/logrotate.d/: All configuration files in the /etc/logrotate.d/ directory will be loaded during log rotation (ps: the configuration in this directory must have rpm packages in the operating system)
The two logs /var/log/btmp and /var/log/wtmp have no corresponding rpm packages in the operating system, so they are regarded as “orphan” logs, and are configured in the global configuration of /etc/logrotate.conf.
Custom logrotate configuration
Some system services on Linux operations have been configured with log cutting rules by default, which can be viewed by viewing the files in the /etc/logrotate.d/ directory
[root@server ~]# ls /etc/logrotate.d/
bootlog chrony firewalld jenkins syslog wpa_supplicant yum
If a new application is deployed on the operating system, or you need to configure cutting rules for other log files, you can customize the configuration file of logrotate to make it effective. Here take the /var/log/audit/audit.log log as an example
[root@server ~]# cat /etc/logrotate.d/audit
/var/log/audit/*.log {
missingok # No error will be reported if the log is missing when the log is cut
weekly # cut once a week
rotate 10 # Keep up to 10 files after cutting
size + 100M # When the current log capacity exceeds 100M, immediately cut the log
compress # Compress the cut log
dateext # The log after cutting is suffixed with the time 'year-month-day'
notifempty # do not cut when the log is empty
create 0600 root root # Create a new log file when cutting, the mode is 0600, and the log group is root root
}
logrotate provides some command line parameters to test and view configuration and running results
# Test whether the syntax of the configuration file is legal [root@server ~]# logrotate --debug --force /etc/logrotate.d/audit reading config file /etc/logrotate.d/audit Allocating hash table for state file, size 15360 B Handling 1 logs rotating pattern: /var/log/audit/*.log forced from command line (10 rotations) empty log files are not rotated, old logs are removed considering log /var/log/audit/audit.log log needs rotating rotating log /var/log/audit/audit.log, log->rotateCount is 10 dateext suffix '-20211124' glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]' glob finding old rotated logs failed fscreate context set to system_u:object_r:auditd_log_t:s0 renaming /var/log/audit/audit.log to /var/log/audit/audit.log-20211124 creating new /var/log/audit/audit.log mode = 0600 uid = 0 gid = 0 compressing log with: /bin/gzip
parameter explanation
Compression
compress: use compression, the default compression method is gzip
compresscmd: custom compression command, the default compression method is gzip
uncompresscmd: custom decompression command, the default decompression tool is gunzip
compressext: Use the suffix when compressing, the default suffix is .gz in the gzip compression format
compressoptions: compression options, gzip is used by default; if you use other compression options, you need to set compressoptions to match it
delaycompress: delayed compression, the time when the actual compression takes effect occurs at the time of the next log cut
nodelaycompress: Do not use delayed compression, that is, compression during rotation (default configuration)
nocompress: does not use compression
Archive method
copy: Copy the complete log file during rotation, which is often used as a mirror backup of the current log file. When the copy option is specified, the create option will have no effect
copytruncate: Copy the complete log file and clear the original log file during rotation (equivalent to echo > logfile), and the newly written log will continue to be written to the cleared log file. But the logs written in the process of log copy and truncate may be lost. When the copytruncate option is specified, the create option will be disabled
nocopy: the original log file will not be copied during rotation
nocopytruncate: the content of the original log file will not be cleared after copying the original log file during rotation
create: Create a new log file during rotation, you can set the permission, owner, and group of the created file
nocreate: no new log files will be created during rotation
createolddir: If the specified directory does not exist during rotation, it will be created. It supports setting the permissions, owner and group of the directory
nocreateolddir: If the directory specified during rotation does not exist, it will not be created
Archive path
olddir directory: After the directory is configured, the rotated logs will be saved in the specified directory
noolddir: the rotated log is saved in the original directory of the log
Archive deletion
shred: use shred -u (shred delete) when deleting logs (default off)
shredcycles count: Before deleting the log, the overwrite log file will be rewritten first, and it will be deleted after reaching the set number of times
noshred: Use unlink to delete the log (I haven’t figured out the deletion logic yet)
Archiving Rules
hourly: Log cutting is performed every hour
daily: Log cutting is performed once a day
weekly: perform log cutting once a week
monthly: log cut once a month
yealy: Log cutting is performed once a year
size: cut according to log size
rotate count: the number of archive files saved during log rotation
start count: When the log is rotated, it starts from the specified count, for example, start 9, after the log is rotated, 0-8 will be skipped to generate a log like xxx.9,
maxage count: The rotated logs will be deleted after the set date, and only the rotated logs will be applied to this rule
maxsize size: After setting maxsize, if the size of the rotated log exceeds the set size, the log rotation will be performed before the set rotation time (for example: weekly)
minsize size: After setting the minsize, if the size of the rotated log does not meet the set size, even if the set rotation time is reached, the log rotation will not be triggered
Date format
dateext: The suffix name of the log after cutting is in the format of “YYYYMMDD”
nodateext: Do not use the format with the suffix name date when rotating the log
dateformat: Customize the time format of the log suffix, only %Y %m %d %H %s can be used. For example dateformat %m/%d/%Y
dateyesterday: Use yesterday’s time instead of today’s time when cutting logs
extension: Log files use the specified ext extension after rotation. If compression is used, usually ext will also add the extension of the compressed file, usually .gz. For example, if you have a log file named mylog.foo, you can use extension ext to rotate the log to mylog.1.foo.gz instead of mylog.foo.1.gz
Rotation rules
ifempty: when the log is empty, it will also be rotated according to the rules
notifempty: do not rotate when the log is empty
missingok: When the rotated log does not exist, continue to rotate the next log without error
nomissingok: the log does not exist during wheeling, and there will be an error message (default configuration)
mail configuration
mail address: After configuring the email address, the rotated log information will be sent to the mailbox
nomail: Do not send rotated log information to any mailbox (default configuration)
mailfirst: After the mailbox is configured, the log file generated this time will be sent to the mailbox after rotation
maillast: After configuring the mailbox, send the last rotated log file to the mailbox (default configuration)
Script to execute when archiving
include file_or_directory: Before the rotation, it will try to read the file or directory of the include configuration. If the configuration is a directory, all the files in the directory will be loaded into the rotation configuration; but for the file or configuration path whose file extension ends with taboo When loading configuration for multiple directories, pipelines, etc., it will be ignored
prerotate/endscript: A custom command (script) will be executed before the log is rotated. Typically, the full path to the rotated log is passed in as the first parameter
postrotate/endscript: After the log is rotated, a custom command (script) will be executed. Typically, the full path to the rotated log is passed in as the first parameter
firstaction/endscript: Execute this statement before executing prerotate/endscript and at least one log will be rotated. The entire mode will be passed to the statement as the first parameter. When the statement is executed abnormally, it will not be executed downward
lastaction/endscript: Execute the statement before executing postrotate/endscript and at least one of them has been rotated. The entire pattern will be passed to the statement as the first parameter. When the statement is executed abnormally, only the error message will be displayed
preremove/endscript: Execute this statement only before deleting the rotated log. The name of the log to be deleted will be passed into the statement as a parameter
sharedscripts: shared mode. After the sharing mode is enabled, when the prerotate and postrotate statements match multiple logs, the prerotate and postrotate statements will only be executed once. In normal mode, each log file matched during rotation will execute the prerotate and postrotate statements individually.
nosharedscripts: When multiple log files are matched during rotation, the prerotate and postrotate statements will be executed once for each log file. (default allocation)