Article directory
- Log file rotation (logrotate)
-
- Round Robin Fundamentals
- Basic configuration for round robin
- Actually test the operation of logrotate
-
- Use Cases
- Rotate function for custom log files
Log file rotation (logrotate)
Log rotation (logrotate) is a tool for managing log files. It can help us automatically cut, compress, and delete log files to save disk space and facilitate log file management. Here are some details about log rotation:
Basic principle of round robin
The log file is constantly recording information, and if it is not processed, it will grow in size and may eventually fill up disk space. The basic principle of round robin is to cut the log file at a certain moment, save the recorded information into a new file, and compress or delete the old log file at the same time. In this way, certain log information can be retained without wasting too much disk space.
Its execution structure is somewhat similar to the following figure
From the illustration above, we can clearly know that when is executed for the first time, the original log file will be backed up and renamed to messages.1, and then an empty messages file will be created to store the log. In the second execution, messages.1 will be backed up and renamed to messages.2, messages will be renamed to messages.1, and an empty messages file will be created to store the logs. If three log files are kept, the oldest log file messages.3 will be deleted and replaced by a new log file on the fourth execution. The file name after the latest log file rotation will already add the date parameter and keep it in the system to avoid data loss.
Basic configuration of round robin
How often to perform such a rotation task can be found in the logrotate.conf file.
[root@localhost ~]# vim /etc/logrotate.conf
# The following settings are the default settings of "logrotate", if other parameters are set for individual files
# It will be based on individual file settings. If the file is not set to the parameter, the content of this file will be used as the default value.
weekly <== By default, the log file is rotated once a week
rotate 4 <== Keep several log files, the default is to keep 4 1
create <== Since the log file is renamed, it means to create a new one to continue storage
dateext <== is this setting value, which can add the date to the name of the file being rotated.
#compress <== Whether the modified log files need to be compressed, if some log files are too large, you can consider using this parameter
include /etc/logrotate.d
# Read all the files in the /etc/logrotate.d/ directory to execute the rotation task.
/var/log/wtmp {<!-- --> <== only for the value set for /var/log/wtmp
monthly <== once a month, replace every week
create 0664 root utmp <== Specifies the permission and account/group of the newly created file
minsize 1M <== The file capacity must exceed 1M before round robin (skip the time parameter)
rotate 1 <== Only one is reserved, that is, only wtmp.1 is reserved.
}
# This wtmp can record the time when the registrant and the system restart, the source host and the time during the login
# Due to the minsize parameter, it may not necessarily be done once a month, depending on the file capacity.
# Since only one log file is kept, you can change it to rotate 5 if you are not satisfied.
From the settings of this file, we can know that /etc/logrotate.d is actually a directory planned by /etc/logrotate.conf. Although all data can be written into /etc/logrotate.conf, this file will be very complicated and inconvenient to maintain. Therefore, it is convenient and reasonable to separate the log file rotation settings of each service and place them in /etc/logrotate.d/ as a file. The default rotation status is set in /etc/logrotate.conf, but each service can set its own log file rotation.
For example, you can change rotate 4 to rotate 9 to save more backup files; for services such as httpd that take up a lot of hard disk space, you can consider compressing log files.
The setting of the file /var/log/wtmp is introduced above, and the setting syntax of logrotate.conf is:
The absolute path and file name of the log file ...{<!-- -->
Individual parameter setting values, such as monthly, compress, etc.
}
Next, let’s use /etc/logrotate.d/syslog to rotate the file of rsyslog.service service to see how to set up its rotation.
/var/log/cron
/var/log/maillog
/var/log/messages
/var/log/secure
/var/log/spooler
{<!-- -->
missing ok
shared scripts
postrotate
/bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
endscript
}
In the above syntax, we know that the correct way to write logrotate is:
-
File name: The absolute path of the log file to be processed is written in front of the file name, and multiple log files can be separated by spaces
-
Parameters: The parameters for the above file name to be cycled are included with {}
-
Execute scripts: External commands can be called for additional command execution. This setting can only be used in conjunction with the sharedscripts…endscript setting. As for the available environments are:
- prerotate: Commands performed before starting logrotate, such as modifying the properties of log files, etc.
- postrotate: A command started after logrotate is done, such as restarting (kill -HUP) a service
- Prerotate and prstrotate are very important execution programs for the processing of files that have been added with special attributes
Then the five file rotation functions set in /etc/logrotate.d/syslog become:
-
This setting is only valid for cron, maillog, messages, secure, spooler in /var/log/
-
The log file is rotated once a week, and 4 log files are retained and the rotated log files are not compressed (the default value has not been changed)
-
After the round robin (postrotate) obtains the PID of syslog, restart syslogd with kill -HUP
Suppose we add the chattr + a attribute to the file /var/log/messages, according to the working principle of logrotate, we know that this /var/log/messages will be renamed /var/log/messages.1. But due to the addition of this + a parameter, it is impossible to rename successfully. then what should we do? Just use prerotate and postrotate to perform the operations that need to be done before and after the log file rotation. Then you can modify the file like this
[root@localhost ~]# vim /etc/logrotate.d/syslog
/var/log/cron
/var/log/maillog
/var/log/messages
/var/log/secure
/var/log/spooler
{<!-- -->
shared scripts
prerotate
/usr/bin/chattr -a /var/log/messages
endscript
shared scripts
postrotate
/bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
/usr/bin/chattr + a /var/log/messages
endscript
}
Probably the process is to remove the a attribute first, so that the log file /var/log/messages can be rotated. Then after the round robin is executed, add this attribute to him.
/bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true Explanation:
The first part /bin/kill -HUPcat /var/run/syslogd.pid 2> /dev/null means sending a HUP signal to the process whose process number is /var/run/syslogd.pid, that is, restarting syslogd process.
Specifically, this command will first use the cat command to read the process number in the /var/run/syslogd.pid file, and then pass the process number to The /bin/kill command sends a HUP signal to the process. 2> /dev/null indicates that the standard error output is redirected to a null device to prevent error messages from being displayed. If the cat command or the /bin/kill command executes incorrectly, the process ID does not exist or the process has exited, etc., this command will not output any error messages.
The second part 2> /dev/null || true indicates that the standard error output is redirected to a null device. If the command execution fails and the return value is non-zero, true will be executed code> command, that is, no processing is done, and the return value is zero.
The function of this command is to ensure that even if there is an error restarting the syslogd process, it will not cause the entire script to fail.
Actually test the operation of logrotate
In the Linux system, we can rotate the log files by manually executing the logrotate command.
logrotate [-vf] logfile options: -v: Start the display mode, which will display the process of logrotate running -f: Regardless of whether it conforms to the data in the configuration file, each log file is forced to perform a round-robin operation.
Use case
Execute a logrotate to see what the whole process is?
[root@localhost ~]# logrotate -v /etc/logrotate.conf reading config file /etc/logrotate.conf # indicates that the main configuration file is being read; including /etc/logrotate.d # is reading the configuration file in the directory reading config file bootlog # is reading the bootlog configuration file in the directory reading config file chrony # is reading the chrony configuration file in the directory ?… ?… ?… Handling 8 logs # A total of 18 log files are recorded ?… ?… ?… rotating pattern: /var/log/cron /var/log/maillog /var/log/messages /var/log/secure /var/log/spooler weekly (4 rotations) empty log files are rotated, old logs are removed considering log /var/log/cron log does not need rotating (log has been already rotated) considering log /var/log/maillog log does not need rotating (log has been already rotated) considering log /var/log/messages log does not need rotating (log has been already rotated) considering log /var/log/secure log does not need rotating (log has been already rotated) considering log ?… ?… ?…
Force the operation of logrotate
[root@localhost ~]# logrotate -vf /etc/logrotate.conf ?… ?… ?… rotating log /var/log/messages, log->rotateCount is 4 dateext suffix '-20230518' glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]' glob finding old rotated logs failed ?… ?… # see it? The entire round-robin operation is carried out step by step in this way [root@localhost ~]# ll /var/log/messages* ; lsattr /var/log/messages -rw-------. 1 root root 155 May 18 23:19 /var/log/messages -rw-------. 1 root root 488944 May 18 23:11 /var/log/messages-20230518 -----a---------- /var/log/messages # Actively add the hidden attribute of a
Which of the above -f has the meaning of [enforcement], if all settings are correct, then in theory, your /var/log directory will change, and there should be no error messages.
Let’s do some sample exercises to understand the logrotate function in more detail
Customize the rotation function of the log file
If you want to additionally write [all information] to /var/log/admin.log, how should you configure it? This file is now, you want to add the hidden attribute + a to the file, and set the following relevant information
- Log files are rotated one by one
- If the log file is larger than 10MB, it will take the initiative to rotate without considering the one-month period
- Save 5 backup files
- Backup files need to be compressed
How should I set it up?
- First set the parameters that need to be added
[root@localhost ~]# vim /etc/rsyslog.conf # Add by localhost 2023/5/17 # Add some instructions when modifying by yourself *.info /var/log/admin.log
- restart rsyslog.server
[root@localhost ~]# systemctl restart rsyslog.service [root@localhost ~]# ll /var/log/admin.log -rw-------. 1 root root 816 May 17 10:18 /var/log/admin.log # You can see that the log file has been created # In this way. All information will be written to admin.log
- Add + a attribute first
[root@localhost ~]# chattr + a /var/log/admin.log [root@localhost ~]# lsattr /var/log/admin.log -----a---------- /var/log/admin.log [root@localhost ~]# mv /var/log/admin.log /var/log/admin.log.1 mv: cannot move "/var/log/admin.log" to "/var/log/admin.log.1": Operation not permitted # Here, the hidden attribute of adding a is determined, so root cannot move this log file
- Start to create a logrotate configuration file, just add a file to the /etc/logrotate.d directory
[root@localhost ~]# vim /etc/logrotate.d/admin
/var/log/admin.log {<!-- -->
every month
size=10M
rotate 5
compress
shared scripts
prerotate
/usr/bin/chattr -a /var/log/admin.log
endscript
shared scripts
postrotate
/bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
/usr/bin/chattr + a /var/log/admin.log
endscript
}
- Test the display information of logrotate related functions
[root@localhost ~]# logrotate -v /etc/logrotate.conf ?… ?… rotating pattern: /var/log/admin.log 10485760 bytes (5 rotations) empty log files are rotated, old logs are removed considering log /var/log/admin.log log does not need rotating (log size is below the 'size' threshold) not running prerotate script, since no logs will be rotated not running postrotate script, since no logs were rotated ?… ?… # Because it is less than a month, the file is not larger than 10MB, so there is no need to rotate
- Test the display of forced logrotate and related functions
[root@localhost ~]# logrotate -vf /etc/logrotate.conf [root@localhost ~]# lsattr /var/log/admin.log* -----a---------- /var/log/admin.log ---------------- /var/log/admin.log-20230518.gz
see it? In this way, we can create our own logrotate configuration file, which is very simple and convenient. In particular, note that the /etc/rsyslog.conf and /etc/logrotate.d/* files are often used in conjunction.