Linux: Logging system rsyslog and log rotation logrotate

1. Log system rsyslog

1.1. Importance of logs

Linux systems and many programs will generate various error messages, warning messages and other prompt messages. These various messages should be recorded in log files. The program that completes this process is rsyslog. Linux system logs are the main way for administrators to understand the operation of the system, so they need to have a detailed understanding of the Linux log system.

1.2. Basics of log management

rsyslog-log management service (questions of concern: what type of program, what logs are generated, and where to put them)
logrotate-log rotation tool (split and manage a large number of logs, delete old logs)

1.3. Log processing process

rsyslogd: Most of the log records are related to system operations, security, authentication sshd, su, scheduled tasks at, cron… httpd/nginx/mysql: You can do it in your own way logging

[root@localhost ~]# ps aux |grep rsyslogd
root 752 0.0 0.2 216440 5548 ? Ssl 20:18 0:00 /usr/sbin/rsyslogd -n

1.4. Common log files (system, process, application)

Dynamicly view the tail of the log file (tail -f)
[root@localhost ~]# tailf /var/log/messages

Log file	Explanation
/var/log/messages	System main log file
/var/log/secure	Record authentication and security logs
/var/log/maillog	Related to mail postfix
/var/log/cron	Logs generated by crond and at processes
/var/log/dmesg	Record hardware-related information logs loaded when the system starts
/var/log/yum.log	yum’s log
/var/log/mysqld.log	MySQL log
/var/log/xferlog	Related to accessing the FTP server
/var/log/boot.log	System startup process log record storage
/var/log/wtmp	Currently logged in users (you can directly enter w on the command line to view)
/var/log/lastlog	The login status of all users (you can Enter lastlog directly on the command line to view)

1.5. Website log management

1.Installation
[root@localhost ~]# yum install -y httpd //Install the http process
[root@localhost ~]# systemctl start httpd //Start the http process
[root@localhost ~]# systemctl stop firewalld //Turn off the firewall
2. Adjust the network
[root@localhost ~]# systemctl restart network
[root@localhost ~]# ip a //ens below
3. Open the browser and enter your server IP to browse
4. Observation log:
[root@localhost ~]# tailf /var/log/httpd/access_log

1.6.rsyslogd configuration

1.Related procedures
[root@localhost ~]# yum install rsyslog logrotate //Installed by default
2. Start the program
[root@localhost ~]# systemctl start rsyslog.service
3.Related documents
[root@localhost ~]# rpm -qc rsyslog //Observe the configuration file of the log program

/etc/rsyslog.conf: main configuration file of rsyslogd (key)
/etc/sysconfig/rsyslog: rsyslogd related files, definition level (learn about it)
/etc/logrotate.d/syslog: related to log rotation (cutting)

1.7. Main configuration file

(Tell the rsyslogd process what logs should be saved and where they should be saved)

[root@localhost ~]# vim /etc/rsyslog.conf

RULES
That is, rules are a set of strategies for generating logs and storing logs; they consist of three parts (composed of device + level + storage location), FACILITY + LEVEL + FILE

–	–
authpriv.*	/var/log/secure (SSH information)
mail.*	-/var/log/maillog (send mail)
cron.*	/var/log/cron (create task)

There is a - symbol here, indicating that it is recorded in an asynchronous manner, because the log is generally relatively large.
*.info;mail.none;authpriv.none;cron.none /var/log/messages : System log excludes email, authentication, and schedule logs

FACILITY & amp;LEVEL
Facility device: It is the system’s definition of a certain type of APP event. For example, AUTHPRIV is a security event, and CRON is a planned task event. Used to collect similar program logs.

[root@localhost ~]# man 3 syslog

Device Type	Explanation
LOG_SYSLOG	Logs generated by syslogd itself
LOG_AUTHPRIV	Security authentication
LOG_CRON	Scheduler (cron and at)
LOG_MAIL	Mail system mail subsystem
LOG_USER (default)	User-related
LOG_DAEMON	Background process
LOG_FTP	File server ftp daemon
LOG_KERN	Kernel device kernel messages
LOG_LPR	Printer device printer subsystem
LOG_LOCAL0 through LOG_LOCAL7	User-defined device

level level	Explanation
LOG_EMERG	Fatal, the service cannot continue to run, such as the configuration file is lost
LOG_ALERT	Alarm, which needs to be handled immediately, such as the disk is empty and used 95%
LOG_CRIT	Emergency behavior
LOG_ERR	Error behavior
LOG_WARNING	Warning information
LOG_NOTICE	Common, important standard information
LOG_INFO	Standard information
LOG_DEBUG	Debugging information, required for troubleshooting, Generally not recommended to use

From bottom to top, from low to high levels, less and less information is recorded
Rule diagram:

Program type examples
Regarding the connection between the program and the device, the program itself will decide which type of device to send the log to. For example, the SSH program will select a security device. (This is defined by the developer)

1. Modify the device type of the ssh program
[root@localhost ~]# vim /etc/ssh/sshd_config
#SyslogFacility AUTHPRIV
SyslogFacility LOCAL5

2. Modify the rules of the rsyslog program
[root@localhost ~]# vim /etc/rsyslog.conf
local5.* /var/log/serverzz

3. Restart the rsyslog program and ssh program
[root@localhost ~]# systemctl restart rsyslog.service sshd
4. Use another terminal to log in to the server and observe the new log files.

2. Log rotation (cutting) logrotate

Note: You can target any log file (rsyslog log, Nginx access or error log…)
Without log rotation, log files will grow larger and larger
The oldest log files in the system will be discarded to save space
logrotate is executed every day by cooperating with the scheduled task crond

2.1.logrotate configuration file

Main configuration file: /etc/logrotate.conf (determines how each log file is rotated)
Subconfiguration file: /etc/logrotate.d/*

2.2. Configuring log rotation

[root@localhost ~]# vim /etc/logrotate.conf
Weekly //The cycle of rotation, one week rotation, the unit is year, month, day
rotate 4 //retain 4 copies
create //Create a new file after rotation
dateext //use date as suffix
#compress //During log rotation, old logs are compressed
include /etc/logrotate.d //Includes the configuration files in this directory and will reference the files configured in this directory.
/var/log/wtmp {<!-- --> //How to set rotation for this log file
 monthly //Rotate once a month
 minsize 1M //The minimum value is 1M before rotation, otherwise it will not rotate even if the time is up.
 create 0664 root utmp //Create a new file after rotation and set permissions
 rotate 2 //retain 2 copies
}
/var/log/btmp {<!-- -->
 missingok //no prompt if lost
 monthly
 create 0600 root utmp
 rotate 1
}

2.3. Example 1: Rotate file /var/log/yum.log

[root@localhost ~]# vim /etc/logrotate.d/yum
/var/log/yum.log {<!-- -->
 missingok //Do not remind if lost
 #notifempty //Empty files are not rotated
 #size 30k //As long as it reaches 30k, it will be rotated (size defaults to maxsize)
 #yearly
 daily //Rotate once a day
 rotate 3
 create 0644 root root
}

test:

[root@localhost ~]# /usr/sbin/logrotate -f /etc/logrotate.conf //Manual rotation
[root@localhost ~]# ls /var/log/yum*
/var/log/yum.log /var/log/yum.log-20191110

[root@localhost ~]# grep yum /var/lib/logrotate/logrotate.status //View the latest rotation time of all log files
"/var/log/yum.log" 2019-11-10-21:26:14

2.4. Example 2: Log rotation data_log

[root@localhost ~]# mkdir -p /data/logsfile/log/

Generate log file contents:

[root@localhost ~]# touch /data/logsfile/log/{<!-- -->a..z}.log
for a in {<!-- -->1..10};do
for i in {<!-- -->a..z};
do cat /var/log/vmware-vmsvc-root.log >>/data/logsfile/log/$i.log;
done;
done

Edit the log rotation file:

[root@localhost ~]# vim /etc/logrotate.d/data_log
/data/logsfile/log/*.log
{
 missingok
 daily
 dateext
 compress
 rotate 3
 create 0644 root root
}