Scheduled tasks and log management in Linux

Quickly view the command ctrl + f to search and you will directly locate the required knowledge points and command explanations (if there is anything incorrect, please feel free to comment in the comment area, and the editor will modify it in time)

Scheduled tasks

Concept:
It runs in the background and automatically executes tasks at the scheduled time. The premise is that the scheduled tasks need to be set manually in advance.
For example:
1. Periodic task execution
2. Clear the contents of the /tmp directory
3. Backup of mysql database
4. redis data backup

Scheduled execution once

Timing software under Linux
The at scheduled task tool relies on the atd service and is suitable for scheduling tasks that will be executed once and ended.

# Install at software
yum-y install at
# If the service is not enabled
systemctl start atd
#method one
Syntax format:
1. at time #Enter
2. Enter your scheduled task instructions
3. ctrl + D # The task is finished
# Time details
now + 1min # one minute later
HH:MM # hour:minute 10:30
YYYY-mm-dd # Year Month Day 2030-10-10
now + 1hours # one hour later
now + 1minutes # after 1 minute
now + 1days # one day later
now + 1weeks # one week later
# Method 2
at time < file
#case
# Create new files and store tasks
vim at.jobs
fill content
# Specify the task setting time
at 10:30 < at.jobs

# View one-time tasks
atq or at -l
# Cancel scheduled tasks
atrm serial number or at -d serial number
# For example
atrm 7 # Cancel the specified scheduled task

Cyclic scheduling tasks

cron is the name of the service software that periodically executes commands or specifies program tasks in the background process mode of the Linux system.
After the Linux system is started, the cronie software will start automatically. The default is to periodically (check once every minute) check whether there are any task plans that need to be executed in the system
If so, proceed as planned
Use cron tools to solve repetitive and periodic automatic backup tasks

Usage of crond

The way the crond process submits tasks is different from that of atd. crond needs to read the configuration file, and the configuration file has a fixed format to manage files through the crontab command.

# Install cronie software
yum -y install cronie
# Make sure the service is enabled
# storage location
/var/spool/cron
# Management method
crontab -l # List the current user’s scheduled tasks
crontab -r # Delete all scheduled tasks for the current user
crondtab -e # Edit the current user’s scheduled tasks
crondtab -u # Manage scheduled tasks for other users. Only root administrators.

System scheduled tasks

The cron service will check when working
|- Scheduled tasks in the /var/spool/cron folder
|- /etc/cron.d directory
|- /etc/anacrontab

# View the system's cycle scheduling tasks
[root@baiz etc]# ll /etc | grep 'cron*'
drwxr-xr-x. 2 root root 21 November 8 11:44 cron.d # System scheduled tasks
drwxr-xr-x. 2 root root 42 October 20 17:20 cron.daily # Daily tasks
drwxr-xr-x. 2 root root 22 November 8 11:13 cron.hourly # Hourly tasks
drwxr-xr-x. 2 root root 6 June 10 2014 cron.monthly # Monthly tasks
drwxr-xr-x. 2 root root 6 June 10 2014 cron.weekly # Weekly tasks
# Configuration file for crontab system scheduled tasks
SHELL=/bin/bash # interpreter
PATH=/sbin:/bin:/usr/sbin:/usr/bin #Command source
MAILTO=root # Who to send execution results to?

User scheduled tasks

Syntax:
crontab -e # Edit the current user's scheduled tasks
crontab -l # View the current user's periodic scheduled tasks
crontab -e # Delete a periodic scheduled task for the current user. After opening, just delete the specified line [Modify]
crontab -r # Delete all periodic scheduled tasks under the current user

# 1. Prepare the executable file to save the instructions (suffix is .sh)
vim /root/cronDemo.sh
File content format:
#!/bin/bash
mkdir /test4/fileDemo
touch /test4/fileDemo/a.txt
# 2. Edit periodic scheduled tasks
crontab -e
File content format:
* * * * * bash /root/cronDemo.sh
Use other users to create periodic scheduled tasks
# Execute periodic scheduled tasks on behalf of a user
crontab -u username -e # Based on root user
# View a user’s scheduled tasks
crontab -u username -l
#Delete periodic scheduled tasks for the specified user
crontab -u username -r

Core files
user rights file
/etc/cron.deny # Users listed in this file are not allowed to use the crontab command
/etc/cron.allow # The users listed in this file are allowed to use the crontab command. It does not exist by default and needs to be created manually (ignored)
/etc/crontab # Configuration file for system scheduled tasks

Case:
00 02 * * * Command 1 # Execute command 1 every day at two o'clock in the morning
00 02 14 2 * Command 3 # Execute command 3 at two o'clock in the morning on February 14th
00 02 * 6 5 Command 5 # Execute command 5 at two o'clock in the morning on Friday in June every year
00 02 1,5,8 * * Command 6 # Execute command 6 on the 1st, 5th, and 8th of each month at 2 o'clock in the morning

Log rotation and segmentation

The importance of logs:
Linux system logs are the main way for programmers to understand the operation of the system, so they need to have a detailed understanding of the Linux log system.
The Linux system kernel and many programs will generate various error messages, warning messages, and other prompt messages during their work. These messages are recorded in their respective log files.
The program rsyslog needs to be used to complete the logging process. rsyslog can save logs to different files according to the category and priority of the log.

Log management
rsyslog
Log rotation
logrotate

1. rsyslogd: Most of the log records related to the operating system, security, authentication, sshd su, scheduled tasks at cron...
2. httpd nginx mysql... can have its own logging method
/usr/local/nginx/logs/
|- access.log records log information of normal access responses
|- error.log records log information of abnormal access

Log configuration file
/etc/rsyslog.conf

Common log files (system, process, application)

# Dynamically view the tail of the system main log file
tail -f /var/log/messages
# Record authentication and security logs
tail -f /var/log/secure
# Related to email postfix
tail -f /var/log/maillog
# crond at the log generated by the process
tail -f /var/log/cron
# System startup related
tail -f /var/log/dmesg
# yum related logs
tail -f /var/log/yum.log
# directive
last # The most recently logged in user
\tcontent
User name Terminal location Login IP Start time End time Duration
# directive
lastlog # View the login status of all users

Log priority

Log levels are divided into: 9 types Log level code 0 ~ 7
0 debug # Record debugging information. The most log information
1 info # General information log most commonly used
2 notice # The most important information
3 warning # warning message
4 error # Error level information that prevents a certain function or software from working properly
5 crit # Severe error level information that prevents the entire system or the entire software from working properly
6 alert # Information that needs to be modified immediately
7 emerg # Kernel crash and other serious information
none # Log nothing

Log rotation and segmentation

1. If there is no log rotation, the log file will become larger and larger.
2. Lose old files in the system and save space (system protection mechanism)
3. Log files can be split (divided according to size 10MB/piece, rotated according to time 1 day/time, 1 week/time)

/etc/logrotate.conf # Configure log rotation

Case:
# Case 1 rotation log /var/log/yum.log
[root@bai logrotate.d]# vim /etc/logrotate.d/yum
/var/log/yum.log {<!-- -->
missingok
notifempty # Empty files are not rotated
size 30k # As long as it reaches 30kb, it will be rotated regardless of whether the rotation period is reached or not.
daily # rotate once a day
create 0600 root root
}
Case 2:
# Manual rotation
[root@bai /]# logratate -f rotate configuration file
# Configure nginx log rotation
access.log
error.log
/usr/local/nginx/logs/access.log{<!-- -->
missingok
notifempty
weekly
size 100k
create 0600 root root
}

Log rotation and cutting can solve the problem: a single log file will not be too large.
Generate multiple small log files (including backups) based on file size limits or rotation cycles
The Linux system will automatically clean up log files that have not been operated for a long time after a period of time.
If the circulating log files are still valuable, export them promptly and regularly.