[Shell command collection system management] Linux automatic rotation (log rotation) log file logrotate command user guide

Directory title

  • Description
    • grammar format
    • parameter description
    • Error condition
  • Notes
  • Bottom layer implementation
  • Example
    • Example 1
    • Example 2
    • Example 3
    • Example 4
    • Example 5
    • Example 6
    • Example 7
  • Implemented in c language
  • Conclusion

Shell Command Column: Full Analysis of Linux Shell Command

description

logrotate is a log file management tool in the Linux system. It is used to automatically rotate (log rotation) log files in order to control the size and retention period of log files, and can also compress and delete old log files.

The role of logrotate mainly has the following aspects:

  1. Control log file size: Log files can grow over time and can take up too much disk space if not managed. Logrotate can limit the size of each log file by periodically rotating log files to achieve the purpose of controlling the size of log files.

  2. Retain log history: In addition to controlling the size of log files, logrotate can also set the number or retention time of log files. When the specified conditions are met, the old log files will be deleted or compressed, thereby retaining a certain log history.

  3. Avoid large log files affecting performance: When a log file is too large, the operations of reading and writing the file may become slow, affecting system performance. By rotating the log files regularly, you can keep the log files from getting too large and maintain the normal operation of the system.

  4. Compress old log files: logrotate can compress old log files to reduce disk space usage. Compressed log files can be further archived or transmitted for subsequent analysis and use.

  5. Automated management: logrotate can be set through configuration files to automate the management of log files. The conditions and behaviors of rotation can be customized according to requirements to suit different log file management requirements.

In general, logrotate is a very practical tool, which can help system administrators to effectively manage log files, control the size of log files, keep a certain log history, avoid the impact of excessive log files on system performance, and can Old log files are compressed and deleted to save disk space.

Grammar format

logrotate [OPTIONS] CONFIG_FILE

parameter description

  • -d, --debug: enable debug mode, output more detailed debugging information.
  • -f, --force: Force rotation even if log files do not need to be rotated.
  • -v, --verbose: Display verbose output for verbose operations.
  • -s, --state STATE_FILE: Specifies the path of the state file, which is used to record the state of the last rotation.
  • -c, --create: Create a new empty log file after rotation.
  • -m, --mail MAIL_COMMAND: Execute the specified mail command after rotation, and send the rotation result to the specified mailbox.
  • -n, --no-cleanup: Do not delete old rotated log files.
  • -t, --test: Test the syntax and rotation of the configuration file, but do not perform the actual rotation.
  • -v, --version: Display the version information of logrotate.
  • -h, --help: Display the help information of logrotate.

Error

  • If the specified configuration file does not exist or cannot be read, the logrotate command will report an error.
  • The logrotate command may fail if the target log file is being written by another process when the rotation operation is performed.
  • The logrotate command will also fail if the specified directory does not exist or is inaccessible when performing the rotation operation.

Notes

There are a few things to consider when using the logrotate command:

  1. Correctness of configuration files: logrotate uses configuration files to define rotation rules and behaviors. When creating the configuration file, make sure the syntax is correct, the parameters are set appropriately, and the paths and file names are correct. Each log file path in the configuration file should be valid and have appropriate read and write permissions.

  2. Backup and restore: Before performing log rotation, make sure you have backed up important log files. The rotated log files may be compressed, deleted, or renamed, so it is important to be able to easily revert to the previous log files if needed.

  3. Regularly check the rotation effect: Regularly check the effect of the rotation operation to ensure that the log files after rotation meet expectations. Check the size, number, and content of the log files to ensure that the rotation rules and settings are correct.

  4. Log file permissions and ownership: Make sure the log file permissions and ownership are set correctly. If logrotate is run as root, the rotated log file may be owned by root, which may prevent other users from accessing or writing to the file.

  5. Log file writing and reading: Make sure that no other process is writing or reading the log file while the rotation is in progress. Otherwise, the rollover operation may fail or result in data loss.

  6. Regularly monitor the execution of logrotate: logrotate can be set to execute automatically on a regular basis, but it still needs to monitor its execution regularly. Check logrotate’s log files and output to make sure it is performing as expected and that no errors or exceptions occur.

  7. Backup and compression strategy: According to actual needs, formulate a suitable backup and compression strategy. You can choose whether to compress the old log files according to the rotation rules and the requirements of keeping history, and restore them when needed.

In general, when using the logrotate command, you need to carefully consider the correctness of the configuration file, backup and recovery strategies, log file permissions and ownership, log file writing and reading, regular monitoring of logrotate execution, and backup and recovery. Matters needing attention such as compression strategy to ensure the smooth progress of the log rotation operation.

Bottom layer implementation

The underlying implementation of the logrotate command mainly depends on the following aspects:

  1. Configuration file parsing: logrotate will first parse the specified configuration file and read the rotation rules and settings in it. It will analyze information such as the path of each log file, the rotation condition, and the storage strategy to determine how to perform the rotation operation.

  2. File renaming and backup: Before performing the rotation operation, logrotate will rename or backup the current log file. This is usually accomplished by renaming the original log file to a new filename or copying it to a backup directory. This allows the original log files to be preserved and restored if needed.

  3. Create a new empty log file: After the rotation operation is complete, logrotate will create a new empty log file to continue recording new logs. This ensures continuous logging and no log entries are lost.

  4. Compress old log files: According to the settings in the configuration file, logrotate can choose to compress the old log files after rotation. This is usually achieved by using gzip or other compression tools to reduce disk space usage.

  5. Delete old log files: According to the settings in the configuration file, logrotate can choose to delete the old log files after rotation. This is usually dictated by a retention policy, which can determine the number of log files to keep based on time, volume, or other criteria, and delete older files that exceed this limit.

  6. Perform follow-up operations: After the rotation operation is completed, logrotate can also perform some follow-up operations, such as executing the specified script, sending an email to notify the rotation result, and so on. These operations can be performed according to the settings in the configuration file.

In general, the underlying implementation of the logrotate command mainly implements the log rotation function by parsing configuration files, renaming and backing up log files, creating new empty log files, compressing old log files, deleting old log files, and performing follow-up operations. . These operations can effectively manage and control the size of log files, preserve history and save disk space.

Example

Example 1

  • Function: Rotate the specified log file and keep the last 5 rotated log files.
  • Sample configuration file:
/var/log/nginx/error.log {
    rotate 5
}

Example 2

  • Function: Rotate all log files with the suffix .log in the specified directory, and keep the last 10 rotated log files.
  • Sample configuration file:
/var/log/app/*.log {
    rotate 10
}

Example 3

  • Function: Rotate the specified log file and compress the old log file after rotation.
  • Sample configuration file:
/var/log/syslog {
    rotate 7
    compress
}

Example 4

  • Function: Rotate the specified log file, and move the old log file to the specified directory after rotation.
  • Sample configuration file:
/var/log/auth.log {
    rotate 4
    move /var/log/old_logs
}

Example 5

  • Function: Rotate the specified log file and execute the specified script after rotation.
  • Sample configuration file:
/var/log/application.log {
    rotate 3
    postrotate
        /usr/local/bin/custom_script.sh
    endscript
}

Example 6

  • Function: Rotate the specified log file, and forcibly rename the old log file after rotation to prevent it from being written by other processes.
  • Sample configuration file:
/var/log/application.log {
    rotate 5
    copytruncate
}

Example 7

  • Function: Rotate the specified log file and create a new empty log file after rotation.
  • Sample configuration file:
/var/log/application.log {
    rotate 4
    create
    missing ok
}

Implemented in c language

The following is a sample code that uses C language to implement a simplified version of the logrotate command, and the code contains detailed annotations.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>

int main() {<!-- -->
    // Specify the log file path and rotation conditions
    char logFile[] = "/var/log/application.log";
    int maxFileSize = 1000000; // 1MB

    // Get log file size
    struct stat st;
    if (stat(logFile, &st) == -1) {<!-- -->
        perror("Failed to get file size");
        exit(1);
    }
    off_t fileSize = st.st_size;

    // Determine if rotation is required
    if (fileSize >= maxFileSize) {<!-- -->
        // Rename or backup log file
        char backupFile[256];
        snprintf(backupFile, sizeof(backupFile), "%s.bak", logFile);
        if (rename(logFile, backupFile) == -1) {<!-- -->
            perror("Failed to rename file");
            exit(1);
        }

        // create new empty log file
        FILE *newLogFile = fopen(logFile, "w");
        if (newLogFile == NULL) {<!-- -->
            perror("Failed to create new log file");
            exit(1);
        }
        fclose(newLogFile);

        // compress old log files (optional)
        //...

        // delete old log files (optional)
        //...
    }

    return 0;
}

The above sample code is a simplified implementation of the logrotate command, which determines whether rotation needs to be performed by obtaining the size of the log file. If the log file size exceeds the specified threshold, the original log file is renamed or backed up, and a new empty log file is created. In addition, logic for compressing old log files and deleting old log files can also be added to the code as needed.

Please note that this is just a simplified sample code, the actual function of the logrotate command is more complex, including configuration file parsing, status recording, follow-up operations, etc. To implement the full logrotate command, more code and logic is required.

Conclusion

During our exploration, we have gained a deep understanding of the powerful functions and wide applications of Shell commands. However, learning these techniques is just the beginning. The real power comes in how you incorporate them into your daily work to increase efficiency and productivity.

Psychology tells us that learning is a process of continuous and active participation. So, I encourage you not only to read and understand these commands, but also to practice them. Try creating your own commands and gradually master shell programming as part of your daily routine.

At the same time, please remember that sharing is a very important part of the learning process. If you find this blog helpful to you, please like it and leave a comment. Sharing your own problems or interesting experiences when using Shell commands can help more people learn from them.
In addition, I also welcome you to bookmark this blog and come back to check it at any time. Because review and repeated practice are also the key to consolidating knowledge and improving skills.

Finally, please remember: everyone can become an expert in Shell programming through continuous learning and practice. I look forward to seeing you make even more progress on this journey!

Read my CSDN homepage to unlock more exciting content: Bubble’s CSDN homepage