4. Log writing – TinyWebServer

Log writing – TinyWebServer

1. Foreword

I have finished writing the lock class in the last issue. This issue is about diary writing.
For logs, you need to understand several basic concepts to better understand and write logs.

  1. What is a log?
  2. What are the commonly used log levels?
  3. What is the basic format of the log?
  4. The role of asynchronous log class refresh buffer
  5. What does synchronous and asynchronous logging mean?
  6. What is singleton pattern?
  7. Why use singleton pattern?
  8. What are the common implementation methods of singleton pattern?

These are all points that the author, as a novice, didn’t understand at first. If you have any other questions, please leave a message for discussion!

2. Answers to questions

I. What is a log?

Logs are automatically created by the server and record running status, error information, and access data files.

II. What are the commonly used log levels?

  • DEBUG: Used to record detailed debugging information, usually only used during the development and debugging phases to help developers identify and solve problems. These logs typically contain the program’s internal state and variable values.

  • INFO (information): Used to record general information when the program is running, such as the startup and shutdown of the application, the occurrence of important events, and the results of some key operations. INFO level logs are usually used to track the normal operation of the application.

  • WARNING: Used to record problems that may affect the normal operation of the application but are not fatal. Warning level logs usually indicate potential problems or anomalies, but the application can still continue to run.

  • ERROR: Used to log error events that may prevent an application from functioning properly or performing critical operations. ERROR level logs usually require developer attention and intervention.

  • CRITICAL: Used to log critical error events that may cause the application to crash or prevent it from continuing to run. CRITICAL level logs usually indicate the need for immediate action to fix the problem.

  • FATAL (fatal): Some log systems also include the FATAL level, which is usually similar to the CRITICAL level, indicating that the application has encountered an unrecoverable error and needs to be processed immediately.

The current project involves only the first 4 levels

III. What is the basic format of the log?

  • Timestamp: Record the date and time when the log event occurred, usually expressed in a standard date and time format, such as ISO 8601 format (for example: 2023-09-12T14:30:00Z).

  • Log level: Indicates the severity and urgency of the log message, such as DEBUG, INFO, WARNING, ERROR, or CRITICAL.

  • LogSource: Indicates the name of the component, module, or system that generated the log message, allowing you to identify the source of the log message.

  • Message content: Contains specific information describing the log event, typically a block of text that can include details about the event, error messages, warnings, or other relevant data.

  • Additional data: Optional, includes additional data related to the log event, usually in the form of key-value pairs to provide more contextual information. For example, you can log the value of a specific variable or other relevant properties of the event.

Here is a simple example log entry showing the basic format of the above elements:

2023-09-12T14:30:00Z INFO MyApp - User 'Alice' logged in successfully.

4. The role of asynchronous log refresh buffer

  1. Reduce I/O overhead: Asynchronous logging usually stores log messages in a memory buffer and then asynchronously writes the log data in the buffer to disk or other storage media. This can reduce frequent disk I/O operations and improve performance, because disk I/O is usually a relatively slow operation. However, if the buffer is not refreshed in time, a large amount of log data may remain in the memory, increasing the I/O burden during refresh.

  2. Ensure log integrity: If you do not flush the buffer regularly, but wait for the program to end before flushing, then when the program crashes or exits abnormally, the log data that has not been flushed may be lost. Manually flushing the buffer ensures that data that has been written to the buffer is written to disk in a timely manner during different stages of the application or under abnormal circumstances to maintain log integrity.

  3. Log real-time requirements: Some applications have high real-time requirements for log messages, that is, they require log messages to be written to disk or other storage media immediately for real-time monitoring or auditing. In this case, manually flushing the buffer is an important step to ensure that log messages are written in a timely manner.

It should be noted that in the asynchronous log class, the timing of flushing the buffer is usually determined by a certain strategy or trigger condition, rather than manually flushing after each log message. Common strategies include:

  • Scheduled refresh: Refresh the buffer regularly to ensure that log data is written to disk regularly. For example, refresh every certain time or every second.

  • Buffer size limit: When the buffer reaches a certain size, the refresh operation is automatically triggered to avoid the buffer becoming too large.

  • Flush before closing the program: Ensure that the data in the buffer is flushed to disk when the program is about to shut down gracefully.

Flushing buffers is an important design consideration in asynchronous logging implementations to balance performance with the reliability and real-time nature of log data. Different application scenarios and performance requirements may lead to different refresh strategies.

V. What does synchronous and asynchronous log writing mean?

  1. Synchronized log writing:
  • Synchronous log writing means that in the execution process of the application, when a log message is issued, the program will wait for the log message to be written to the target storage (usually a file or database) before continuing to perform subsequent operations.
    This means that the application blocks while logging each log message until the write operation completes and then can continue.
  • Writing logs synchronously can ensure the integrity and reliability of log messages, but may have a negative impact on application performance, especially in high concurrency or high throughput situations.
  1. Asynchronous log writing:
  • Asynchronous log writing means that in the execution process of the application, when a log message is issued, the program does not wait for the log message to be written to the target storage, but immediately continues to perform subsequent operations.
  • Writing log messages occurs in the background or in a separate thread to reduce the performance impact on the main application.
  • Asynchronous logging is generally more suitable for applications with high performance requirements because it reduces the latency caused by waiting for disk or network write operations.

Choosing synchronous or asynchronous logging depends on your application’s needs and performance requirements:

  • If the application requires high reliability and ensuring that every log message is logged, and performance requirements are not a primary concern, then synchronous logging may be more appropriate.

  • If the application has high performance requirements and can tolerate that some log messages may be lost or delayed in writing, then asynchronous log writing may be more suitable because it can reduce the impact on application performance.

A common asynchronous log writing strategy is to use a buffer to first store multiple log messages in memory and then write them to the target storage in batches regularly to reduce the frequency of write operations and improve performance. . However, it should be noted that in asynchronous logging, exceptions should be handled carefully to ensure that important log information is not lost.
Here is a quote from the president’s picture. Now everyone should understand

VI. What is singleton mode?

One of the most commonly used design patterns ensures that a class has only one instance and provides a global access point to it, which instance is shared by all program modules.

VII. Why use singleton mode?

Using singleton mode logging classes can ensure that logging is used and managed consistently throughout the entire application, improving code maintainability and scalability. At the same time, it also helps you have better control over the configuration and performance of logging.

VIII. What are the common implementation methods of singleton mode?

Lazy Man Style and Hungry Man Style are the two most common implementation methods.

  1. Lazy Initialization: Create an instance only when the instance is requested for the first time, lazy initialization
  2. Eager Initialization: Create an instance when the class is loaded and initialize it immediately.

3. Code structure analysis

To write this project well, it should be written according to the following ideas

  1. This project can be used both synchronously and asynchronously. Among them, asynchronous requires the use of blocking queue.

  2. Implementation of blocking queue

    • Introducing “locker.h”: Blocking queue, when adding to the blocking queue and writing to the file, only one thread is allowed to operate. Multiple threads at the same time can cause data errors.
    • Use templates: The blocking queue is not sure what type of data to write, so it uses templates.
    • Member functions: Current length, maximum length, head, tail, Circular queue
    • Member variables: Constructor, destructor, get head element, get tail element, empty test, full test, add, head deletion, timed head deletion, clear

  3. Log implementation

    • Introducing “block_queue.h”:

    • Use singleton mode: The lazy mode is used here

    • Use variadic macros: Flexible and convenient way to handle different numbers and types of log messages.

    • **Member functions:** Constructor, destructor, synchronous writing, asynchronous writing, initialization, buffer refresh

    • Member variables:

      • Related to writing files:

        1. Open the file pointer of the log
        2. pathname
        3. log file name
        4. Maximum number of log lines
        5. Number of log lines recorded
        6. Log buffer size
        7. Because it is classified by day, the current time is recorded on that day

      • Whether to synchronize the flag bit

      • blocking queue

      • Close log

For specific code implementation, just look at the warehouse. I won’t go into details.

4. Preview of the next issue

Writing data connection pool

Five. Finally

If it helps, please like it!