This article analyzes Python’s logging module in detail, from basic introduction to practical applications and best practices. We explain how to use this module for logging efficiently and how to avoid common pitfalls through concrete code examples, aiming to help readers better master this powerful tool.
1. Introduction to Python log module
The concept of logs and their role in software development
During the development process, in order to record the running status of the application, we usually use the method of printing logs. This method can not only help us understand the running status of the software system, but also help us quickly locate the problem when an error occurs in the system.
For example, let’s say you have the following piece of code that simply outputs some information:
print("This is some information.")
Output:
This is some information.
However, if we need to record more complex information, such as error messages, warnings, or other important runtime information, just using print will be insufficient. This is where we need the logging module.
Introduction to Python logging module
Python’s built-in logging module provides us with a complete logging solution. In many cases, you may want your application to be able to output some form of status information while it is running, especially if the application needs to handle long-running tasks or when you are faced with a problem that needs to be diagnosed. The logging module is useful Is your right-hand man.
The logging module can help us capture, process and record log information, allowing us to quickly record log information anywhere the program is running. Compared with the simple print function, it is more flexible and can output logs to multiple places at the same time, such as: console, file, HTTP GET/POST, SMTP, Socket, etc., and the log level of each output can be set independently .
Here is a simple example to illustrate how to use the logging module:
import logging
# Create a logger and set the log level to INFO
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
# Add a StreamHandler to send log messages to console
console_handler = logging.StreamHandler()
logger.addHandler(console_handler)
# Log an informational message
logger.info("This is an informational message.")
This code will output the following information to the console:
This is an informational message.
Basic composition of logging module
The logging module mainly consists of the following parts:
Logger: used to provide an interface for direct use by applications.
Handler: Sends log records (generated by the logger) to the appropriate destination output.
Filter: Provides more granular tools for deciding which log records to output.
Formatter: Specifies the final output format of the log record.
2. Detailed explanation of logging module
Basic use of logging
Using Python’s logging module is fairly simple. Here is a basic example of how to create a log and output it to the console.
import logging
# This will log the message to the console
logging.warning('This is a warning message')
This code will output the following warning message:
WARNING:root:This is a warning message
Understanding log levels
In the logging module, we have 5 levels to describe the importance of logs. These levels are:
DEBUG: Detailed information, usually only used when diagnosing a problem.
INFO: Confirmation that things are working as expected.
WARNING: Something unexpected happened, or a problem may occur in the near future (for example, “Insufficient disk space”). But the software is still working fine.
ERROR: The software cannot perform certain functions due to a more serious problem.
CRITICAL: A serious error indicating that the program itself may not be able to continue running.
By default, the logging module records logs to the console and only processes logs with a level of WARNING or above.
Loggers, Handlers and Formatters
In this part we will explain in detail the three main components of Loggers, Handlers and Formatters.
The role and use of Loggers
Logger is a log object whose main task is to record logs. Anywhere in the application code that requires logging, you can create a logger instance and use it to record the required information. Here is a simple example of using logger:
import logging
# Create a logger
logger = logging.getLogger(__name__)
# Log some messages
logger.debug("This is a debug message.")
logger.info("This is an informational message.")
logger.warning("Careful! Something does not look right.")
logger.error("You have encountered an error.")
logger.critical("The program cannot recover from this situation!")
Note: When we run this code, we don’t see any output. This is because by default, the logger level is set to WARNING, so only logs with levels above WARNING will be processed.
Types and functions of Handlers
The Handler object is responsible for sending log records to the appropriate destination. Different handlers can send logs to the console, files, emails, and even HTTP POST parameters, etc. Here is a simple example of how to use a handler to log to a file and console:
import logging
# Create a logger
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
#Create a file handler
file_handler = logging.FileHandler('my_log.log')
logger.addHandler(file_handler)
# Create a console handler
console_handler = logging.StreamHandler()
logger.addHandler(console_handler)
# Log some messages
logger.debug("This is a debug message.")
logger.info("This is an informational message.")
logger.warning("Careful! Something does not look right.")
logger.error("You have encountered an error.")
logger.critical("The program cannot recover from this situation!")
Formatters functions and custom log formats
Formatter objects specify the final order, structure, and content of log records. You can customize the format of the log information to make the log information more readable. Here is an example of how to use formatter:
import logging
# Create a logger
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
# Create a console handler
console_handler = logging.StreamHandler()
#Create a formatter
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
# Add the formatter to the console handler
console_handler.setFormatter(formatter)
# Add the console handler to the logger
logger.addHandler(console_handler)
# Log some messages
logger.debug("This is a debug message.")
logger.info("This is an informational message.")
logger.warning("Careful! Something does not look right.")
logger.error("You have encountered an error.")
logger.critical("The program cannot recover from this situation!")
3. Application of Python log module in practice
Use logs to record exception information
In Python programming, it is often necessary to catch and handle exceptions. At this time, it is very convenient to use the logging module to record exception information. In the logging module, we can use the exception() method to record exception stack information. As shown in the following example:
import logging
logger = logging.getLogger(__name__)
try:
a = [1, 2, 3]
value = a[3]
except IndexError as e:
logger.error("Unhandled exception", exc_info=True)
```
When running this code, the logger will record the exception information that occurs, as follows:
```python
ERROR:__main__:Unhandled exception
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
IndexError: list index out of range
Use RotatingFileHandler for log rolling
When our application runs for a long time and generates a large amount of logs, writing all the logs to one file may cause the log file to be too large. At this time, we can use RotatingFileHandler for log rolling. When it reaches a certain size or a certain period of time, RotatingFileHandler will automatically back up the current log file and create a new log file to continue writing. As shown in the following example:
import logging
from logging.handlers import RotatingFileHandler
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
#Create a file handler
handler = RotatingFileHandler('my_log.log', maxBytes=2000, backupCount=10)
logger.addHandler(handler)
# Log some messages
for _ in range(10000):
logger.info("Hello, world!")
```
This code will create a new log file when the log file size reaches 2000 bytes and keep the latest 10 log files.
## Configure log level
Depending on our needs, the log level can be changed at runtime. For example, when we are debugging an application, we may need to output all levels of logs. But in a production environment, we may only care about error and above level logs. We can change the log level through the setLevel() function. As shown in the following example:
```python
import logging
# Create a logger
logger = logging.getLogger(__name__)
# Set log level to DEBUG
logger.setLevel(logging.DEBUG)
# Log some messages
logger.debug("This is a debug message.")
logger.info("This is an informational message.")
logger.warning("Careful! Something does not look right.")
logger.error("You have encountered an error.")
logger.critical("The program cannot recover from this situation!")
4. Best practices for Python logging module
Use __name__ to create a logger at the module level
In Python, the __name__ variable is a built-in variable that represents the name of the current module. When we create loggers on a per-module level and use __name__ as the name, we can easily track which module the logging occurred in.
import logging # Create a logger at the module level logger = logging.getLogger(__name__)
Use appropriate log level
Different log levels represent different severities. Correct use of log levels can help us find the information we care about in a large amount of logs. Generally speaking, for very serious errors, we should use CRITICAL or ERROR; for warning information, we should use WARNING; for general running information, we should use INFO; for debugging information, we should use DEBUG.
Use structured log messages
When our application has a large amount of logs, we may want to log the log messages in a parsable way. For example, we can log using JSON format. In this way, we can use various log analysis tools to analyze the logs.
import logging
import json
# Create a logger
logger = logging.getLogger(__name__)
# Log a structured message
logger.info(json.dumps({
'action': 'User login',
'username': 'user123',
'ip_address': '123.123.123.123',
'status': 'success',
}))
Use exception logging
When an exception is caught, we should use logger.exception() so that the complete exception stack information can be recorded in the log.
import logging
logger = logging.getLogger(__name__)
try:
x=1/0
except ZeroDivisionError:
logger.exception("Zero Division Error Caught.")
Such logs will contain enough information to help us find and fix the problem.
Do not record sensitive information in logs
Logs may be used by attackers to find vulnerabilities in the system, so we must not record sensitive information such as passwords, keys, and users’ private data in logs.
5. Summary
In this article, we introduce Python’s logging module in detail, including its basic introduction, detailed explanation, practical applications, and some best practices. To summarize the above:
- The logging module is a flexible and powerful logging tool built into Python. It can output information during program running to various output sources, such as standard output, files, emails, networks, etc.
- The logging module provides multiple levels of logging, including DEBUG, INFO, WARNING, ERROR, and CRITICAL. We can set different log levels according to needs to record and display information of different severities.
- In practice, we can use the logging module to log exception information, use RotatingFileHandler for log rolling, and change the log level at runtime.
- For best practices for logging modules, we mentioned creating loggers at the module level using __name__, using appropriate log levels, using structured log messages, using exception logging, and not logging sensitive information.
Python’s logging module is a very powerful tool. I hope you will have a deeper understanding and more flexible use after reading this article.
———————————END——————- ——–
Digression
Interested friends will receive a complete set of Python learning materials, including interview questions, resume information, etc. See below for details.
CSDN gift package:The most complete “Python learning materials” on the entire network are given away for free! (Safe link, click with confidence)
1. Python learning routes in all directions
The technical points in all directions of Python have been compiled to form a summary of knowledge points in various fields. Its usefulness is that you can find corresponding learning resources according to the following knowledge points to ensure that you learn more comprehensively.
2. Essential development tools for Python
The tools have been organized for you, and you can get started directly after installation! 
3. Latest Python study notes
When I learn a certain basic and have my own understanding ability, I will read some books or handwritten notes compiled by my seniors. These notes record their understanding of some technical points in detail. These understandings are relatively unique and can be learned. to a different way of thinking.
4. Python video collection
Watch a comprehensive zero-based learning video. Watching videos is the fastest and most effective way to learn. It is easy to get started by following the teacher’s ideas in the video, from basic to in-depth.
5. Practical cases
What you learn on paper is ultimately shallow. You must learn to type along with the video and practice it in order to apply what you have learned into practice. At this time, you can learn from some practical cases.
6. Interview Guide
CSDN gift package:The most complete “Python learning materials” on the entire network are given away for free! (Safe link, click with confidence)
If there is any infringement, please contact us for deletion.