Implement global exception handling mechanism in Python projects without specific framework constraints

This blog is reprinted from my personal blog: https://jarlor.github.io/2023/10/26/python-cataching-global-exception/

Preface

Global exception handling mechanisms are usually built into mature web frameworks (such as Flask and Django) or are supported by specialized error monitoring and logging platforms (such as Sentry). However, these outstanding frameworks are not designed for global exception handling, and introducing them as just one mechanism in a non-framework-targeted Python project may be too unwieldy. This article aims to create a custom global exception handling mechanism in two ways, compare the differences and connections between the two ways, and give specific application scenarios.

Prerequisite knowledge

  • Decorator
  • Exception handling in python

Implement global exception handling mechanism

This chapter will provide two implementation methods, namely implementation based on hook function and implementation based on decorator.

Expected goals

Trigger exceptions anywhere in the main module of the project and handle them in one place.

Preparatory phase

Project structure

First clarify the project structure:

.
├── by_decorator
│ ├── __init__.py
│ ├── class_1.py
| ├── run.py
│ └── sub_module
│ ├── __init__.py
│ └── class_2.py
├── by_hook_func
│ ├── __init__.py
│ ├── class_1.py
| ├── run.py
│ └── sub_module
│ ├── __init__.py
│ └── class_2.py
└── cust_exceptions.py

Among them, by_decorator and by_hook_func are the main project modules prepared for the two implementation methods respectively, and cust_exceptions.py is a custom exception.

Define exceptions

Define two custom exceptions in cust_exceptions.py.

class CustomException_1(Exception):
    pass

class CustomException_2(Exception):
    pass

Implemented based on hook function

Define exception hook functions and implement agents

Define the hook function in by_hook_func\__init__.py.

import sys
import traceback

from cust_exceptions import CustomException_1, CustomException_2


def handle_global_exceptions(exc_type, exc_value, exc_traceback):
    # Customize global exception handling function
    #Here you can execute your exception handling logic, such as logging, sending alerts, etc.
    if issubclass(exc_type, CustomException_1):
        msg = exc_value.args[0]
        print(f"Successfully caught exception CustomException_1 Message:{<!-- -->msg}")
    elif issubclass(exc_type, CustomException_2):
        msg = exc_value.args[0]
        print(f"Successfully caught exception CustomException_2 Message:{<!-- -->msg}")
    else:
       #For other exceptions, directly output the exception information
 traceback.print_exception(exc_type, exc_value, exc_traceback)

#Agent to the original exception hook function of the system
sys.excepthook = handle_global_exceptions
Test

Call the class in by_hook_func\run.py.

from by_hook_func.class_1 import Class_1
from by_hook_func.sub_module.class_2 import Class_2

class_1 = Class_1()
class_2 = Class_2()

The execution results are as follows:

You can see that the exception output related information is correctly captured, and at the same time actively interrupts the program.

Implemented based on decorator

Define global exception catching decorator

Define the decorator in by_decorator\__init__.py.

from functools import wraps

from cust_exceptions import CustomException_2, CustomException_1


def catch_exceptions(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        try:
            result = func(*args, **kwargs)
            return result
        except CustomException_1 as e:
            # Processing logic for exception CustomException_1
            print("Successfully caught exception CustomException_1")
        except CustomException_2 as e:
            # Processing logic for exception CustomException_1
            print("Exception CustomException_2 successfully caught")

    return wrapper
Apply Decorator

Raise custom exceptions in by_decorator\class_1.py and by_decorator\sub_module\class_2.py to test whether they can be caught.

1. Decorate the class that triggers the exception in by_decorator\class_1.py:

from by_decorator import catch_exceptions
from cust_exceptions import CustomException_1

@catch_exceptions
class Class_1():
    def __init__(self):
        raise CustomException_1(f"I threw an exception CustomException_1" in {<!-- -->__file__})

2. Decorate the class that triggers the exception in by_decorator\sub_module\class_2.py:

from by_decorator import catch_exceptions
from cust_exceptions import CustomException_1

@catch_exceptions
class Class_2():
    def __init__(self):
        raise CustomException_1(f"I threw an exception CustomException_1" in {<!-- -->__file__})
Test

Call the class in by_decorator\run.py.

from by_decorator.class_1 import Class_1
from by_decorator.sub_module.class_2 import Class_2

class_1 = Class_1()
class_2 = Class_2()

The execution results are as follows:

You can see that the exception output related information is correctly captured, and at the same time the program is not actively interrupted.

Analysis

Combining the test results of the two implementation methods, the main difference is that after an exception is triggered, the exception caught based on the hook function will interrupt the current process after being handled, but the exception caught based on the decorator will be handled. The current process will not be interrupted.

Exception catching based on hook functions directly intercepts the lowest-level system-level exception handling. In other words, we have replaced the system’s exception handling mechanism. Therefore, after processing, it is directly judged as causing a system-level exception, that is, an error is reported, which will naturally interrupt the current process.

The essence of exception catching based on decorators is the processing idea of try except. It’s just that we have gathered these exception handling methods together in the form of decorators to avoid writing try except everywhere. In the file by_decorator\run.py, the way we call the function and handle the exception is essentially as follows:

from by_decorator.class_1 import Class_1
from by_decorator.sub_module.class_2 import Class_2
from cust_exceptions import CustomException_1

#Original call and processing method
# class_1 = Class_1()
# class_2 = Class_2()


try:
    class_1 = Class_1()
except CustomException_1 as e:
    msg = e.args[0]
    print(f"Successfully caught exception CustomException_1 Message:{<!-- -->msg}")
class_2 = Class_2()

Application scenarios

After the above analysis, we can get that the global exception capture based on the decorator is essentially to capture those business exceptions that cannot be adjusted in our scenario but have debugging value. Global exception capture based on hook functions can capture those technical exceptions that we have not predicted at all.

Several nouns are mentioned here, and the relationship between these nouns is shown in the figure below:

Term explanation

The scene can be adjusted/the scene cannot be adjusted

Scenario adjustment: refers to whether the program running site can return to the site to continue executing the code after an exception is triggered and handled.

Attention! The scene here does not refer to the process of the current program.

Code examples are given here:

for i in range(10):
    try:
        if i==5:
            raise Exception("Value of i is 5")
        print(i)
    except Exception as e:
        print(e.args[0])
        continue

In this code, an exception is triggered when i==5 and is successfully caught and handled. You can notice that continue is executed after the exception is caught. That is, skip this cycle and successfully enter the next cycle. This process is the scene adjustment after exception handling. The execution result of this code is as follows:

Business abnormality/technical abnormality

Business exceptions: refer to exceptions related to the business logic of the application.

Technical Exceptions: Refers to exceptions related to technical aspects of the application.

Business exceptions refer to problems that are usually not caused by programming errors or technical problems, but by user input, business rules or external factors. Technical anomalies are usually caused by programming errors, insufficient resources, network problems, hardware problems or other technical factors.

Code examples are given here:

class RoomFullException(Exception):
    # Room type is full exception
    pass


def book_room(curr_i):
    if curr_i % 2 == 0:
        raise RoomFullException(curr_i)
    else:
        return f"Successfully booked room {<!-- -->curr_id}"


if __name__ == '__main__':
    # ID of the room type to be booked
    room_type_ids = [1001, 1002, '1003', 1004]
    for curr_id in room_type_ids:
        try:
            # Book a room based on the current ID
            print(book_room(curr_id))
        except RoomFullException:
            print(f"Reservation failed, room type {<!-- -->curr_id} is full")

This is a demo for booking a room type based on the target room type ID.

It can be seen that we can only successfully book room types whose room type ID is an even number. And this rule can be given by the hotel management. For the program, once the reservation fails, a RoomFullException exception will be thrown. This exception is a business exception.

It can also be noticed that in the list room_type_ids, there is an element '1003', which is a string. When the ID is called by the function book_room( ) is bound to trigger a TypeError exception, because in python, strings cannot perform remainder operations. Anomalies like this are called technical anomalies.

Here are the results of this program:

Applicable scenarios

After the previous analysis, we can use these two global exception handling methods together to achieve multi-level interception effects.

  • First-level interception: Use try except to capture. Try first to catch scene-adjustable exceptions and handle them normally.
  • Secondary interception: Use the global exception catching mechanism based on decorators to catch exceptions that cannot be adjusted in the scene and perform remedial error handling.
  • Third-level interception: Use the global exception catching mechanism based on hook functions to catch serious errors, and perform certain operations before throwing serious errors, such as encapsulating errors, etc.