Why
The development and adoption of the concept of Threaded IRQs in the Linux kernel can be traced back to 2008. The main motivation behind interrupt threading is to improve real-time performance and responsiveness, especially for real-time and embedded systems.
Before interrupt threading, interrupt handlers (Interrupt Service Routines, ISRs) run directly in the interrupt context, which causes some problems:
The interrupt context cannot be preempted, so the interrupt handler needs to execute and return as soon as possible. Longer interrupt processing may cause the system’s real-time performance degradation and response delay.
While an interrupt handler is running, the corresponding interrupt is disabled. This may result in missed interrupts or delayed interrupt processing.
Kernel APIs that cause scheduling, such as mutexes or sleeping wait queues, cannot be used in interrupt handlers.
Interrupt threading solves these problems by splitting the interrupt handler into two parts:
A fast hardware interrupt handler (Fast hardware ISR) is responsible for handling hardware interrupts, disabling interrupt sources, and passing the remaining work to thread interrupt handlers.
A threaded IRQ handler, which runs in an independent thread context, can be preempted, and use kernel APIs such as mutexes.
Interrupt threading provides the following advantages:
Improves real-time performance and responsiveness because threaded interrupt handlers can be preempted, allowing other high-priority tasks to preempt execution if necessary.
Allows the use of kernel APIs, such as mutexes, in interrupt handlers, simplifying driver design and synchronization.
How
Core function irq_thread
irq_thread is the core function of the threaded interrupt handler, responsible for executing threaded interrupt processing. The following is the implementation of the irq_thread function. I have added detailed comments to the code to help understand its working principle.
static int irq_thread(void *data)
{<!-- -->
/* The 'data' parameter contains a pointer to the interrupt descriptor, which includes interrupt handler information */
struct irq_desc *desc = data;
struct irqaction *action = desc->action;
/* Get pointer to threaded interrupt handler */
irqreturn_t (*handler)(int, void *) = action->handler;
irqreturn_t ret;
/* loop infinitely until thread stop signal is received */
while (!kthread_should_stop()) {<!-- -->
/* Wait for the hardware interrupt handler to wake up this thread
* When the hard interrupt handler returns IRQ_WAKE_THREAD, the 'desc->irqs_queued' counter is incremented.
* This thread will wait for the counter to become non-zero. */
wait_event_interruptible(desc->wait_for_irq, desc->irqs_queued);
/* Call the threaded interrupt handler to handle the interrupt */
ret = handler(action->irq, action->dev_id);
/* After interrupt handling is complete, check that the handler handled the interrupt correctly */
if (ret == IRQ_HANDLED) {<!-- -->
/* If the threaded interrupt handler returns IRQ_HANDLED, the interrupt was handled correctly.
* At this point, we decrement the 'desc->irqs_queued' counter. */
atomic_dec( & desc->irqs_queued);
}
}
/* Thread exits normally, returns 0 */
return 0;
}
The irq_thread function first obtains the interrupt descriptor (irq_desc) and interrupt handler (handler) information from the incoming data parameter. Then, it enters an infinite loop until it receives a thread stop signal.
Inside the loop, the wait_event_interruptible function puts the thread into a wait state until the hard interrupt handler increments the desc->irqs_queued counter. This usually happens when the hard interrupt handler finishes processing and returns IRQ_WAKE_THREAD. Once the thread is woken up, it calls the threaded interrupt handler to handle the interrupt.
If the threaded interrupt handler returns IRQ_HANDLED, indicating that the interrupt has been handled correctly, the irq_thread function will decrement the desc->irqs_queued counter. This way, as long as there are outstanding interrupts, the thread will continue to handle them.
When the thread receives a stop signal (for example, when the device driver is unloaded), the kthread_should_stop function returns true, making the irq_thread function jump out of the loop and return 0, indicating that the thread has exited normally.
Where does the
irq_thread core function come from? Who create it
The irq_thread function is called by the kernel thread scheduler. When a device driver requests a threaded interrupt through the request_threaded_irq function, the kernel will create a new kernel thread to handle the threaded interrupt. This kernel thread will run the irq_thread function.
In the request_threaded_irq function, a kthread (kernel thread) will be created to execute the irq_thread function. The following is a partial implementation of the request_threaded_irq function:
struct irqaction *new_action;
struct task_struct *t;
/* ... */
/* Create a new kernel thread to execute the irq_thread function. 'irq' is the interrupt number and 'desc' is the corresponding interrupt descriptor. */
t = kthread_create(irq_thread, desc, "irq/%d-%s", irq, new_action->name);
if (IS_ERR(t)) {<!-- -->
/* error handling */
}
/* Set the priority of the kernel thread */
sched_setscheduler_nocheck(t, SCHED_FIFO, param);
/* Wake up the newly created kernel thread */
wake_up_process(t);
The request_threaded_irq function first creates a new kernel thread through the kthread_create function, which will run the irq_thread function. The first parameter of kthread_create is the function to be run, that is, irq_thread, and the second parameter is the parameter passed to irq_thread, that is, the interrupt description character desc.
After creating a new kernel thread, the request_threaded_irq function sets the priority of the thread (via the sched_setscheduler_nocheck function) and wakes up the thread (via the wake_up_process function) . In this way, the newly created kernel thread starts to execute the irq_thread function, and passes desc as the data parameter.
To sum up, the irq_thread function is called by the kernel thread scheduler, and the data parameter comes from the request_threaded_irq function, which contains the interrupt descriptor.
Core function kthread_should_stop
kthread_should_stop is a key function in the kernel thread, which is used to check whether the thread is notified to stop. When a kernel thread needs to be terminated, the kthread_stop function is usually called. This triggers a flag that kthread_should_stop can check to determine if the thread should exit.
Here is a simple kernel thread example to demonstrate the usage of kthread_should_stop:
#include <linux/module.h>
#include <linux/kthread.h>
#include <linux/delay.h>
static struct task_struct *example_thread;
// example kernel thread function
static int example_thread_func(void *data)
{<!-- -->
// main loop in thread
while (!kthread_should_stop()) // check if the thread should stop
{<!-- -->
printk(KERN_INFO "example_thread_func: running\\
");
ssleep(1); // sleep for 1 second
}
printk(KERN_INFO "example_thread_func: Thread is about to stop\\
");
return 0;
}
// Create a kernel thread when loading a module
static int __init example_init(void)
{<!-- -->
printk(KERN_INFO "example_init: initialize module\\
");
example_thread = kthread_run(example_thread_func, NULL, "example_thread");
if (IS_ERR(example_thread))
{<!-- -->
printk(KERN_ERR "example_init: Unable to create kernel thread\\
");
return PTR_ERR(example_thread);
}
return 0;
}
// Stop the kernel thread when unloading the module
static void __exit example_exit(void)
{<!-- -->
printk(KERN_INFO "example_exit: uninstall module\\
");
kthread_stop(example_thread); // stop kernel thread
}
module_init(example_init);
module_exit(example_exit);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Your Name");
MODULE_DESCRIPTION("Kthread example");
In this example, we create a kernel thread example_thread with the thread function example_thread_func. In the example_thread_func function, we use kthread_should_stop to check if the thread should stop. When we unload the module, the example_exit function is called, which uses kthread_stop to stop the kernel thread. This causes kthread_should_stop to return true, causing the thread function to exit the loop and end the thread.
An example, how to use
Existing problems, talks more
The scheduling and context switching of threaded interrupt handlers may incur some degree of performance overhead.
In some cases, over-reliance on threaded interrupt handlers can lead to a decrease in the overall responsiveness of the system, especially under high load or a large number of interrupt sources.
Threaded interrupt handling may not be suitable for all types of interrupts. For example, interrupts with high real-time requirements still need to use traditional hardware interrupt handlers.
In summary, the introduction of interrupt threading in the Linux kernel solves real-time performance and responsiveness issues while allowing a richer kernel API to be used in interrupt handlers. However, it may cause a certain degree of performance overhead, and may not be suitable for all types of interrupts in some scenarios.
Summary, generalization
Despite these problems, interrupt threading has become very common in many areas of real-time and embedded systems. Many drivers and systems have successfully implemented threaded interrupt handlers, improving the system’s real-time performance and responsiveness.
To take full advantage of the benefits of interrupt threading and reduce potential problems, driver developers and system designers need to consider the following:
Use threaded handlers only for interrupts that require real-time performance and responsiveness. For interrupts that do not require real-time performance, traditional hardware interrupt handlers can continue to be used.
Carefully weigh the priorities of threaded interrupt handlers to ensure that high-priority tasks can preempt execution in a timely manner.
When choosing to use threaded interrupt handlers, take care to avoid unnecessary context switching and scheduling overhead. Where appropriate, other real-time scheduling strategies such as SCHED_FIFO or SCHED_RR may be considered.
Performance analysis and tuning of interrupt threading handlers for specific systems and application scenarios to achieve optimal performance and responsiveness.
By judicious use of interrupt threading techniques, system designers and driver developers can minimize potential performance overhead and problems while improving real-time performance and responsiveness.
References
Here are some suggested reading papers on interrupt threading. Some papers may involve real-time operating systems, Linux kernel optimization and other related fields, all of which can provide you with in-depth understanding and inspiration of interrupt threading:
Mogul, Jeffrey C., and K. K. Ramakrishnan. “Eliminating receive livelock in an interrupt-driven kernel.” ACM Transactions on Computer Systems (TOCS) 15.3 (1997): 217-252.
Nieh, Jason, and Monica S. Lam. “The design, implementation and evaluation of SMART: A scheduler for multimedia applications.” ACM SIGOPS Operating Systems Review 31.5 (1997): 52-63.
Duda, K. J., and D. R. Cheriton. “Borrowed-virtual-time (BVT) scheduling: supporting latency-sensitive threads in a general-purpose scheduler.” ACM SIGOPS Operating Systems Review 33.5 (1999): 261-276.
Brandenburg, Bjorn B., and James H. Anderson. “Scheduling soft-real-time applications on multiprocessors: A decoupled approach.” Proceedings of the 2009 ACM SIGPLAN/SIGBED conference on Languages, compilers, and tools for embedded systems. 2009.
Bletsas, Konstantinos, and Bj?rn Andersson. “Preemption-light multiprocessor scheduling of sporadic tasks with high utilization bound.” Real-Time Systems Symposium, 2009, RTSS 2009. 30th IEEE. IEEE, 2009.
Calandrino, John M., and James H. Anderson. “On the design and implementation of a cache-aware soft real-time scheduler for multicore platforms.” Proceedings of the 14th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA). 2008.
Goyal, Pawan, Xingang Guo, and Harrick M. Vin. “A hierarchical CPU scheduler for multimedia operating systems.” ACM SIGOPS Operating Systems Review 30.5 (1996): 107-121.
Devi, UmaMaheswari C., and James H. Anderson. “Bounding tardiness under global EDF scheduling on a multiprocessor.” Real-Time Systems Symposium, 2005. (RTSS 2005). 26th IEEE International. IEEE, 2005.
Leontyev, Hennadiy, and James H. Anderson. “Tardiness bounds for global EDF scheduling on a multiprocessor.” Real-Time Systems Symposium, 2006. RTSS’06. 27th IEEE International. IEEE, 2006.
Marquet, Pascal, et al. “LITMUS^ RT: A status report.” Proceedings of the 9th Real-Time Linux Workshop. 2007.
Nieh, Jason, et al. “A comparison of Internet-scale workloads on Linux and Solaris.” ACM SIGMETRICS Performance Evaluation Review 29.4 (2002): 41-49.
Brandenburg, Bjorn B., et al. “A comparison of scheduling latency in Linux, PREEMPT-RT, and LITMUS^ RT.” 13th Real