Spinlock mechanism in Linux – API usage

Operations implemented by spinlock in different scenarios

• 1: spin_lock function call
• 2: Adjust and adapt based on spin_lock according to relevant scene requirements
• • Turn off scheduled spin lock
  • • Implementation of spinlock in SMP multi-core system
    • Spinlock implementation under UP single-core system
  • Turn off hard interrupt spin lock
  • Turn off the spin lock of soft interrupt
  • Selection of the above spinlock locking APIs in different scenarios

One: spin_lock function call

The implementation of spinlock locking is based on the arch_spin_lock() function, but kernel programming usually uses spin_lock(), and there are several layers of calling relationships between them. Let’s look at the outermost layer first (the code is located in /include/linux/spinlock.h):

static __always_inline void spin_lock(spinlock_t *lock)
{<!-- -->
    raw_spin_lock( & amp;lock->rlock);
}

#define raw_spin_lock(lock) _raw_spin_lock(lock)

Next, the implementation of _raw_spin_lock() will branch.

Two: Adjust adaptation based on spin_lock according to relevant scene requirements

Close the scheduled spin lock

Implementation of spinlock in SMP multi-core system

For SMP, the implementation of _raw_spin_lock() is as follows (defined in /include/linux/spinlock_api_smp.h):

#ifdef CONFIG_INLINE_SPIN_LOCK
#define _raw_spin_lock(lock) __raw_spin_lock(lock)
#endif

static inline void __raw_spin_lock(raw_spinlock_t *lock)
{<!-- -->
    preempt_disable();
    ...
    LOCK_CONTENDED(lock, do_raw_spin_trylock, do_raw_spin_lock);
}

Using inline functions can reduce the cost of function calls and improve execution speed, but it is not conducive to tracking and debugging, so the kernel provides the configuration option “CONFIG_INLINE_SPIN_LOCK” for users to choose.

The closer you go to the inner layer, the more underscores “_” are in front of the function name. It can be seen thatin the innermost __raw_spin_lock(), preempt_disable() is called to turn off scheduling. That is to say, after the code running on a CPU uses spin_lock() to try to lock, thread scheduling and preemption based on the CPU are disabled. This also reflects the semantics of spinlock as a “busy loop” form of lock. .

At the do_raw_spin_lock() step, we enter arch_spin_lock() related to the architecture.

static inline void **do_raw_spin_lock**(raw_spinlock_t *lock)
{<!-- -->
    __acquire(lock);
    **arch_spin_lock( & amp;lock->raw_lock);**
    ...
}

Implementation of spinlock in UP single-core system

Let’s take a look at the implementation of _raw_spin_lock() for the UP system (the code is located in /include/linux/spinlock_api_up.h):

#define _raw_spin_lock(lock) __LOCK(lock)

#define __LOCK(lock) \
  do {<!-- --> preempt_disable(); ___LOCK(lock); } while (0)

#define ___LOCK(lock) \
  do {<!-- --> __acquire(lock); (void)(lock); } while (0)

In the UP environment, it is no longer necessary to prevent multiple CPUs from accessing shared variables simultaneously, so the function of spin_lock() is only to turn off scheduling, which is equivalent to (or degenerates into) preempt_disable().

The reason why the UP system also supports the use of spinlock-related functions is because the same set of code can support both UP and SMP applications. You only need to choose whether to use “CONFIG_SMP” in the configuration.

Close the spin lock of hard interrupt

spin_lock() can prevent thread scheduling, but it cannot prevent the arrival of hardware interrupts and the subsequent execution of the interrupt handling function (hardirq). What impact will this have?

Just imagine, supposethread T on a CPU holds a spinlock. After an interrupt occurs, the CPU switches to executing the corresponding hardirq. If the hardirq also tries to hold this spinlock, it will not be successful and the hardirq will not be able to exit. Before hardirq actively exits, thread T cannot continue to execute to release the spinlock, which will eventually cause the code on the CPU to be unable to continue running forward, forming a deadlock (dead lock).

In order to prevent this from happening, we need to use thespin_lock_irq() function, a combination of spin_lock() and local_irq_disable(), which can turn off interrupts while spinlock is locked.
Because interrupt closing operations can be nested, more often we use local_irq_save() to record the status of interrupt closing. Correspondingly, a more commonly used function is spin_lock_irqsave()< /strong>.

static inline unsigned long __raw_spin_lock_irq_save(raw_spinlock_t *lock)
{<!-- -->
    unsigned long flags;
\t
    local_irq_save(flags);
    __raw_spin_lock(lock);
    return flags;
}

If hardirq does not share variables with threads, can spin_lock() be used directly? No, because the corresponding softirq function may be executed before switching back to the interrupted thread. If the softirq may access variables shared with the thread, the thread should use spin_lock_bh(), a two-in-one function of spin_lock() plus local_bh_disable(), otherwise it may also cause dead lock.

“bh” represents bottom half, and bottom half in Linux includes softirq, tasklet and workqueue. Since workqueue runs in the process context, “bh” here only targets softirq and tasklet.

Selection of the above spinlock locking APIs in different scenarios

If interrupts are turned off, hardirq will not be executed, and the corresponding softirq will not be executed. It can be seen that using spin_lock_irqsave() is undoubtedly the safest, but it is also the most expensive.

From the perspective of program performance, in the process context, for variables that will not be shared with hardirq/softirq, you should try to use the more lightweight spin_lock(). If it will only be shared with softirq but not hardirq, spin_lock_bh() should be used.

For the hardirq context, because Linux does not support hardirq nesting (refer to the discussion in the comment area of this article), during hardirq execution, the CPU’s response to interrupts is turned off by default, so spin_lock() can be used directly.

As for the softirq context, because it may be interrupted by hardirq, for variables that will be shared with hardirq, spin_lock_irqsave() needs to be used.

In short, before using a lock, you must know who your opponents are who may compete with you for this lock.

At this point, we can answer the question left above, that is, why a CPU attempts to obtain (or compete for) at most one spinlock in a context. The thread uses spin_lock() to try to acquire spinlock A. At this time, an interrupt occurs. If hardirq acquires spinlock B, then the CPU is trying to acquire 2 spinlocks at the same time.

If hardirq does not try to acquire spinlock, and enters softirq after execution, softirq tries to acquire spinlock B, and then is interrupted by another interrupt. The new hardirq tries to acquire spinlock C during execution, then the CPU is trying to acquire spinlock C at the same time. 3 spinlocks.

If nmi is added, and so on, a CPU will try to acquire up to 4 spinlocks at the same time.

spin_lock_irqsave()/spin_lock_bh() can prevent deadlocks caused by hardirq/softirq and thread shared variables,but this is only one situation in which deadlocks may occur, and it can also be said to be avoided by simply choosing the appropriate API. Deadlock.
This article is reproduced from: https://zhuanlan.zhihu.com/p/90634198