Comprehensive analysis of multithreading in linux system
Directory
?edit
Introduction introduction:
1. Thread basic introduction:
2. Concept:
3. Thread definition
4. Thread creation
5. Thread exit
6. Thread synchronization
7. Thread attributes
Introduction to Introduction:
In the traditional Unix model, when a process needs something to be performed by another entity, the process forks (forks) a child process and lets the child process do the processing. Most network server programs under Unix are written in such a way that the parent process accepts connections, spawns child processes, and the child process handles interactions with clients.
While this model has worked well for many years, there are a few issues with forking:
forks are expensive. The memory image is copied from the parent process to the child process, all descriptors are copied in the child process, and so on. At present, some Unix implementations use a technology called copy-on-write, which can avoid copying the data space of the parent process to the child process. Despite this optimization technique, forking is still expensive.
After the fork child process, you need to use inter-process communication (IPC) to pass information between the parent and child processes. The information before fork is easy to transfer, because the child process has a copy of the parent process data space and all descriptors from the beginning. But returning information from the child process to the parent process requires more work.
Threads help with both of these problems. Threads are sometimes called lightweight processes because threads are “lighter” than processes, and generally speaking, creating a thread is 10 to 100 times faster than creating a process.
All threads in a process share the same global memory, which makes it easy for threads to share information, but this simplicity also creates synchronization problems.
All threads in a process share not only global variables, but also: process instructions, most data, open files (such as descriptors), signal handlers and signal dispositions, current working directory, user ID, and group ID. But each thread has its own thread ID, register set (including program counter and stack pointer), stack (used to store local variables and return addresses), error, signal mask, and priority.
Thread programming in Linux conforms to the Posix.1 standard and is called Pthreads. All pthread functions start with pthread_. Before calling them, include the pthread.h header file, a function library libpthread implementation.
1. Thread basic introduction:
data structure:
pthread_t: thread ID
pthread_attr_t: attribute of the thread
Operation function:
pthread_create(): Create a thread
pthread_exit(): terminate the current thread
pthread_cancel(): Interrupt the operation of another thread
pthread_join(): Block the current thread until another thread finishes running
pthread_attr_init(): Initialize the attributes of the thread
pthread_attr_setdetachstate(): Set the attribute of detachment state (determine whether this thread can be combined when it is terminated)
pthread_attr_getdetachstate(): Get the attributes of the detached state
pthread_attr_destroy(): delete thread attributes
pthread_kill(): Send a signal to the thread
Synchronous function:
for mutex and condition variables
pthread_mutex_init() initializes the mutex
pthread_mutex_destroy() deletes the mutex
pthread_mutex_lock(): Occupy a mutex (blocking operation)
pthread_mutex_trylock(): Attempts to occupy a mutex (non-blocking operation). That is, when the mutex is free, it will hold the lock; otherwise, return immediately.
pthread_mutex_unlock(): Release the mutex
pthread_cond_init(): Initialize condition variables
pthread_cond_destroy(): Destroy condition variables
pthread_cond_signal(): Wake up the first thread that calls pthread_cond_wait() and goes to sleep
pthread_cond_wait(): Wait for the special condition of the condition variable to occur
Thread-local storage (or thread-specific data in Pthreads terminology):
pthread_key_create(): assigns a key used to identify thread-specific data in a process
pthread_setspecific(): Set thread-specific bindings for specified thread-specific data keys
pthread_getspecific(): Gets the calling thread's key binding and stores that binding in the location pointed to by value
pthread_key_delete(): Destroys an existing thread-specific data key
pthread_attr_getschedparam(); get thread priority
pthread_attr_setschedparam(); set thread priority
2. Concept:
Components of a thread:
Thread ID Thread ID
Stack stack
Policy priority
Signal mask signal code
Errno error code
Thread-Specific Data special data
3. Thread definition
1) pthread_t pthread_ID, used to identify a thread, can not simply be regarded as an integer, it may be a structure, related to the implementation
2) The pthread_equal function is used to compare whether two pthread_t are equal
#include <pthread.h>
int pthread_equal(pthread_t tid1, pthread_t tid2)
3) The pthread_self function is used to obtain the thread id of this thread
pthread_t *restrict tidp: Returns the Thread ID of the last created Thread
const pthread_attr_t *restrict attr: Specify the Attributes of the thread, which will be discussed later, and now you can use NULL
void *(*start_rtn)(void *): specifies the thread function pointer, the function returns a void , and the parameter is also void
void *restrict arg: parameters passed to the thread function
Returns an error value.
Each thread in a process is identified by a thread ID (thread ID), whose data type is pthread_t (usually unsigned int). If the new thread is successfully created, its ID will be returned through the tid pointer.
Each thread has many attributes: priority, starting stack size, whether it should be a daemon thread, etc. When creating a thread, we can specify these attributes by initializing a pthread_attr_t variable to override the default value. We usually use the default value, in which case we declare the attr parameter as a null pointer.
Finally, when a thread is created, we specify a function that it will execute. A thread starts by calling this function, and terminates either explicitly (by calling pthread_exit) or implicitly (by having the function return). The address of the function is specified by the func parameter, and the calling parameter of this function is a pointer arg, if we need multiple calling parameters, we must pack them into a structure, and then pass its address as the only parameter to the start function.
In the declaration of func and arg, the func function takes a generic pointer (void *) parameter and returns a generic pointer (void *), which allows us to pass a pointer (pointing to anything we want) to the thread , a pointer (again to whatever we want) is returned by the thread. If the call is successful, it returns 0, and when an error occurs, it returns a positive Exxx value.
2) The pthread function does not set errno when an error occurs, but directly returns an error value
3) Under the Linux system, in the old kernel, since Thread is also regarded as a special Process that can share address space and resources, different Threads created in the same Process have different Process IDs (call getpid get). In the new 2.6 kernel, Linux adopts the NPTL (Native POSIX Thread Library) threading model (refer to http://en.wikipedia.org/wiki/Native_POSIX_Thread_Library and http://www-128.ibm.com/ developerworks/linux/library/l-threading.html?ca=dgr-lnxw07LinuxThreadsAndNPTL), under this threading model, different threads calling getpid in the same process return the same PID.
4) No assumptions can be made about the running order of the new thread created and the current creator thread
5. Thread exit
exit, _Exit, _exit are used to terminate the current process, not the thread
There are three ways to abort a thread:
a. return in the thread function
b. Canceled by another thread in the same process
c. The thread calls the pthread_exit function
Usage of pthread_exit and pthread_join functions:
a. Thread A calls pthread_join(B, &rval_ptr), is blocked, and enters the Detached state (if it has entered the Detached state, the pthread_join function returns EINVAL). If you are not interested in the end code of B, rval_ptr can pass NULL.
b. Thread B calls pthread_exit(rval_ptr), exits thread B, and the end code is rval_ptr. Note that the life cycle of the memory pointed to by rval_ptr should not point to the data in B’s Stack.
c. Thread A resumes running, the pthread_join function call ends, and the end code of thread B is saved to the rval_ptr parameter. If thread B is Canceled, then the value of rval_ptr is PTHREAD_CANCELLED.
The prototypes of the two functions are as follows:
#include <pthread.h>
void pthread_exit(void*rval_ptr);
int pthread_join(pthread_t thread, void**rval_ptr);
This function waits for a thread to terminate. Comparing threads to processes, pthread_creat is similar to fork, and pthread_join is similar to waitpid. We have to wait for the tid of the thread. Unfortunately, we have no way to wait for any thread to end. If the status pointer is non-null, the thread’s return value (a pointer to an object) will be stored at the location pointed to by status.
A Thread can request another Thread to be Canceled by calling the pthread_cancel function:
This function makes the specified thread appear to have called pthread_exit(PTHREAD_CANCELLED). However, the specified thread can choose to ignore or perform its own processing, which will be discussed later. In addition, this function does not cause a Block, but just sends a Cancel request.
A thread can arrange certain functions to be called automatically when it exits, similar to the atexit() function. The following function needs to be called:
These two functions maintain a Stack of function pointers, which can push/pop function pointers and function parameter values. The order of execution is from the top of the stack to the bottom of the stack, which is the opposite of the order of push.
The thread cleanup handlers specified by pthread_cleanup_push will be called in the following cases:
a. Call pthread_exit
b. Corresponding cancel request
c. Call pthread_cleanup_pop() with a non-zero argument. (If pthread_cleanup_pop() is called with 0, the handler will not be called
There is a rather weird requirement that since these two functions may be implemented by macros, the calls of these two functions must be in the same Scope and paired, because there may be one in the implementation of pthread_cleanup_push {, while pthread_cleanup_pop may have a }.
Therefore, in general, these two functions are used to handle unexpected situations, for example:
void*thread_func(void*arg)
{
pthread_cleanup_push(cleanup, "handler")
// do something
Pthread_cleanup_pop(0);
return((void*)0);
}
The correlation between process function and thread function:
Process Primitive
Thread Primitive
Description
fork
pthread_create
create New control flow
exit
pthread_exit
Exit existing control flow
waitpid
pthread_join
wait for control flow and get end code
atexit
pthread_cleanup_push
Register the function that is called when the control flow exits
getpid
pthread_self
Get the control flow id
abort
pthread_cancel
Request abnormal exit
By default, the end state of a thread A is saved until pthread_join is called for the thread, that is to say, even if thread A has ended, as long as no thread B calls pthread_join(A), the exit state of A is always saved . When the thread is in the Detached state, when the thread exits, its resources can be recycled immediately, and this exit state is also lost. In this state, the pthread_join function cannot be called for this thread. We can make the specified thread enter the Detach state by calling the pthread_detach function:
#include <pthread.h>
int pthread_detach(pthread_t tid);
By modifying the attr parameter of the pthread_create function, we can specify that a thread enters the Detached state immediately after creation
6. Thread synchronization
Mutex: Mutex
Each off-the-shelf writes data sequentially to the same file, and the final result is unimaginable. So use a mutex to ensure that only one thread is executing a piece of code for a period of time.
a. for mutually exclusive access
b. Type: pthread_mutex_t, must be initialized to PTHREAD_MUTEX_INITIALIZER
(for a statically allocated mutex, equivalent to pthread_mutex_init(…, NULL)) or call pthread_mutex_init. Mutex should also be destroyed with pthread_mutex_destroy. The prototypes of these two functions are as follows:
#include <pthread.h>
int pthread_mutex_init(
pthread_mutex_t* restrict mutex,
constpthread_mutexattr_t*restrict attr)
int pthread_mutex_destroy(pthread_mutex_t*mutex);
c. pthread_mutex_lock is used to Lock Mutex, if the Mutex has been Locked, the function call will Block until the Mutex is Unlocked, then the function will Lock the Mutex and return. pthread_mutex_trylock is similar, except that when the Mutex is locked, it will not block, but return an error value EBUSY.
pthread_mutex_unlock is to unlock a mutex. The prototypes of these three functions are as follows:
#include <pthread.h>
int pthread_mutex_lock(pthread_mutex_t*mutex);
int pthread_mutex_trylock(pthread_mutex_t*mutex);
int pthread_mutex_unlock(pthread_mutex_t*mutex);
d. give an example
void reader_function (void);
void writer_function (void);
char buffer;
int buffer_has_item=0;
pthread_mutex_t mutex;
struct timespec delay;
void main (void)
{
pthread_t reader;
/* Define the delay time */
delay.tv_sec=2;
delay.tv_nec =0;
/* Initialize a mutex object with default properties */
pthread_mutex_init ( & mutex, NULL);
pthread_create( &reader, pthread_attr_default,(void*) &reader_function), NULL);
writer_function();
}
void writer_function (void){
while (1) {
/* Lock the mutex */
pthread_mutex_lock ( & mutex);
if(buffer_has_item==0){
buffer=make_new_item();
buffer_has_item=1;
}
/* Open the mutex */
pthread_mutex_unlock( & mutex);
pthread_delay_np( & amp; delay);
}
}
void reader_function(void){
while (1) {
pthread_mutex_lock( & mutex);
if(buffer_has_item==1){
consume_item(buffer);
buffer_has_item=0;
}
pthread_mutex_unlock( & mutex);
pthread_delay_np( & amp; delay);
}
}
It should be noted that deadlocks are likely to occur in the process of using mutexes: two threads try to occupy two resources at the same time, and lock the corresponding mutexes in different orders, for example, both threads need to lock the mutexes. Exclusive lock 1 and mutex 2, thread a first locks mutex 1, thread b first locks mutex 2, and then a deadlock occurs.
At this point we can use the function pthread_mutex_trylock, which is a non-blocking version of the function pthread_mutex_lock. When it finds that the deadlock is inevitable, it will return the corresponding information, and the programmer can deal with the deadlock accordingly. In addition, different mutex types deal with deadlocks differently, but the most important thing is that programmers should pay attention to this in programming
Read-write lock: Reader-Writer Locks
a. Multiple threads can obtain a read lock (Reader-Writer lock in read mode) at the same time, but only one thread can obtain a write lock (Reader-writer lock in write mode)
b. Read-write locks have three states
i. One or more threads acquire a read lock, and other threads cannot acquire a write lock
ii. One thread acquires a write lock, and other threads cannot acquire a read lock
iii. No thread has acquired this read-write lock
c. The type is pthread_rwlock_t
d. Create and close methods as follows:
#include <pthread.h>
int pthread_rwlock_init(
pthread_rwlock_t * restrict rwlock,
constpthread_rwlockattr_t*restrict attr)
int pthread_rwlock_destroy(pthread_rwlock_t*rwlock);
e. The method of obtaining a read-write lock is as follows:
#include <pthread.h>
int pthread_rwlock_rdlock(pthread_rwlock_t*rwlock);
int pthread_rwlock_wrlock(pthread_rwlock_t*rwlock);
int pthread_rwlock_unlock(pthread_rwlock_t*rwlock);
int pthread_rwlock_tryrdlock(pthread_rwlock_t*rwlock);
int pthread_rwlock_trywrlock(pthread_rwlock_t*rwlock);
pthread_rwlock_rdlock: Get a read lock
pthread_rwlock_wrlock: Get a write lock
pthread_rwlock_unlock: Release the lock, whether it is a read lock or a write lock, this function is called
Note that the specific implementation may limit the number of threads that can acquire read locks at the same time, so you need to check the error value when calling pthread_rwlock_rdlock, and the other two pthread_rwlock_wrlock and pthread_rwlock_unlock generally do not need to be checked, if our code is written correctly.
Conditional Variable: condition variable
An obvious disadvantage of a mutex is that it has only two states: locked and unlocked. Condition variables make up for the lack of mutexes by allowing threads to block and wait for another thread to send a signal, and they are often used together with mutexes. When used, a condition variable is used to block a thread. When the condition is not met, the thread often unlocks the corresponding mutex and waits for the condition to change. Once some other thread changes the condition variable, it will notify the corresponding condition variable to wake up one or more threads that are blocked by the condition variable. These threads will relock the mutex and retest that the condition is met. Generally speaking, condition variables are used for synchronization between threads.
a. Conditions must be protected by Mutex
b. The type is: pthread_cond_t, which must be initialized to PTHREAD_COND_INITIALIZER (condition for static allocation, equivalent to pthread_cond_init(…, NULL)) or call pthread_cond_init
#include <pthread.h>
int pthread_cond_init(
pthread_cond_t * restrict cond,
constpthread_condxattr_t*restrict attr)
int pthread_cond_destroy(pthread_cond_t*cond);
c. The pthread_cond_wait function is used to wait for a condition to occur (=true). pthread_cond_timedwait is similar, but returns an error value ETIMEDOUT when waiting for a timeout. The timeout time is specified with the timespec structure. In addition, both functions need to pass in a Mutex for protection conditions
When the count value is 0, the decrement function is blocked at pthread_cond_wait, and the mutex count_lock is opened. At this time, when the function increment_count is called, the pthread_cond_signal() function changes the condition variable and tells decrement_count() to stop blocking.
Note that the time of timespec is absolute time rather than relative time, so you need to call the gettimeofday function to obtain the current time, then convert it into the timespec structure, and add the offset.
e. There are two functions for notifying the thread that the condition is met (=true):
#include <pthread.h>
int pthread_cond_signal(pthread_cond_t*cond);
int pthread_cond_broadcast(pthread_cond_t*cond);
The difference between the two is that the former will wake up a single thread, while the latter will wake up multiple threads.
7. Thread attributes
1. Thread attribute setting
We use the pthread_create function to create a thread. In this thread, we use the default parameters, that is, the second parameter of the function is set to NULL. Indeed, for most programs, it is enough to use the default properties, but it is still necessary for us to understand the relevant properties of threads.
The attribute structure is pthread_attr_t, which is also defined in the header file pthread.h. The attribute value cannot be set directly, and related functions must be used to operate. The initialization function is pthread_attr_init, and this function must be called before the pthread_create function. The attribute object mainly includes whether to bind, whether to separate,
Stack address, stack size, priority. The default properties are unbound, non-detached, default stack, and the same level of priority as the parent process.
2. Binding
Regarding thread binding, another concept is involved: light process (LWP: Light Weight Process). A light process can be understood as a kernel thread, which is located between the user layer and the system layer. The system allocates thread resources and controls threads through light processes. A light process can control one or more threads.
By default, how many light processes are started and which light processes control which threads are controlled by the system. This situation is called unbound. In the binding state, as the name suggests, a certain thread is fixedly “bound” to a light process.
The bound thread has a high response speed, because the scheduling of the CPU time slice is oriented to the light process, and the bound thread can guarantee that it always has a light process available when needed. By setting the priority and scheduling level of the bound light process, the bound thread can meet the requirements such as real-time response.
The function to set the thread binding state is pthread_attr_setscope, which has two parameters, the first is a pointer to the attribute structure, and the second is the binding type, which has two values: PTHREAD_SCOPE_SYSTEM (bound) and PTHREAD_SCOPE_PROCESS ( unbound). The following code creates a bound thread.
#include <pthread.h>
pthread_attr_t attr;
pthread_t tid;
/*Initialize attribute values, all set to default values*/
pthread_attr_init( &attr);
pthread_attr_setscope( &attr, PTHREAD_SCOPE_SYSTEM);
pthread_create( &tid, &attr,(void*) my_function, NULL);
3. Thread separation state
The detached state of a thread determines how a thread terminates itself. When a non-detached thread terminates, its thread ID and exit status will remain until another thread calls pthread_join. A detached thread will release all resources when it terminates, and we cannot wait for it to terminate.
The function to set the thread detach state is pthread_attr_setdetachstate(pthread_attr_t *attr, int detachstate)
The second parameter can be selected as PTHREAD_CREATE_DETACHED (detached thread) or PTHREAD _CREATE_JOINABLE (non-detached thread).
One thing to note here is that if you set a thread as a separate thread, and this thread runs very fast, it is likely to terminate before the pthread_create function returns. After it terminates, it may hand over the thread number and system resources to other Thread usage, such that the thread calling pthread_create gets the wrong thread number.
To avoid this situation, certain synchronization measures can be taken. One of the simplest methods is to call the pthread_cond_timewait function in the created thread, let the thread wait for a while, and leave enough time for the function pthread_create to return. Setting a waiting time is a common method in multi-threaded programming.
4. Priority
It is stored in the structure sched_param. Use the function pthread_attr_getschedparam and the function pthread_attr_setschedparam to store. Generally speaking, we always take the priority first, and then store it back after modifying the obtained value. Below is a simple example.
The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge. CS introductory skill tree Introduction to LinuxFirst acquaintance with Linux28426 People are learning systematically
