Recently, I glanced at the restart script of the project and found that the operation and maintenance has been using the kill-9
What is the difference between kill -9 and kill -15?
In the past, the usual steps for us to release WEB applications were to pack the code into a war package, and then drop it on a Linux machine configured with an application container (such as Tomcat, Weblogic). At this time, we want to start/close the application in a very simple way Simple, just run the startup/shutdown scripts in it. And springboot provides another way to package the entire application together with the built-in tomcat server, which undoubtedly brings great convenience to publishing applications, but it also creates a problem: how to close the springboot application? An obvious method is to find the process id according to the application name, and kill the process id to achieve the effect of closing the application.
The above scenario description leads to my question: how to gracefully kill a springboot application process? Here we only take the most commonly used Linux operating system as an example. In Linux, the kill command is responsible for killing the process, followed by a number, which represents the signal number (Signal). Execute the kill-l command to view all the signal numbers.
xu@ntzyz-qcloud ~ % kill -l HUP INT QUIT ILL TRAP ABRT BUS FPE KILL USR1 SEGV USR2 PIPE ALRM TERM STKFLT CHLD CONT STOP TSTP TTIN TTOU URG XCPU XFSZ VTALRM PROF WINCH POLL PWR SYS
This article mainly introduces the ninth signal code KILL and the fifteenth signal number TERM.
Let’s briefly understand the difference between the two: kill-9pid can be understood as the operating system forcibly kills a process from the kernel level, while kill-15pid can be understood as sending a notification to tell the application to actively shut down. This comparison is still a bit abstract, so let’s look at the performance of the application to see what is the difference between the two commands to kill the application.
Code preparation
Since the author has a lot of contact with springboot, I will discuss a simple springboot application as an example and add the following code.
1 Add a class that implements the DisposableBean interface
@Component
public class TestDisposableBean implements DisposableBean{
@Override
public void destroy() throws Exception {
System.out.println("Test Bean has been destroyed...");
}
}
2 Add the hook when the JVM shuts down
@SpringBootApplication
@RestController
public class TestShutdownApplication implements DisposableBean {
public static void main( String [] args) {
SpringApplication.run( TestShutdownApplication.class, args);
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
@Override
public void run() {
System.out.println("Execute ShutdownHook ...");
}
}));
}
}
Test steps
-
Execute
java-jar test-shutdown-1.0.jarto run the application -
Test
kill-9pid,kill-15pid, and output log content afterctrl + c
Test results
kill-15 pid & amp; ctrl + c, the effect is the same, the output is as follows
2018-01-14 16:55:32.424 INFO 8762 --- [ Thread-3] ationConfigEmbeddedWebApplicationContext : Closing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@2cdf8d8a: startup date [Sun Jan 15: 16:5 24 UTC 2018]; root of context hierarchy 2018-01-14 16:55:32.432 INFO 8762 --- [ Thread-3] o.s.j.e.a. AnnotationMBeanExporter : Unregistering JMX-exposed beans on shutdown Execute ShutdownHook... Test bean destroyed ... java -jar test-shutdown-1.0.jar 7.46s user 0.30s system 80% cpu 9.674 total
kill-9 pid, did not output any application logs
[1] 8802 killed java-jar test-shutdown-1.0.jar java -jar test-shutdown-1.0.jar 7.74s user 0.25s system 41% cpu 19.272 total
It can be found that kill -9 pid caught the application by surprise and did not leave any chance for the application to respond. On the other hand, kill -15 pid is more elegant. First, AnnotationConfigEmbeddedWebApplicationContext (an implementation class of ApplicationContext) receives the notification, then executes the Shutdown Hook in the test code, and finally executes DisposableBean#destory( ) method. Which one is better and which one is worse, the decision is made.
Generally, we will deal with the logic of “aftermath” when the application is closed, such as
-
close socket connection
-
clean up temporary files
-
Send a message notification to the subscriber to inform you that you are offline
-
Notify the child process that it will be destroyed
-
Release of various resources
-
…
And kill -9 pid directly simulates a system downtime, the system is powered off, which is too unfriendly to the application, so don’t use the harvester to prune the flowers in the pots. Instead, use kill -15 pid instead. If it is found in an actual operation that kill -15 pid cannot close the application, you can consider using the kernel-level kill -9 pid , but please be sure to find out the reason why kill -15 pid cannot be closed afterwards.
How does springboot handle -15 TERM Signal?
As explained above, using kill -15 pid can shut down the springboot application more gracefully. We may have the following doubts: How does springboot/spring respond to this shutdown behavior? Is tomcat shut down first, followed by JVM, or the reverse order? How are they related to each other?
Try to analyze from the log, AnnotationConfigEmbeddedWebApplicationContext prints the behavior of Closing, go directly to the source code to find out, and finally find the key code in its parent class AbstractApplicationContext:
@Override
public void registerShutdownHook() {
if (this. shutdownHook == null) {
this. shutdownHook = new Thread() {
@Override
public void run() {
synchronized (startupShutdownMonitor) {
doClose();
}
}
};
Runtime.getRuntime().addShutdownHook(this.shutdownHook);
}
}
@Override
public void close() {
synchronized (this.startupShutdownMonitor) {
doClose();
if (this. shutdownHook != null) {
Runtime.getRuntime().removeShutdownHook(this.shutdownHook);
}
}
}
protected void doClose() {
if (this. active. get() & amp; & amp; this. closed. compareAndSet(false, true)) {
LiveBeansView. unregisterApplicationContext(this);
// Post the closing event in the app
publishEvent(new ContextClosedEvent(this));
// Stop all Lifecycle beans, to avoid delays during individual destruction.
if (this. lifecycleProcessor != null) {
this.lifecycleProcessor.onClose();
}
// Spring's BeanFactory may cache singleton beans
destroyBeans();
// close application context & amp;BeanFactory
closeBeanFactory();
// Execute the closing logic of the subclass
onClose();
this. active. set(false);
}
}
For the convenience of typesetting and understanding, I removed some exception handling codes in the source code and added related comments. When the container is initialized, the ApplicationContext has registered a Shutdown Hook, which calls the Close() method, so when we execute kill -15 pid, the JVM receives the shutdown command, triggers the Shutdown Hook, and then close() method to deal with some aftermath. What are the specific aftermath measures? It depends entirely on the doClose() logic of ApplicationContext, including destroying the cached singleton object mentioned in the comment, publishing the close event, closing the application context, etc. In particular, when the implementation class of ApplicationContext is When AnnotationConfigEmbeddedWebApplicationContext, it also handles the logic of closing some built-in application servers such as tomcat/jetty.
After seeing these details inside springboot, you should understand the necessity of closing the application gracefully. Both JAVA and C provide the encapsulation of Signal. We can also manually capture these Signals of the operating system. I won’t introduce too much here. Interested friends can try to capture them by themselves.
Is there any other way to close the application gracefully?
The spring-boot-starter-actuator module provides a restful interface for graceful shutdown.
Add dependencies
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency>
Add configuration
#Enable shutdown endpoints.shutdown.enabled=true # disable password authentication endpoints.shutdown.sensitive=false
In production, please note that this port needs to set permissions, such as using it with spring-security.
Execute the curl-X POST host:port/shutdown command, and you can get the following return after the shutdown is successful:
{
"message"
:
"Shutting down, bye..."
}
Although springboot provides such a method, as far as I know, no one has used this method to shut down. The kill -15 pid method achieves the same effect, and it is listed here only for the completeness of the scheme. .
How to destroy the thread pool as a member variable?
Although the JVM will help us reclaim certain resources when it is closed, if some services use a large number of asynchronous callbacks and timing tasks, improper handling may cause business problems. Among them, how to close the thread pool is a typical problem.
@Service
public class SomeService {
ExecutorService executorService = Executors. newFixedThreadPool(10);
public void concurrentExecute() {
executorService. execute(new Runnable() {
@Override
public void run() {
System.out.println("executed...");
}
});
}
}<br><br>
We need to find a way to close the thread pool when the application is closed (JVM is closed, the container stops running).
Initial solution: do nothing. Under normal circumstances, this will not be a big problem, because the JVM will be released when it is closed, but obviously it did not achieve the two words that this article has been emphasizing, yes-elegant.
The disadvantage of method 1 is that the tasks submitted in the thread pool and the unexecuted tasks in the blocking queue become extremely uncontrollable. Do you exit immediately after receiving the shutdown command? Or wait for the task execution to complete? Or wait for a certain period of time before the task is completed before closing?
Program improvements:
After seeing the disadvantages of the initial solution, I immediately thought of using the DisposableBean interface, like this:
@Service
public class SomeService implements DisposableBean{
ExecutorService executorService = Executors. newFixedThreadPool(10);
public void concurrentExecute() {
executorService. execute(new Runnable() {
@Override
public void run() {
System.out.println("executed...");
}
});
}
@Override
public void destroy() throws Exception {
executorService. shutdownNow();
//executorService. shutdown();
}
}
Then the question came again, is it shutdown or shutdownNow? These two methods are often misused, so simply compare the two methods.
ThreadPoolExecutor will become SHUTDOWN after shutdown, unable to accept new tasks, and then wait for the execution of ongoing tasks to complete. It means that shutdown is just a command, and whether it is closed or not depends on the thread itself.
ThreadPoolExecutor’s processing of shutdownNow is different. After the method is executed, it becomes STOP and calls the Thread.interrupt() method on the executing thread (but if the thread does not handle the interrupt, nothing will happen), so it does not Doesn’t mean “close immediately”.
Check the java doc of shutdown and shutdownNow, you will find the following tips:
shutdown() : Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted.Invocation has no additional effect if already shut down.This method does not wait for previously submitted ple.com tecuts toexe Use {@link #awaitTermination awaitTermination} to do that.
shutdownNow(): Attempts to stop all actively executing tasks, halts the processing of waiting tasks, and returns a list of the tasks that were awaiting execution. These tasks are drained (removed) from the task queue upon return from this method.This method does not wait for actively executing tasks to terminate. Use {@link #awaitTermination awaitTermination} to do that. There are no guarantees beyond best-effort attempts to stop processing actively executing tasks. This implementation cancels tasks via {@link Thread#interrupt} , so any task that fails to respond to interrupts may never terminate.
Both of them suggest that we need to execute the awaitTermination method additionally, and it is not enough to just execute shutdown/shutdownNow.
Final solution: Referring to the recycling strategy of the thread pool in spring, we got the final solution.
public abstract class ExecutorConfigurationSupport extends CustomizableThreadFactory
implements DisposableBean{
@Override
public void destroy() {
shutdown();
}
public void shutdown() {
if (this. waitForTasksToCompleteOnShutdown) {
this.executor.shutdown();
}
else {
this.executor.shutdownNow();
}
awaitTerminationIfNecessary();
}
private void awaitTerminationIfNecessary() {
if (this. awaitTerminationSeconds > 0) {
try {
this.executor.awaitTermination(this.awaitTerminationSeconds, TimeUnit.SECONDS));
}
catch (InterruptedException ex) {
Thread. currentThread(). interrupt();
}
}
}
}
The comments are kept, some log codes are removed, and a solution to gracefully close the thread pool is presented before our eyes.
1 Use the waitForTasksToCompleteOnShutdown flag to control whether you want to terminate all tasks immediately, or wait for the task to complete and then exit.
2 executor.awaitTermination(this.awaitTerminationSeconds, TimeUnit.SECONDS)); Control the waiting time to prevent the task from running indefinitely (as emphasized earlier, even shutdownNow cannot guarantee that the thread will stop running).
More graceful shutdown strategies to think about
As mentioned in our series of articles analyzing the principle of RPC, the service governance framework generally takes into account the problem of graceful downtime. The usual practice is to block the traffic beforehand and then shut down the application. A common practice is to remove the service node from the registry, and the subscriber receives the notification, removes the node, and shuts down gracefully; when it comes to database operations, you can use the ACID feature of the transaction to ensure that no abnormal data will appear even if there is a crash shutdown. Not to mention normal offline; for example, message queue can rely on ACK mechanism + message persistence, or transaction message guarantee; for services with many scheduled tasks, special attention should be paid to the problem of graceful shutdown when dealing with offline, because this is A long-running service is more susceptible to downtime problems than other situations. You can use idempotent and flag bits to design timed tasks…
The support of features such as transaction and ACK can make the service as reliable as possible even in the case of downtime, power outage, kill -9 pid, etc.; and we also need to think about kill -15 pid, normal offline, etc. downtime policy. Finally, I will add some understanding of jvm shutdown hook when sorting out this problem.
When the virtual machine begins its shutdown sequence it will start all registered shutdown hooks in some unspecified order and let them run concurrently. When all the hooks have finished it will then run all uninvoked finalizers if finalization-on-exit has been enabled . Finally, the virtual machine will halt.
The shutdown hook will keep the JVM running until the hook is terminated. This also enlightens us that if we execute a blocking operation when receiving the kill -15 pid command, we can wait for the task to complete before closing the JVM. At the same time, it also explains the problem that some applications cannot exit after executing kill -15 pid. Yes, the interrupt is blocked.
Reposted from: https://mp.weixin.qq.com/s/RQaVlxA9uiP0G3GHACzPwQ