Graceful shutdown of Application inside Pods in Kubernetes
As in any environment, even in a containerised environment, it’s highly desirable to have a graceful shutdown of the server process. We can explore how the Pod’s termination lifecycle events in Kubernetes play their part.
Let’s assume we are running a Spring Boot based API or an Application which uses tomcat server is containerised and deployed on a Kubernetes cluster.
In a Pod lifecycle, it will be created, killed, redeployed or get crashed and then restarted, any number of times. As a developer we should expect these scenarios and take measures to handle these gracefully.
So our goal here is to gracefully shutdown the application process, in this case the tomcat server of the Spring Boot application, so that there is a minimal impact on the end user.
Lets see the events involved in the termination lifecycle of Kubernetes
- Pod is set to Terminating state
- Execution of preStop hook
- SIGTERM signal
- Grace Period Wait
- SIGKILL Signal
In this post, we are more focused on the Step 2 & 4, preStop and Grace Period wait time.
The preStop hook is a special command or a request sent to the containers in the pod. We can use this command to execute steps needed for the proper shutdown of the required process.
Most applications will not need this explicit step as SIGTERM signal to the container will invoke the necessary shutdown process but preStop gives the option to invoke any process or command which needs to be executed before the kill.
In the above Pod definition, preStop hook is used to explicitly invoke the stop command on Tomcat server.
Three commands are passed in our example.
- Write a shutdown message into /usr/share/message1
- Sleep 10 (Just enough time to view logs in this example)
- Catalina.sh script to stop the tomcat server.
As you can see, this gives the capability to execute user defined custom actions before the Pod shutdown.
Note: If the PreStop handler fails, event FailedPreStopHook is broadcasted. We can examine these events by running the following command
kubectl describe pod <pod_name>
In Kubernetes, pod definitions, we can specify a time period which will make K8S to wait for the process to end gracefully and this can be achieved by defining the property terminationGracePeriodSeconds
The default value for terminationGracePeriodSeconds is 30 seconds. We can define a custom value for this property based on the application running in the container.
In the above sample, we have defined a timeout of 45 seconds.
With either or both of these Kubernetes options, we can achieve graceful shutdown of our deployed applications..
Since Pods can be created through Kubernetes Deployment objects, Pods can be updated or deployed using Rolling Updates. This is an important feature which will help in the end user experience.. But more on this in my another post…
Check out my CKAD certification post if you are interested in getting one…