Why we need kubernetes Jobs
As you learned in the previous chapters, a Pod created via a Deployment, StatefulSet, or DaemonSet, runs continuously. When the process running in one of the Pod’s containers terminates, the Kubelet restarts the container. The Pod never stops on its own, but only when you delete the Pod object. Although this is ideal for running web servers, databases, system services, and similar workloads, it’s not suitable for finite workloads that only need to perform a single task.
A finite workload doesn’t run continuously, but lets a task run to completion. In Kubernetes, you run this type of workload using the Job resource. However, a Job always runs its Pods immediately, so you can’t use it for scheduling tasks. For that, you need to wrap the Job in a CronJob object. This allows you to schedule the task to run at a specific time in the future or at regular intervals.
In this chapter you’ll learn about Jobs and CronJobs.
Introducing the Job resource
The Job resource resembles a Deployment in that it creates one or more Pods, but instead of ensuring that those Pods run indefinitely, it only ensures that a certain number of them complete successfully.
As you can see in the following figure, the simplest Job runs a single Pod to completion, whereas more complex Jobs run multiple Pods, either sequentially or concurrently. When all containers in a Pod terminate with success, the Pod is completed. When all the Pods have completed, the Job itself is also completed.
Three different Job examples. Each Job is completed once its Pods have completed successfully.
Working with the Job resource
Creating Job resource
Here is an example Job config. It computes π to 2000 places and prints it out. It takes around 10s to complete.
root@AlexRampUpVM-01:/tmp# cat job_pi.yaml
apiVersion: batch/v1
Kind: Job
metadata:
name:pi
spec:
template:
spec:
containers:
- name: pi
image: perl:5.34.0
command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]
restartPolicy: Never
backoffLimit: 4
root@AlexRampUpVM-01:/tmp# kubectl apply -f job_pi.yaml
job.batch/pi created
root@AlexRampUpVM-01:/tmp# kubectl get job
NAME COMPLETIONS DURATION AGE
pi 0/1 7s 7s
Displaying the detailed Job status
To see more details about the Job, use the kubectl describe command as follows:
root@AlexRampUpVM-01:/tmp# kubectl describe jobs.batch pi
Name:pi
Namespace: default
Selector: controller-uid=17deea3a-6a9b-4402-bfbb-18456762306f
Labels: controller-uid=17deea3a-6a9b-4402-bfbb-18456762306f
job-name=pi
Annotations: batch.kubernetes.io/job-tracking:
Parallelism: 1
Completions: 1
Completion Mode: NonIndexed
Start Time: Tue, 24 Oct 2023 08:18:13 + 0000
Completed At: Tue, 24 Oct 2023 08:19:02 + 0000
Duration: 49s
Pods Statuses: 0 Active (0 Ready) / 1 Succeeded / 0 Failed
Pod Template:
Labels: controller-uid=17deea3a-6a9b-4402-bfbb-18456762306f
job-name=pi
Containers:
pi:
Image: perl:5.34.0
Port: <none>
Host Port: <none>
Command:
perl
-Mbignum=bpi
-wle
print bpi(2000)
Environment: <none>
Mounts: <none>
Volumes: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 106s job-controller Created pod: pi-8f4zg
Normal Completed 57s job-controller Job completed
View the standard output of the jobs:
root@AlexRampUpVM-01:/tmp# kubectl logs jobs/pi 3.14159265358979323846264338327950288419716939937510582097494459230781640628620899862803482534211706...
Deleting the Job
To delete the specific job, you can run the command kubectl delete jobs . for example,
root@AlexRampUpVM-01:/tmp# kubectl delete jobs pi -n default job.batch "pi" deleted
Introducing the CronJob resource
When you create a Job object, it starts executing immediately. Although you can create the Job in a suspended state and later un-suspend it, you cannot configure it to run at a specific time. To achieve this, you can wrap the Job in a CronJob object.
In the CronJob object you specify a Job template and a schedule. According to this schedule, the CronJob controller creates a new Job object from the template. You can set the schedule to do this several times a day, at a specific time of day, or on specific days of the week or month. The controller will continue to create Jobs according to the schedule until you delete the CronJob object. The following figure illustrates how a CronJob works.
As you can see in the figure, each time the CronJob controller creates a Job, the Job controller subsequently creates the Pod(s), just like when you manually create the Job object. Let’s see this process in action.
Working with the CronJob resource
Creating a CronJob
This example CronJob manifest prints the current time and a hello message every minute:
root@AlexRampUpVM-01:/tmp# cat cronjob_hello.yaml
apiVersion: batch/v1
Kind: CronJob
metadata:
name: hello
spec:
schedule: "* * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: hello
image: busybox:1.28
imagePullPolicy: IfNotPresent
command:
-/bin/sh
- -c
- date; echo Hello from the Kubernetes cluster
restartPolicy: OnFailure
root@AlexRampUpVM-01:/tmp# kubectl apply -f cronjob_hello.yaml
cronjob.batch/hello created
Inspecting the CronJob status in detail
Apply the manifest file to create the CronJob. Use the kubectl get cj command to check the object:
root@AlexRampUpVM-01:/tmp# kubectl get cronjob NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE hello * * * * * False 0 40s 6m33s
The kubectl get cronjobs command only shows the number of currently active Jobs and when the last Job was scheduled. Unfortunately, it doesn’t show whether the last Job was successful. To get this information, you can either list the Jobs directly or check the CronJob status in YAML form as follows:
root@AlexRampUpVM-01:/tmp# kubectl get cronjob hello -o yaml
apiVersion: batch/v1
Kind: CronJob
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"batch/v1","kind":"CronJob","metadata":{"annotations":{},"name":\ "hello","namespace":"default"},"spec":{"jobTemplate":{"spec":{"template":{"spec\ ":{"containers":[{"command":["/bin/sh","-c","date; echo Hello from the Kubernetes cluster"]," image":"busybox:1.28","imagePullPolicy":"IfNotPresent","name":"hello"}],"restartPolicy":"OnFailure"} }}},"schedule":"* * * * *"}}
creationTimestamp: "2023-10-24T08:27:07Z"
generation: 1
name: hello
namespace:default
resourceVersion: "18560481"
uid: 4f5ae695-9f63-422f-8aaf-52cba08db4f4
spec:
concurrencyPolicy: Allow
failedJobsHistoryLimit: 1
jobTemplate:
metadata:
creationTimestamp: null
spec:
template:
metadata:
creationTimestamp: null
spec:
containers:
- command:
-/bin/sh
- -c
- date; echo Hello from the Kubernetes cluster
image: busybox:1.28
imagePullPolicy: IfNotPresent
name: hello
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy:ClusterFirst
restartPolicy: OnFailure
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
schedule: '* * * * *'
successfulJobsHistoryLimit: 3
suspend: false
status:
lastScheduleTime: "2023-10-24T08:34:00Z"
lastSuccessfulTime: "2023-10-24T08:34:05Z"
As you can see, the status section of a CronJob object shows a list with references to the currently running Jobs (field active), the last time the Job was scheduled (field lastScheduleTime), and the last time the Job completed successfully (field lastSuccessfulTime ). From the last two fields you can deduce whether the last run was successful.
Inspecting Events associated with a CronJob
To see the full details of a CronJob and all Events associated with the object, use the kubectl describe command as follows:
root@AlexRampUpVM-01:/tmp# kubectl describe cronjob hello
Name: hello
Namespace: default
Labels: <none>
Annotations: <none>
Schedule: * * * * *
Concurrency Policy: Allow
Suspend: False
Successful Job History Limit: 3
Failed Job History Limit: 1
Starting Deadline Seconds: <unset>
Selector: <unset>
Parallelism: <unset>
Completions: <unset>
Pod Template:
Labels: <none>
Containers:
hello:
Image: busybox:1.28
Port: <none>
Host Port: <none>
Command:
/bin/sh
-c
date; echo Hello from the Kubernetes cluster
Environment: <none>
Mounts: <none>
Volumes: <none>
Last Schedule Time: Tue, 24 Oct 2023 08:29:00 + 0000
Active Jobs: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 100s cronjob-controller Created job hello-28302268
Normal SawCompletedJob 90s cronjob-controller Saw completed job: hello-28302268, status: Complete
Normal SuccessfulCreate 40s cronjob-controller Created job hello-28302269
Normal SawCompletedJob 36s cronjob-controller Saw completed job: hello-28302269, status: Complete
As can be seen in the command output, the CronJob controller generates a SuccessfulCreate Event when it creates a Job, and a SawCompletedJob Event when the Job completes.
Deleting the CronJob
To delete the specific cronjob, you can run the command kubectl delete cronjob . for example,
root@AlexRampUpVM-01:/tmp# kubectl delete cronjobs hello -n default cronjob.batch "hello" deleted