Introduction
In this article we will explore how to create an operator that can prefetch our images (from our deployments to all nodes) using the Operator SDK, you might be wondering why would you want to do this? the main idea is to get the images in advance so you don’t have to pull them when the pod actually needs to start running in a given node, this can speed up things a bit and it’s also an interesting exercise.
If you have read the article Cloud native applications with kubebuilder and kind aka kubernetes operators you will note that the commands are really similar between each other, since now the operator-sdk uses kubebuilder, you can read more here.
The source for this article is here
Prerequisites
Creating our local cluster
Kind config for multi-cluster
This is the kind config necessary to have a multi-node setup locally: kind create cluster --config kind.yaml
Creating the cluster
We will need a cluster to run and test our operator, so kind is pretty straight forward and lightweight enough to run anywhere.
Creating our operator
Here we bootstrap our go project aka as kubernetes operator
Creating our API
This will be the object that it will hold all the important information for a given image, the files that we need to modify at first hand are in: controllers/*_controller.go and api/v1/*_types.go
Building and pushing (docker image)
Basic build and push of the operator image with the projects helper
Deploying
Now that we have the project built into a docker image and stored in dockerhub then we can install our CRD and then deploy the operator
Deploy the operator
Then we can deploy our operator
Validate that our operator was deployed
Check that our pods are running
So far everything is peachy but our operator is kind of useless at the moment, so let’s drop some code to make it do what we want…
Our code
A lot of what we use is generated however we need to give it some specific permissions and behaviour to our operator so it does what we want when we create an object in kubernetes
Our manifest
This will be the manifest that we will be using to tell our operator which deployments we want to prefetch images for
Sample nginx deployment
This nginx deployment will be used to validate that the images are fetched in all nodes
We don’t actually need to do this, but this way it’s easy to make sure that a pod won’t be scheduled if the label is not present: kubectl label nodes kind-worker3 nginx-schedulable="true"
Our actual logic (this made me chuckle so much bootstrap just to get here, but imagine having to do all that by yourself)
This is where things actually happen, first we get our Spec updated: You can find this file here
Then we can put some code, I will add more comments later in the code to explain what everything does: Basically what we do is set a timer to create a pod in each node to force it fetch the image that the deployments (that we filter by labels) needs or is going to use, by doing this if the node already has the image nothing happens and it will be removed in the next run, however if the image is not there it will be fetched so if anything happens and a pod needs to be actually scheduled there it won’t need to download everything so it should be relatively faster. You can find this file here
What we should be seeing in our cluster
Cleaning up
To clean up the operator from the cluster you can do, and also remember to clean up your clusters or whatever you are using if it’s in the cloud to avoid unexpected bills
Closing notes
Be sure to check the links if you want to learn more about the project and I hope you enjoyed it, see you on twitter or github!
- https://sdk.operatorframework.io/docs/building-operators/golang/tutorial/
- https://sdk.operatorframework.io/docs/building-operators/golang/operator-scope/
- https://opensource.com/article/20/3/kubernetes-operator-sdk
The source for this article is here
Errata
If you spot any error or have any suggestion, please send me a message so it gets fixed.
Also, you can check the source code and changes in the generated code and the sources here