Before proceeding in building a DIY Kubernetes cluster, some knowledge of Kubernetes and Linux will be helpful.
There are many blog posts and tutorials about creating a DIY Kubernetes cluster and I thought I’d give it a shot. I found the following sources very helpful in getting my very own cluster up and running:
If you want to use them instead go ahead, otherwise I will take you through what I did and what worked for me.
What to buy?
There are many options when it comes to building a DIY cluster. You can go for the cheap raspberry pi zero’s and build a cluster out of that, or go for the high end mini pc’s that have a lot better spec. I ended up going a for a reasonably pricey 5 node cluster consisting of the following parts:
- Raspberry Pi x5
- Case x1
- SD Cards x5
- Micro USB x5
- Ethernet Cable x5
- Router x1
- Switch x1
- Power Supply x1
If you do intend to purchase these parts, please research other websites first as these links are examples and cheaper options are sure to be available.
Note: Raspberry Pi's run on the ARM architecture. This is important depending on what you are going to use your cluster for as a lot of the containers that you may want to run are built against x86 architecture meaning that it will not run on the Raspberry Pi's. This can be solved by purchasing a UDOO or something similar which has a x86 architecture.
Building the Cluster
Once all the parts have been gathered there will be a few things that need to be done before building the cluster.
First you will need to install a Raspberry Pi OS, I went for Raspbian Jessie Lite, on each of the SD cards. To do this I used a tool called Etcher. Once all the SD cards have been set up you will also need to create and empty file called ‘ssh’ before you insert them into the Raspberry Pi’s.
Next, the cluster can be built so that we get everything up and running. I will talk you through the process of building mine. First, I connected the power supply and the network switch to the mains. I then connected my router to the power supply. Now that the router has power I was then able to configure it to act like a network bridge between my home network and the cluster network. This is handy if you want to use static IP’s for your cluster nodes, otherwise you can connect them directly to your home network. Once the router was configured I connected it to the network switch so that any other devices connected via the network switch had internet access. I then assembled the Raspberry Pi’s and case. This step probably took me the longest as the screws where so small and fiddly. I then inserted the SD cards into the Raspberry Pi’s and connected all of them to the power supply and network switch. Finally, I tidied everything up and this was the end result:
Now that the cluster is built, it is time to get it running Kubernetes. I will be using Kubeadm to create my Kubernetes cluster. To do this you will need to complete the following steps on each of the cluster nodes:
SSH into the node and change the hostname. The name wants to be relative to what the node is going to be. For example, 4 of the nodes in this cluster will be Kubernetes worker nodes, therefore their hostnames could be something like ‘worker1’, ‘worker2’, ‘worker3’ and ‘worker4’. As for the 5th node, it will be the Kubernetes master node, therefore the hostname could be ‘master’. To change the hostname on a Raspberry Pi use the following command:
Reboot the node once the change has been made.
Next, docker will need to be installed and some configuration will need to be done. There is a handy command that does all of this for you given the setup that I have:
curl -sL https://gist.githubusercontent.com/alexellis/fdbc90de7691a1b9edb545c17da2d975/raw/b04f1e9250c61a8ff554bfe3475b6dd050062484/prep.sh | sudo sh
If this does not work or you are curious to know what it does then click here to see where I got it from. Hopefully that answers any thoughts or questions you may have.
Now that everything required to run Kubernetes is installed we can now set up our master and worker nodes.
To setup our master node run the following command:
sudo kubeadm init
A problem I encountered within my cluster when running that command was that it would hang and error when trying to initialise the master node. The following had to be completed in order for me to setup my master node. I created the following config file called kubeadm.yml.
apiVersion: kubeadm.k8s.io/v1alpha1 kind: MasterConfiguration etcd: extraArgs: 'listen-peer-urls': 'http://127.0.0.1:2380'
And used that when initialising the master node.
sudo kubeadm init --config kubeadm.yml
This could take a while so be patient.
Once the master node has initialised we can connect the other worker nodes to the cluster using the command that was outputted.
sudo kubeadm join --token 5aaee9.0afd0c68d95311e5 192.168.1.251:6443 --discovery-token-ca-cert-hash sha256:e759812509474de12fa90d082ebec2fdb0f44182c9630e69781eb5350631056c
Run the command on each node and that should join them to the cluster. Next, we need to check that they have connected, to do this we need to set the Kubernetes configuration up.
When initialising the master node an admin.conf file is created which contains the config you will need to communicate to the cluster using kubectl. There are two ways of setting the Kubernetes configuration. Either setting the KUBECONFIG environment variable or have a .kube/config file set. I will set the .kube/config file. To do this execute the following commands on the master node.
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
This will create a copy of the admin.conf and set it in your .kube/config file.
Note: If you want to access the Kubernetes cluster on a different machine then you will want to copy this config onto the machine that wants access to the cluster. I copied this onto my Windows machine at the following location C:\Users\<User>\.kube\config Then set my KUBECONFIG environment variable to point to that location. CMD: SET KUBECONFIG="C:\Users\<User>\.kube\config" PowerShell: [Environment]::SetEnvironmentVariable("KUBECONFIG", "C:\Users\<User>\.kube\config")
Now that the Kubernetes config is set, we can run the following command to see if all the nodes have clustered.
kubectl get nodes
Awesome! Now we can finish setting up the networking. I took Hanselman’s approach and used weave for networking within the Kubernetes cluster. There are many others which can be viewed here but weave worked straight away for me.
kubectl apply -f https://git.io/weave-kube-1.6
Double check that everything is up and running using:
kubectl get pods --namespace kube-system
And that’s it, I have a running Kubernetes cluster.
One final thing to make developing against your own Raspberry Pi Kubernetes cluster even better is to set up the Kubernetes dashboard. As the cluster is made up of Raspberry Pi’s, the ARM version of the dashboard will need to be deployed.
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/alternative/kubernetes-dashboard-arm.yaml
Note: If this link does not work or you want to use the development manifest they can be found here.
Next I ran,
from my Windows machine to see if I could get to the dashboard. I hit a problem which was to do with the fact that the dashboard was running on a worker node, not a master. I believe this is a problem related with Kubeadm. To solve this I had to modify the ./kubernetes-dashboard-arm.yaml file and add the following to the deployment.
nodeSelector: node-role.kubernetes.io/master: ""
Once I had redeployed the dashboard with that change I was then able to hit the dashboard at:
And there we have it.
I hope this was helpful and if there are any problems or questions related to this post I’ll be happy to answer and resolve them in the comments.