How to setup Multi-node Kubernets Cluster in 10 minutes

Advances in technology have allowed businesses to drastically reduce time needed to setup complex systems and operations. Kubernetes is one such technology and it is a powerful tool for managing an array of microservices and applications. Manually setting up and configuring a Kubernetes cluster can require a significant amount of time and expertise, but in this article we will show you how you can setup a multi-node Kubernetes cluster in just 10 minutes.

First, you need to decide on the type of nodes you will use. Generally, you will want to use virtual machines, such as those provided by Amazon Web Services or Google Compute Engine. Once you have decided, you will need to provision the nodes and make sure they’re all connected to each other.

The next step is to install all the components needed to setup the cluster, such as kubeadm, kubectl, and the Kubernetes control plane. All of these components can be installed easily and fairly quickly. Once the components are installed, you will need to initialize the cluster. This involves giving the nodes information about each other, such as their IP addresses and CIDRs. This enables the nodes to communicate with each other.

The next step is to configure the different services and applications that you wish to run in the cluster. This involves setting up volumes, services, deployments, and replication controllers. This task can take anywhere from a few minutes to a few hours, depending on your needs.

Once everything is set up and running, you can use kubectl to control the cluster and cope with different scenarios. This allows you to quickly deploy applications and manage the entire cluster through a simple command line interface.

How to setup Multi-node Kubernets Cluster. If you have credits on Public cloud accounts, then you can use managed Kubernetes services such as GKE, AKS, EKS etc. But building self-hosted Kubernetes clusters are not straightforward tasks as there are many components to be installed and configured. But there are multiple ways to automate the kubernetes cluster deployment by using infrastructure as code (IaC) tool such as Vagrant and Terraform. In this article, you will learn how to deploy a multi-node Kubernetes cluster using Vagrant and VirtualBox. With this method you can deploy and destroy Kubernetes clusters whenever you needed.

Prerequisites

Vagrant installed
VirtualBox installed
Ansible installed (If you are on Windows, then you can use ansible_local as Ansible is not available to install on Windows. Check the steps to cover this)

Prepare the environment

You need to create your Vagrantfile with necessary instructions and in this case you can use the code from ginigangadharan/vagrant-iac-usecases repository.

Clone the repository and switch to virtualbox-kubernetes directory.

$ git clone https://github.com/ginigangadharan/vagrant-iac-usecases
$ cd vagrant-iac-usecases/virtualbox-kubernetes

Check your Vagrantfile and other files such as Ansible playbooks.

Explaining the `Vagrantfile` content

IMAGE_NAME = "bento/ubuntu-16.04" can be changed with other versions. (Try and report issues if any)
NODES = 2 defines the number of worker nodes in the cluster. Based on this value, Vagrant will create multiple node automatically.
CLUSTER_NAME = "demo" can be changed if you need another cluster in the same workstation (cluster name will be prefixed with VM name to identify)
APISERVER_ADVERTISE_ADDRESS = "192.168.56.50" and NODE_IP_ADDRESS_RANGE = "192.168.56" are used based on vboxnet12 Host-only adaptor. If you using different network, then change this IP accordingly.
Required ports already forwarded as follows

 config.vm.network "forwarded_port",
        guest: 8001,
        host:  8001,
        auto_correct: true
        
        config.vm.network "forwarded_port",
        guest: 30000,
        host:  30000,
        auto_correct: true
        
        config.vm.network "forwarded_port",
        guest: 8443,
        host:  8443,
        auto_correct: true

Ansible will be used as provisioner as follows. If you are on Windows, then instruct Vagrant to use ansible inside the guest VM itself by changing node.vm.provision "ansible" to node.vm.provision "ansible_local".

            node.vm.provision "ansible" do |ansible|
                ansible.playbook = "kubernetes-setup/node-playbook.yml"
                ansible.extra_vars = {
                    node_ip: APISERVER_ADVERTISE_ADDRESS,
                    apiserver_advertise_address: APISERVER_ADVERTISE_ADDRESS
                }
            end

Create the Kubernetes cluster using Vagrant

Build the cluster using Vagrant (Make sure you are inside virtualbox-kubernetes directory)

$ vagrant up

Once Vagrant finished the VM creation and Ansible provisioning, check your VirtualBox GUI for the virtual machines. You can also verify the VM status using Vagrant as follows

$ vagrant status
Current machine states:

k8s-master                running (virtualbox)
node-1                    running (virtualbox)
node-2                    running (virtualbox)

This environment represents multiple VMs. The VMs are all listed
above with their current state. For more information about a specific
VM, run `vagrant status NAME`.

Configure Kubernetes cluster

You can access the Vagrant VMs using vagrant ssh command. Login to the master node as follows.

$ vagrant ssh k8s-master
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.4.0-185-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage


127 packages can be updated.
97 updates are security updates.



This system is built by the Bento project by Chef Software
More information can be found at https://github.com/chef/bento
Last login: Mon Jan 17 03:49:16 2022 from 10.0.2.2
vagrant@demo-k8s-master:~$

Configure KUBECONFIG to access the cluster and check cluster nodes.

vagrant@demo-k8s-master:~$ export KUBECONFIG=$HOME/.kube/config

vagrant@demo-k8s-master:~$ kubectl get nodes
NAME          STATUS     ROLES                  AGE     VERSION
demo-node-1   NotReady   <none>                 3m25s   v1.23.1
demo-node-2   NotReady   <none>                 36s     v1.23.1
k8s-master    NotReady   control-plane,master   7m54s   v1.23.1

You can see the NotReady status on nodes as the cluster networking is not up yet.

Deploy Kubernetes networking

Once the cluster is up and running you need to deploy the Kubernetes networking and in this case we will be using Calico

vagrant@demo-k8s-master:~$ cd /vagrant/
vagrant@demo-k8s-master:/vagrant$ kubectl create -f calico.yaml

Wait for few minutes until Calico pods are up and running.

vagrant@demo-k8s-master:/vagrant$ kubectl get po -n kube-system |grep calico
calico-kube-controllers-d4bfdcb9-ln8lm   1/1     Running   0          6m53s
calico-node-gv56w                        1/1     Running   0          6m53s
calico-node-xwxhz                        1/1     Running   0          6m53s
calico-node-zlkxs                        1/1     Running   0          6m53s

Verify DNS pods

vagrant@demo-k8s-master:/vagrant$ kubectl get po -n kube-system | grep dns
coredns-64897985d-dx6qg                  1/1     Running   0          19m
coredns-64897985d-g4zg9                  1/1     Running   0          19m

Access Kubernetes cluster from host

Instead of accessing the cluster from master nodes, you can access the same cluster from your host machine (Laptop or workstation).

Copy the Kubernetes config file from master node (virtual machine) to host. (I am using vagrant scp command here; feel free to use any method to transfer the file)

$ vagrant scp k8s-master:/home/vagrant/.kube/config ~/.kube/demo-cluster
config                                                                     100% 5641     1.1MB/s   00:00

Set KUBECONFIG environment variable.

$ export KUBECONFIG=~/.kube/demo_cluster

Verify access to the cluster. (Make sure you have installed kubectl on your host machine)

$ kubectl get po -A
NAMESPACE              NAME                                         READY   STATUS    RESTARTS   AGE
kube-system            calico-kube-controllers-d4bfdcb9-p2pmd       1/1     Running   0          28m
kube-system            calico-node-qzs2m                            1/1     Running   0          28m
kube-system            calico-node-rdfhp                            1/1     Running   0          28m
kube-system            calico-node-tf29b                            1/1     Running   0          28m
kube-system            coredns-64897985d-6hmbr                      1/1     Running   0          37m
kube-system            coredns-64897985d-hxgf5                      1/1     Running   0          37m
kube-system            etcd-k8s-master                              1/1     Running   0          37m
kube-system            kube-apiserver-k8s-master                    1/1     Running   0          37m
kube-system            kube-controller-manager-k8s-master           1/1     Running   0          37m
kube-system            kube-proxy-g692n                             1/1     Running   0          34m
kube-system            kube-proxy-vkl62                             1/1     Running   0          37m
kube-system            kube-proxy-zj546                             1/1     Running   0          30m
kube-system            kube-scheduler-k8s-master                    1/1     Running   0          37m
kube-system            metrics-server-6c49cf6978-ccprv              0/1     Running   0          11m
kubernetes-dashboard   dashboard-metrics-scraper-799d786dbf-hl4lw   1/1     Running   0          24m
kubernetes-dashboard   kubernetes-dashboard-6b6b86c4c5-hnsqq        1/1     Running   0          24m

$ kubectl get nodes
NAME          STATUS   ROLES                  AGE   VERSION
demo-node-1   Ready    worker                 34m   v1.23.1
demo-node-2   Ready    worker                 30m   v1.23.1
k8s-master    Ready    control-plane,master   37m   v1.23.1

Install and access Kubernetes dashboard

This is an optional step in case you need to access Kubernetes dashboard from your local machine.

Check documentation for latest version of Kubernetes dashboard.

vagrant@demo-k8s-master:/vagrant$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.2.0/aio/deploy/recommended.yaml

Copy Token Value from below command

vagrant@k8s-master:~$ KUBETOKEN=$(kubectl -n kube-system get secret | grep default-token | awk '{print $1}')

Copy Token Value

vagrant@demo-k8s-master:/vagrant$ kubectl -n kube-system describe secret ${KUBETOKEN} | grep token: | awk '{print $2}'

Note: You can also create a seperate user for accessing the dashboard; refer Creating Sample User documentation.

Enable proxy

vagrant@demo-k8s-master:/vagrant$ kubectl proxy --address='0.0.0.0'
Starting to serve on [::]:8001

Now you can open the url http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/ from you host machine browser (Port forwarding has been enabled as part of Vagrantfile) and login with the token which you have copied from previous command.

Housekeeping

Once you finished testing, you can destroy the entire cluster to save disk space and resources. If you need the cluster for few more days, then you can simply shutdown the virtual machines and start it back whenever needed.

$ vagrant destroy
    node-2: Are you sure you want to destroy the 'node-2' VM? [y/N] y
==> node-2: Forcing shutdown of VM...
==> node-2: Destroying VM and associated drives...
    node-1: Are you sure you want to destroy the 'node-1' VM? [y/N] y
==> node-1: Forcing shutdown of VM...
==> node-1: Destroying VM and associated drives...
    k8s-master: Are you sure you want to destroy the 'k8s-master' VM? [y/N] y
==> k8s-master: Forcing shutdown of VM...
==> k8s-master: Destroying VM and associated drives...

In summary, with the right components and a little bit of knowledge, you can easily and quickly setup a multi-node Kubernetes cluster in just 10 minutes. This cluster then provides an extremely powerful platform for running a variety of applications and services.