Contents

Multinode Kubernetes Cluster with LXC and microk8s

In this post, multinode Kubernetes cluster will be built using lxc and microk8s. It is one of the quickest method to deploy multi-node Kubernetes cluster up and running in minutes.

Installing LXC

You can enable lxc on your system either installing lxc or lxd package.

yilgo@pop-os:~$ sudo apt-get install lxc

OR

yilgo@pop-os:~$ sudo apt-get install lxd

Inital configuration of LXC

After installation, initial setup of lxc is necessary,networking and storage e.g.

yilgo@pop-os:~$ lxd init

By default LXC uses default storage pool in the /var/snap/lxd/common/lxd/storage-pools/default which is not preferable. Because of that new storage will be created pointing to different location.

yilgo@pop-os:~$ sudo mkdir -p /data/lxd/storage-pools/lab
yilgo@pop-os:~$ lxc storage create lab dir source=/data/lxd/storage-pools/lab

Creating Profile for Master and Worker Nodes

yilgo@pop-os:~$ lxc profile create k8sm1
yilgo@pop-os:~$ cat k8sm1.yml | lxc profile edit k8sm1
yilgo@pop-os:~$ lxc profile create k8sw1
yilgo@pop-os:~$ cat k8sw1.yml | lxc profile edit k8sw1
yilgo@pop-os:~$ lxc profile create k8sw2
yilgo@pop-os:~$ cat k8sw2.yml | lxc profile edit k8sw2

You can find all definition file in the GitHub repo.

Note: NIC name in the profile name may be different than yours. Modify profile definition accordingly, incorrect NIC name cause fail on IP address allocation.

yilgo@pop-os:~$ lxc storage list
+---------+--------+------------------------------------------------+-------------+---------+
|  NAME   | DRIVER |                     SOURCE                     | DESCRIPTION | USED BY |
+---------+--------+------------------------------------------------+-------------+---------+
| default | dir    | /var/snap/lxd/common/lxd/storage-pools/default |             | 1       |
+---------+--------+------------------------------------------------+-------------+---------+
| lab     | dir    | /data/lxd/storage-pools/lab                    |             | 0       |
+---------+--------+------------------------------------------------+-------------+---------+
lxc network list
+-----------+----------+---------+---------------+------+-------------+---------+
|   NAME    |   TYPE   | MANAGED |     IPV4      | IPV6 | DESCRIPTION | USED BY |
+-----------+----------+---------+---------------+------+-------------+---------+
| lxdbr0    | bridge   | YES     | 10.5.100.1/24 |      |             | 1       |
+-----------+----------+---------+---------------+------+-------------+---------+

Creating both Master and Worker Nodes

yilgo@pop-os:~$ lxc launch ubuntu:20.04 k8sm1 --profile k8sm1 --vm
yilgo@pop-os:~$ lxc launch ubuntu:20.04 k8sw1 --profile k8sw1 --vm
yilgo@pop-os:~$ lxc launch ubuntu:20.04 k8sw2 --profile k8sw2 --vm

It will take time to installation and post configuration of Kubernetes nodes. After installation, you should see the interfaces similar below.

yilgo@pop-os:~$ lxc list
+-------+---------+----------------------------+------+-----------------+-----------+
| NAME  |  STATE  |            IPV4            | IPV6 |      TYPE       | SNAPSHOTS |
+-------+---------+----------------------------+------+-----------------+-----------+
| k8sm1 | RUNNING | 10.5.100.10 (enp5s0)       |      | VIRTUAL-MACHINE | 0         |
|       |         | 10.1.59.192 (vxlan.calico) |      |                 |           |
+-------+---------+----------------------------+------+-----------------+-----------+
| k8sw1 | RUNNING | 10.5.100.15 (enp5s0)       |      | VIRTUAL-MACHINE | 0         |
|       |         | 10.1.64.64 (vxlan.calico)  |      |                 |           |
+-------+---------+----------------------------+------+-----------------+-----------+
| k8sw2 | RUNNING | 10.5.100.16 (enp5s0)       |      | VIRTUAL-MACHINE | 0         |
|       |         | 10.1.128.0 (vxlan.calico)  |      |                 |           |
+-------+---------+----------------------------+------+-----------------+-----------+

Adding Worker Nodes to Cluster

Get a valid token from master node.

ubuntu@k8sm1:~$ microk8s add-node
From the node you wish to join to this cluster, run the following:
microk8s join 10.5.100.10:25000/0cbf2cf89e20f32e1867af7e2c90adbd/f014c7f6cbb3

Join worker nodes to Kubernetes cluster by providing valid token.

ubuntu@k8sw1:~$ microk8s join 10.5.100.10:25000/0cbf2cf89e20f32e1867af7e2c90adbd/f014c7f6cbb3
Contacting cluster at 10.5.100.10
Waiting for this node to finish joining the cluster. .. 
ubuntu@k8sw2:~$ microk8s join 10.5.100.10:25000/0cbf2cf89e20f32e1867af7e2c90adbd/f014c7f6crt4
Contacting cluster at 10.5.100.10
Waiting for this node to finish joining the cluster. .. 

Labelling Kubernetes Nodes

ubuntu@k8sm1:~$ microk8s kubectl label node k8sm1 node-role.kubernetes.io/master=""
ubuntu@k8sm1:~$ microk8s kubectl label node k8sw1 node-role.kubernetes.io/worker=""
ubuntu@k8sm1:~$ microk8s kubectl label node k8sw2 node-role.kubernetes.io/worker=""
ubuntu@k8sm1:~$ microk8s kubectl get nodes
NAME    STATUS   ROLES    AGE     VERSION
k8sm1   Ready    master   15m     v1.22.4-3+adc4115d990346
k8sw1   Ready    worker   8m38s   v1.22.4-3+adc4115d990346
k8sw2   Ready    worker   8m35s   v1.22.4-3+adc4115d990346

Enabling addons

microk8s enable dns
microk8s enable rbac
microk8s enable ingress
microk8s enable metrics-server
microk8s enable dashboard

Creating Ingress resource.

microk8s kubectl apply -f  kubernetes-dashboard-ingress.yaml
yilgo@pop-os: lxc profile list
+---------+-------------------------------------+---------+
|  NAME   |             DESCRIPTION             | USED BY |
+---------+-------------------------------------+---------+
| default | Default LXD profile                 | 0       |
+---------+-------------------------------------+---------+
| k8sm1   | LXD profile for k8s master node-ONE | 1       |
+---------+-------------------------------------+---------+
| k8sw1   | LXD profile for k8s worker node-ONE | 1       |
+---------+-------------------------------------+---------+
| k8sw2   | LXD profile for k8s worker node-TWO | 1       |
+---------+-------------------------------------+---------+

/Kubernetes/kubernetes-dashboard.webp
kubernetes-dashboard