Multinode Kubernetes Cluster with LXC and microk8s
Contents
In this post, multinode Kubernetes cluster will be built using lxc and microk8s. It is one of the quickest method to deploy multi-node
Kubernetes cluster up and running in minutes.
Installing LXC
You can enable lxc on your system either installing lxc or lxd package.
yilgo@pop-os:~$ sudo apt-get install lxc
OR
yilgo@pop-os:~$ sudo apt-get install lxd
Inital configuration of LXC
After installation, initial setup of lxc is necessary,networking and storage e.g.
yilgo@pop-os:~$ lxd init
By default LXC uses default storage pool in the /var/snap/lxd/common/lxd/storage-pools/default which is not preferable.
Because of that new storage will be created pointing to different location.
yilgo@pop-os:~$ sudo mkdir -p /data/lxd/storage-pools/lab
yilgo@pop-os:~$ lxc storage create lab dir source=/data/lxd/storage-pools/lab
Creating Profile for Master and Worker Nodes
yilgo@pop-os:~$ lxc profile create k8sm1
yilgo@pop-os:~$ cat k8sm1.yml | lxc profile edit k8sm1
yilgo@pop-os:~$ lxc profile create k8sw1
yilgo@pop-os:~$ cat k8sw1.yml | lxc profile edit k8sw1
yilgo@pop-os:~$ lxc profile create k8sw2
yilgo@pop-os:~$ cat k8sw2.yml | lxc profile edit k8sw2
You can find all definition file in the GitHub repo.
Note: NIC name in the profile name may be different than yours. Modify profile definition accordingly, incorrect NIC name cause fail on IP address allocation.
yilgo@pop-os:~$ lxc storage list
+---------+--------+------------------------------------------------+-------------+---------+
| NAME | DRIVER | SOURCE | DESCRIPTION | USED BY |
+---------+--------+------------------------------------------------+-------------+---------+
| default | dir | /var/snap/lxd/common/lxd/storage-pools/default | | 1 |
+---------+--------+------------------------------------------------+-------------+---------+
| lab | dir | /data/lxd/storage-pools/lab | | 0 |
+---------+--------+------------------------------------------------+-------------+---------+
lxc network list
+-----------+----------+---------+---------------+------+-------------+---------+
| NAME | TYPE | MANAGED | IPV4 | IPV6 | DESCRIPTION | USED BY |
+-----------+----------+---------+---------------+------+-------------+---------+
| lxdbr0 | bridge | YES | 10.5.100.1/24 | | | 1 |
+-----------+----------+---------+---------------+------+-------------+---------+
Creating both Master and Worker Nodes
yilgo@pop-os:~$ lxc launch ubuntu:20.04 k8sm1 --profile k8sm1 --vm
yilgo@pop-os:~$ lxc launch ubuntu:20.04 k8sw1 --profile k8sw1 --vm
yilgo@pop-os:~$ lxc launch ubuntu:20.04 k8sw2 --profile k8sw2 --vm
It will take time to installation and post configuration of Kubernetes nodes. After installation, you should see the interfaces similar below.
yilgo@pop-os:~$ lxc list
+-------+---------+----------------------------+------+-----------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-------+---------+----------------------------+------+-----------------+-----------+
| k8sm1 | RUNNING | 10.5.100.10 (enp5s0) | | VIRTUAL-MACHINE | 0 |
| | | 10.1.59.192 (vxlan.calico) | | | |
+-------+---------+----------------------------+------+-----------------+-----------+
| k8sw1 | RUNNING | 10.5.100.15 (enp5s0) | | VIRTUAL-MACHINE | 0 |
| | | 10.1.64.64 (vxlan.calico) | | | |
+-------+---------+----------------------------+------+-----------------+-----------+
| k8sw2 | RUNNING | 10.5.100.16 (enp5s0) | | VIRTUAL-MACHINE | 0 |
| | | 10.1.128.0 (vxlan.calico) | | | |
+-------+---------+----------------------------+------+-----------------+-----------+
Adding Worker Nodes to Cluster
Get a valid token from master node.
ubuntu@k8sm1:~$ microk8s add-node
From the node you wish to join to this cluster, run the following:
microk8s join 10.5.100.10:25000/0cbf2cf89e20f32e1867af7e2c90adbd/f014c7f6cbb3
Join worker nodes to Kubernetes cluster by providing valid token.
ubuntu@k8sw1:~$ microk8s join 10.5.100.10:25000/0cbf2cf89e20f32e1867af7e2c90adbd/f014c7f6cbb3
Contacting cluster at 10.5.100.10
Waiting for this node to finish joining the cluster. ..
ubuntu@k8sw2:~$ microk8s join 10.5.100.10:25000/0cbf2cf89e20f32e1867af7e2c90adbd/f014c7f6crt4
Contacting cluster at 10.5.100.10
Waiting for this node to finish joining the cluster. ..
Labelling Kubernetes Nodes
ubuntu@k8sm1:~$ microk8s kubectl label node k8sm1 node-role.kubernetes.io/master=""
ubuntu@k8sm1:~$ microk8s kubectl label node k8sw1 node-role.kubernetes.io/worker=""
ubuntu@k8sm1:~$ microk8s kubectl label node k8sw2 node-role.kubernetes.io/worker=""
ubuntu@k8sm1:~$ microk8s kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8sm1 Ready master 15m v1.22.4-3+adc4115d990346
k8sw1 Ready worker 8m38s v1.22.4-3+adc4115d990346
k8sw2 Ready worker 8m35s v1.22.4-3+adc4115d990346
Enabling addons
microk8s enable dns
microk8s enable rbac
microk8s enable ingress
microk8s enable metrics-server
microk8s enable dashboard
Creating Ingress resource.
microk8s kubectl apply -f kubernetes-dashboard-ingress.yaml
yilgo@pop-os: lxc profile list
+---------+-------------------------------------+---------+
| NAME | DESCRIPTION | USED BY |
+---------+-------------------------------------+---------+
| default | Default LXD profile | 0 |
+---------+-------------------------------------+---------+
| k8sm1 | LXD profile for k8s master node-ONE | 1 |
+---------+-------------------------------------+---------+
| k8sw1 | LXD profile for k8s worker node-ONE | 1 |
+---------+-------------------------------------+---------+
| k8sw2 | LXD profile for k8s worker node-TWO | 1 |
+---------+-------------------------------------+---------+
