In this post, multinode Kubernetes cluster will be built using lxc and microk8s. It is one of the quickest method to deploy multi-node Kubernetes cluster up and running in minutes.
Installing LXC
You can enable lxc on your system either installing lxc or lxd package.
yilgo@pop-os:~$ sudo apt-get install lxc
yilgo@pop-os:~$ sudo apt-get install lxd
After installation, initial setup of lxc is necessary,networking and storage e.g.
yilgo@pop-os:~$ lxd init
By default LXC uses default storage pool in the /var/snap/lxd/common/lxd/storage-pools/default which is not preferable. Because of that new storage will be created pointing to different location.
By default LXC uses default storage pool in the /var/snap/lxd/common/lxd/storage-pools/default which is not preferable. Because of that new storage will be created pointing to different location.
yilgo@pop-os:~$ sudo mkdir -p /data/lxd/storage-pools/lab
yilgo@pop-os:~$ lxc storage create lab dir source=/data/lxd/storage-pools/lab
Creating Profile for Master and Worker Nodes
yilgo@pop-os:~$ lxc profile create k8sm1
yilgo@pop-os:~$ cat k8sm1.yml | lxc profile edit k8sm1
yilgo@pop-os:~$ lxc profile create k8sw1
yilgo@pop-os:~$ cat k8sw1.yml | lxc profile edit k8sw1
yilgo@pop-os:~$ lxc profile create k8sw2
yilgo@pop-os:~$ cat k8sw2.yml | lxc profile edit k8sw2
You can find all definition file in the GitHub repo.
Note: NIC name in the profile name may be different than yours. Modify profile definition accordingly, incorrect NIC name cause fail on IP address allocation.
yilgo@pop-os:~$ lxc storage list
+---------+--------+------------------------------------------------+-------------+---------+
| NAME | DRIVER | SOURCE | DESCRIPTION | USED BY |
+---------+--------+------------------------------------------------+-------------+---------+
| default | dir | /var/snap/lxd/common/lxd/storage-pools/default | | 1 |
+---------+--------+------------------------------------------------+-------------+---------+
| lab | dir | /data/lxd/storage-pools/lab | | 0 |
+---------+--------+------------------------------------------------+-------------+---------+
lxc network list
+-----------+----------+---------+---------------+------+-------------+---------+
| NAME | TYPE | MANAGED | IPV4 | IPV6 | DESCRIPTION | USED BY |
+-----------+----------+---------+---------------+------+-------------+---------+
| lxdbr0 | bridge | YES | 10.5.100.1/24 | | | 1 |
+-----------+----------+---------+---------------+------+-------------+---------+
Creating both Master and Worker Nodes
yilgo@pop-os:~$ lxc launch ubuntu:20.04 k8sm1 --profile k8sm1 --vm
yilgo@pop-os:~$ lxc launch ubuntu:20.04 k8sw1 --profile k8sw1 --vm
yilgo@pop-os:~$ lxc launch ubuntu:20.04 k8sw2 --profile k8sw2 --vm
It will take time to installation and post configuration of Kubernetes nodes. After installation, you should see the interfaces similar below.
yilgo@pop-os:~$ lxc list
+-------+---------+----------------------------+------+-----------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-------+---------+----------------------------+------+-----------------+-----------+
| k8sm1 | RUNNING | 10.5.100.10 (enp5s0) | | VIRTUAL-MACHINE | 0 |
| | | 10.1.59.192 (vxlan.calico) | | | |
+-------+---------+----------------------------+------+-----------------+-----------+
| k8sw1 | RUNNING | 10.5.100.15 (enp5s0) | | VIRTUAL-MACHINE | 0 |
| | | 10.1.64.64 (vxlan.calico) | | | |
+-------+---------+----------------------------+------+-----------------+-----------+
| k8sw2 | RUNNING | 10.5.100.16 (enp5s0) | | VIRTUAL-MACHINE | 0 |
| | | 10.1.128.0 (vxlan.calico) | | | |
+-------+---------+----------------------------+------+-----------------+-----------+
Adding Worker Nodes to Cluster
Get a valid token from master node.
ubuntu@k8sm1:~$ microk8s add-node
From the node you wish to join to this cluster, run the following:
microk8s join 10.5.100.10:25000/0cbf2cf89e20f32e1867af7e2c90adbd/f014c7f6cbb3
Join worker nodes to Kubernetes cluster by providing valid token.
ubuntu@k8sw1:~$ microk8s join 10.5.100.10:25000/0cbf2cf89e20f32e1867af7e2c90adbd/f014c7f6cbb3
Contacting cluster at 10.5.100.10
Waiting for this node to finish joining the cluster. ..
ubuntu@k8sw2:~$ microk8s join 10.5.100.10:25000/0cbf2cf89e20f32e1867af7e2c90adbd/f014c7f6crt4
Contacting cluster at 10.5.100.10
Waiting for this node to finish joining the cluster. ..
Labelling Kubernetes Nodes
ubuntu@k8sm1:~$ microk8s kubectl label node k8sm1 node-role.kubernetes.io/master=""
ubuntu@k8sm1:~$ microk8s kubectl label node k8sw1 node-role.kubernetes.io/worker=""
ubuntu@k8sm1:~$ microk8s kubectl label node k8sw2 node-role.kubernetes.io/worker=""
ubuntu@k8sm1:~$ microk8s kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8sm1 Ready master 15m v1.22.4-3+adc4115d990346
k8sw1 Ready worker 8m38s v1.22.4-3+adc4115d990346
k8sw2 Ready worker 8m35s v1.22.4-3+adc4115d990346
Enabling addons
microk8s enable dns
microk8s enable rbac
microk8s enable ingress
microk8s enable metrics-server
microk8s enable dashboard
Creating Ingress Resource
microk8s kubectl apply -f kubernetes-dashboard-ingress.yaml
yilgo@pop-os: lxc profile list
+---------+-------------------------------------+---------+
| NAME | DESCRIPTION | USED BY |
+---------+-------------------------------------+---------+
| default | Default LXD profile | 0 |
+---------+-------------------------------------+---------+
| k8sm1 | LXD profile for k8s master node-ONE | 1 |
+---------+-------------------------------------+---------+
| k8sw1 | LXD profile for k8s worker node-ONE | 1 |
+---------+-------------------------------------+---------+
| k8sw2 | LXD profile for k8s worker node-TWO | 1 |
+---------+-------------------------------------+---------+
