Multinode Kubernetes Cluster with LXC and microk8s
Contents
In this post, multinode Kubernetes cluster will be built using lxc and microk8s. It is one of the quickest method to deploy multi-node
Kubernetes cluster up and running in minutes.
Installing LXC
You can enable lxc on your system either installing lxc or lxd package.
[email protected]:~$ sudo apt-get install lxc
OR
[email protected]:~$ sudo apt-get install lxd
Inital configuration of LXC
After installation, initial setup of lxc is necessary,networking and storage e.g.
[email protected]:~$ lxd init
By default LXC uses default storage pool in the /var/snap/lxd/common/lxd/storage-pools/default which is not preferable.
Because of that new storage will be created pointing to different location.
[email protected]:~$ sudo mkdir -p /data/lxd/storage-pools/lab
[email protected]:~$ lxc storage create lab dir source=/data/lxd/storage-pools/lab
Creating Profile for Master and Worker Nodes
[email protected]:~$ lxc profile create k8sm1
[email protected]:~$ cat k8sm1.yml | lxc profile edit k8sm1
[email protected]:~$ lxc profile create k8sw1
[email protected]:~$ cat k8sw1.yml | lxc profile edit k8sw1
[email protected]:~$ lxc profile create k8sw2
[email protected]:~$ cat k8sw2.yml | lxc profile edit k8sw2
You can find all definition file in the GitHub repo.
Note: NIC name in the profile name may be different than yours. Modify profile definition accordingly, incorrect NIC name cause fail on IP address allocation.
[email protected]:~$ lxc storage list
+---------+--------+------------------------------------------------+-------------+---------+
| NAME | DRIVER | SOURCE | DESCRIPTION | USED BY |
+---------+--------+------------------------------------------------+-------------+---------+
| default | dir | /var/snap/lxd/common/lxd/storage-pools/default | | 1 |
+---------+--------+------------------------------------------------+-------------+---------+
| lab | dir | /data/lxd/storage-pools/lab | | 0 |
+---------+--------+------------------------------------------------+-------------+---------+
lxc network list
+-----------+----------+---------+---------------+------+-------------+---------+
| NAME | TYPE | MANAGED | IPV4 | IPV6 | DESCRIPTION | USED BY |
+-----------+----------+---------+---------------+------+-------------+---------+
| lxdbr0 | bridge | YES | 10.5.100.1/24 | | | 1 |
+-----------+----------+---------+---------------+------+-------------+---------+
Creating both Master and Worker Nodes
[email protected]:~$ lxc launch ubuntu:20.04 k8sm1 --profile k8sm1 --vm
[email protected]:~$ lxc launch ubuntu:20.04 k8sw1 --profile k8sw1 --vm
[email protected]:~$ lxc launch ubuntu:20.04 k8sw2 --profile k8sw2 --vm
It will take time to installation and post configuration of Kubernetes nodes. After installation, you should see the interfaces similar below.
[email protected]:~$ lxc list
+-------+---------+----------------------------+------+-----------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-------+---------+----------------------------+------+-----------------+-----------+
| k8sm1 | RUNNING | 10.5.100.10 (enp5s0) | | VIRTUAL-MACHINE | 0 |
| | | 10.1.59.192 (vxlan.calico) | | | |
+-------+---------+----------------------------+------+-----------------+-----------+
| k8sw1 | RUNNING | 10.5.100.15 (enp5s0) | | VIRTUAL-MACHINE | 0 |
| | | 10.1.64.64 (vxlan.calico) | | | |
+-------+---------+----------------------------+------+-----------------+-----------+
| k8sw2 | RUNNING | 10.5.100.16 (enp5s0) | | VIRTUAL-MACHINE | 0 |
| | | 10.1.128.0 (vxlan.calico) | | | |
+-------+---------+----------------------------+------+-----------------+-----------+
Adding Worker Nodes to Cluster
Get a valid token from master node.
[email protected]:~$ microk8s add-node
From the node you wish to join to this cluster, run the following:
microk8s join 10.5.100.10:25000/0cbf2cf89e20f32e1867af7e2c90adbd/f014c7f6cbb3
Join worker nodes to Kubernetes cluster by providing valid token.
[email protected]:~$ microk8s join 10.5.100.10:25000/0cbf2cf89e20f32e1867af7e2c90adbd/f014c7f6cbb3
Contacting cluster at 10.5.100.10
Waiting for this node to finish joining the cluster. ..
[email protected]:~$ microk8s join 10.5.100.10:25000/0cbf2cf89e20f32e1867af7e2c90adbd/f014c7f6crt4
Contacting cluster at 10.5.100.10
Waiting for this node to finish joining the cluster. ..
Labelling Kubernetes Nodes
[email protected]:~$ microk8s kubectl label node k8sm1 node-role.kubernetes.io/master=""
[email protected]:~$ microk8s kubectl label node k8sw1 node-role.kubernetes.io/worker=""
[email protected]:~$ microk8s kubectl label node k8sw2 node-role.kubernetes.io/worker=""
[email protected]:~$ microk8s kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8sm1 Ready master 15m v1.22.4-3+adc4115d990346
k8sw1 Ready worker 8m38s v1.22.4-3+adc4115d990346
k8sw2 Ready worker 8m35s v1.22.4-3+adc4115d990346
Enabling addons
microk8s enable dns
microk8s enable rbac
microk8s enable ingress
microk8s enable metrics-server
microk8s enable dashboard
Creating Ingress resource.
microk8s kubectl apply -f kubernetes-dashboard-ingress.yaml
[email protected]: lxc profile list
+---------+-------------------------------------+---------+
| NAME | DESCRIPTION | USED BY |
+---------+-------------------------------------+---------+
| default | Default LXD profile | 0 |
+---------+-------------------------------------+---------+
| k8sm1 | LXD profile for k8s master node-ONE | 1 |
+---------+-------------------------------------+---------+
| k8sw1 | LXD profile for k8s worker node-ONE | 1 |
+---------+-------------------------------------+---------+
| k8sw2 | LXD profile for k8s worker node-TWO | 1 |
+---------+-------------------------------------+---------+
