Kubernetes on OpenStack with Talos Linux

This repository contains a recipe for deploying a Kubernetes cluster based on Talos Linux on top of an OpenStack cloud infrastructure.

Prerequisites

The talosctl tool enables users to interact with a Talos Kubernetes cluster. To install talosctl on your machine, please use the official installer script (as documented here):

curl -sL https://talos.dev/install | sh

The kubectl tool allows users to interact directly with the Kubernetes control plane. To install kubectl on your machine, please follow the official Kubernetes documentation.

Finally, the Python python-openstackclient library allows users to interact with the OpenStack control plane. You can install the required clients directly from PyPI using pip:

pip install python-openstackclient python-glanceclient python-manilaclient

To enable Kubernetes to authenticate with the OpenStack layer, you have to create an Application Credential and download it into the project's folder. Please ensure that you select at least the member and load-balancer_member roles to grant Kubernetes sufficient permissions to manage the cluster.

Configure the project

If your OpenStack infrastructure does not already provide a Talos image by default, you can upload one yourself in three simple steps:

1. First, download the correct image using the Talos Linux Image Factory. Be sure to specify Cloud Server as the hardware type and OpenStack as the cloud type. At the end of the web form, download the raw image format by clicking on the proper link.

2. To make the image uploading process smoother, convert the downloaded image from raw to qcow2 format using the qemu-img CLI. Use the following command, substituting the ${ARCH} environment variable with the ISA of your OpenStack machines (either amd64 or arm64):

   qemu-img convert -O qcow2 openstack-${ARCH}.raw openstack-${ARCH}.qcow2

3. Finally, upload the image to your OpenStack infrastructure through the following command:

   openstack image create --private --disk-format qcow2 --file openstack-amd64.qcow2 '<image name>'

At this point, you need to configure the characteristics of your target Kubernetes cluster and provide some details about your OpenStack project by filling in the config.sh file.

Set up the Kubernetes cluster

This project provides three scripts that configure a full-fledged Kubernetes cluster on an OpenStack project. The first script, called 01-bootstrap-cluster, creates all the relevant OpenStack objects, deploys the VMs for the Kubernetes control plane and all its workers, and configures the Talos cluster through talosctl. Finally, the script waits for the Talos cluster to be up and running.

If the script fails abruptly while waiting for the Talos cluster to be healthy (e.g., due to an EOF error), you can restart the process manually through the following command:

talosctl --talosconfig talosconfig health

You should wait until the following lines appear in the talosctl log, which indicate that the Kubernetes nodes are ready to communicate but are waiting for the network plane to be configured.

waiting for all k8s nodes to report ready:
waiting for all k8s nodes to report ready: some nodes are not ready: [...]

At this point, you can run the second script, called 02-cilium-cni.sh. This script configures the Kubernetes network plane through the Cilium library. When the script terminates, you can run the talosctl health command again and wait until the Talos cluster reaches its healthy state.

Finally, the 03-openstack-ccm.sh deploys all the relevant components of the OpenStack cloud controller manager within the Kubernetes cluster.

The previous scripts automatically create a kubeconfig file in the project folder, which can be used to access the Kubernetes cluster. You can either copy it into your .kube folder, or extract the cluster data from it and add them to your existing .kube/config file.

Teardown the Kubernetes cluster

The cleanup.sh script deletes the Kubernetes cluster, all related OpenStack objects (security groups, floating IPs), and all Talos-related configuration files. Please be careful that this is an irreversible operation.