Skip to content

Commit 2231531

Browse files
richardcaserichardcase
authored
Merge pull request #144 from nasusoba/addsamples2
docs: updates to README, add samples to `CAPD` and `CAPZ`
2 parents 9b72920 + a2c798a commit 2231531

File tree

11 files changed

+514
-406
lines changed

11 files changed

+514
-406
lines changed

‎README.md

Lines changed: 167 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,55 +1,189 @@
1-
# Cluster API k3s
1+
# Cluster API Provider k3s
22

3-
Cluster API bootstrap provider k3s (CABP3) is a component of [Cluster API](https://github.com/kubernetes-sigs/cluster-api/blob/master/README.md) that is responsible for generating a cloud-init script to turn a Machine into a Kubernetes Node; this implementation brings up [k3s](https://k3s.io/) clusters instead of full kubernetes clusters.
3+
Cluster API Provider k3s provides the following [Cluster API](https://github.com/kubernetes-sigs/cluster-api) (CAPI) providers:
44

5-
CABP3 is the bootstrap component of Cluster API for k3s and brings in the following CRDS and controllers:
6-
- k3s bootstrap provider (KThrees, KThreesTemplate)
5+
- **Cluster API Bootstrap Provider k3s (CABP3)** is responsible for generating the instructions (and encoding them as cloud-init) to turn a Machine into a Kubernetes Node; this implementation brings up [k3s](https://k3s.io/) clusters instead of full kubernetes clusters.
6+
- **Cluster API ControlPlane Provider k3s (CACP3)** is responsible for managing the lifecycle of control plane machines for k3s; this implementation brings up [k3s](https://k3s.io/) clusters instead of full kubernetes clusters.
77

8-
Cluster API ControlPlane provider k3s (CACP3) is a component of [Cluster API](https://github.com/kubernetes-sigs/cluster-api/blob/master/README.md) that is responsible for managing the lifecycle of control plane machines for k3s; this implementation brings up [k3s](https://k3s.io/) clusters instead of full kubernetes clusters.
8+
## Getting Started
99

10-
CACP3 is the controlplane component of Cluster API for k3s and brings in the following CRDS and controllers:
11-
- k3s controlplane provider (KThreesControlPlane)
10+
**Warning**: Project and documentation are in an early stage, there is an assumption that a user of this provider is already familiar with Cluster API. Please consider contributing.
1211

13-
Together these two components make up Cluster API k3s...
12+
### Prerequisites
1413

15-
## Testing it out.
14+
Check out the general [Cluster API Quickstart](https://cluster-api.sigs.k8s.io/user/quick-start.html) page to see the prerequisites for Cluster API.
1615

17-
**Warning**: Project and documentation are in an early stage, there is an assumption that an user of this provider is already familiar with ClusterAPI.
16+
Three main pieces are:
1817

19-
### Prerequisites
18+
1. Management cluster. In the `samples/azure/azure-setup.sh` script, [k3d](https://k3d.io/) is used, but feel free to use [kind](https://kind.sigs.k8s.io/) as well .
19+
2. clusterctl. Please check out [Cluster API Quickstart](https://cluster-api.sigs.k8s.io/user/quick-start.html) for instructions.
20+
3. Infrastructure specific prerequisites:
21+
- For more Azure information go to [CAPZ Getting Started](https://capz.sigs.k8s.io/topics/getting-started.html)
22+
- For more AWS information go to [CAPA Getting Started](https://cluster-api-aws.sigs.k8s.io/)
23+
- For more Nutanix information go to [CAPX Getting Started](https://opendocs.nutanix.com/capx/latest/getting_started/)
24+
- For more OpenStack information go to [CAPO Getting Started](https://cluster-api.sigs.k8s.io/user/quick-start.html)
25+
- For more Vsphere information go to [CAPV Getting Started](https://cluster-api.sigs.k8s.io/user/quick-start.html)
2026

21-
Check out the [ClusterAPI Quickstart](https://cluster-api.sigs.k8s.io/user/quick-start.html) page to see the prerequisites for ClusterAPI.
27+
In this getting started guide we'll be using Docker as the infrastructure provider (CAPD).
2228

23-
Three main pieces are
29+
### Create a management cluster
2430

25-
1. Bootstrap cluster. In the `samples/azure/azure-setup.sh` script, I use [k3d](https://k3d.io/), but feel free to use [kind](https://kind.sigs.k8s.io/) as well.
26-
2. clusterctl. Please check out [ClusterAPI Quickstart](https://cluster-api.sigs.k8s.io/user/quick-start.html) for instructions.
27-
3. Infrastructure Specific Prerequisites:
31+
1. Ensure kind is installed ([instructions](https://kind.sigs.k8s.io/docs/user/quick-start/#installation))
32+
2. Create a kind configuration to expose the local docker socket:
2833

29-
* For more Azure information go to [CAPZ Getting Started](https://capz.sigs.k8s.io/topics/getting-started.html)
30-
* For more AWS information go to [CAPA Getting Started](https://cluster-api-aws.sigs.k8s.io/)
31-
* For more Nutanix information go to [CAPX Getting Started](https://opendocs.nutanix.com/capx/latest/getting_started/)
32-
* For more OpenStack information go to [CAPO Getting Started](https://cluster-api.sigs.k8s.io/user/quick-start.html)
33-
* For more Vsphere information go to [CAPV Getting Started](https://cluster-api.sigs.k8s.io/user/quick-start.html)
34+
```bash
35+
cat > kind-cluster-with-extramounts.yaml <<EOF
36+
kind: Cluster
37+
apiVersion: kind.x-k8s.io/v1alpha4
38+
name: capi-quickstart
39+
nodes:
40+
- role: control-plane
41+
extraMounts:
42+
- hostPath: /var/run/docker.sock
43+
containerPath: /var/run/docker.sock
44+
EOF
45+
```
3446

35-
Cluster API k3s has been tested on AWS, Azure, AzureStackHCI, Nutanix, OpenStack, and Vsphere environments.
47+
3. Run the following command to create the cluster:
3648

37-
To try out the Azure flow, fork the repo and look at `samples/azure/azure-setup.sh`.
49+
```bash
50+
kind create cluster --config kind-cluster-with-extramounts.yaml
51+
```
3852

39-
To try out the AWS flow, fork the repo and look at `samples/aws/aws-setup.sh`.
53+
4. Check that you can connect to the cluster
4054

41-
To try out the Nutanix flow, fork the repo and look at `samples/nutanix/nutanix-setup.sh`.
55+
```bash
56+
kubectl get pods -A
57+
```
4258

43-
To try out the OpenStack flow, fork the repo and look at `samples/openstack/setup.sh`.
59+
### Install the providers
4460

45-
To try out the Vsphere flow, fork the repo and look at `samples/vsphere-capv/setup.sh`.
61+
1. Create a file called `clusterctl.yaml` in the `$HOME/.config/cluster-api` directory with the following content:
4662

47-
### Known Issues
63+
```yaml
64+
# NOTE: the following will be changed to use 'latest' in the future
65+
providers:
66+
- name: "k3s"
67+
url: "https://github.com/k3s-io/cluster-api-k3s/releases/v0.2.1/bootstrap-components.yaml"
68+
type: "BootstrapProvider"
69+
- name: "k3s"
70+
url: "https://github.com/k3s-io/cluster-api-k3s/releases/v0.2.1/control-plane-components.yaml"
71+
type: "ControlPlaneProvider"
72+
```
4873
49-
## Roadmap
74+
> This configuration tells clusterctl where to look for the provider manifests. You could run `clusterctl config -h` to check default clusterctl configuration file path.
75+
76+
2. Install the providers:
77+
78+
```bash
79+
clusterctl init --bootstrap k3s --control-plane k3s --infrastructure docker
80+
```
81+
82+
3. Wait for the pods to start
83+
84+
### Create a workload cluster
85+
86+
There are a number of different cluster templates in the [samples](./samples/) directory.
87+
88+
> Note: there is an issue with CAPD, it would be better you could do this setup beforehand. [Cluster API with Docker - &#34;too many open files&#34;.](https://cluster-api.sigs.k8s.io/user/troubleshooting.html?highlight=too%20many#cluster-api-with-docker----too-many-open-files)
89+
90+
1. Run the following command to generate your cluster definition:
91+
92+
```bash
93+
export KIND_IMAGE_VERSION=v1.30.0
94+
clusterctl generate cluster --from samples/docker/cluster-template-quickstart.yaml test1 --kubernetes-version v1.30.2+k3s2 --worker-machine-count 2 --control-plane-machine-count 1 > cluster.yaml
95+
```
96+
97+
> NOTE: the kubernetes version specified with the k3s suffix `+k3s2`.
98+
99+
2. Check the contents of the generated cluster definition in **cluster.yaml**
100+
3. Ensure the definition is valid by doing a dry run:
101+
102+
```bash
103+
kubectl apply -f cluster.yaml --dry-run=server
104+
```
105+
106+
4. When you are happy apply the definition:
107+
108+
```bash
109+
kubectl apply -f cluster.yaml
110+
```
111+
112+
### Check the workload cluster
113+
114+
- Check the state of the CAPI machines:
115+
116+
```bash
117+
kubectl get machine
118+
```
119+
120+
- Get the kubeconfig for the cluster:
50121

51-
* Support for External Databases
52-
* Fix Token Logic
53-
* Clean up Control Plane Provider Code
54-
* Post an issue!
122+
```bash
123+
clusterctl get kubeconfig test1 > workload-kubeconfig.yaml
124+
```
125+
126+
> Note: if you are using Docker Desktop, you need to fix the kubeconfig by running:
127+
128+
```bash
129+
# Point the kubeconfig to the exposed port of the load balancer, rather than the inaccessible container IP.
130+
sed -i -e "s/server:.*/server: https:\/\/$(docker port test1-lb 6443/tcp | sed "s/0.0.0.0/127.0.0.1/")/g" ./workload-kubeconfig.yaml
131+
```
132+
133+
- Connect to the child cluster
134+
135+
```bash
136+
kubectl --kubeconfig workload-kubeconfig.yaml get pods -A
137+
```
138+
139+
### Deleting the workload cluster
140+
141+
When deleting a cluster created via CAPI you must delete the top level **Cluster** resource. DO NOT delete using the original file.
142+
143+
For the quick start:
144+
145+
```bash
146+
kubectl delete cluster test1
147+
```
148+
149+
### Additional Samples
150+
151+
Cluster API k3s has been tested on AWS, Azure, AzureStackHCI, Nutanix, OpenStack, Docker and Vsphere environments.
152+
153+
- To try out the Azure flow, fork the repo and look at `samples/azure/azure-setup.sh`.
154+
- To try out the AWS flow, fork the repo and look at `samples/aws/aws-setup.sh`.
155+
- To try out the Nutanix flow, fork the repo and look at `samples/nutanix/nutanix-setup.sh`.
156+
- To try out the OpenStack flow, fork the repo and look at `samples/openstack/setup.sh`.
157+
- To try out the Vsphere flow, fork the repo and look at `samples/vsphere-capv/setup.sh`.
158+
159+
## Developer Setup
160+
161+
You could also build and install CABP3 and CACP3 from src:
162+
163+
```sh
164+
# Build image with `dev` tag
165+
make BOOTSTRAP_IMG_TAG=dev docker-build-bootstrap
166+
make CONTROLPLANE_IMG_TAG=dev docker-build-controlplane
167+
168+
# Push image to your registry
169+
export REGISTRY="localhost:5001" # Set this to your local/remote registry
170+
docker tag ghcr.io/k3s-io/cluster-api-k3s/controlplane-controller:dev ${REGISTRY}/controlplane-controller:dev
171+
docker tag ghcr.io/k3s-io/cluster-api-k3s/bootstrap-controller:dev ${REGISTRY}/bootstrap-controller:dev
172+
docker push ${REGISTRY}/controlplane-controller:dev
173+
docker push ${REGISTRY}/bootstrap-controller:dev
174+
175+
# Install CAPI k3s to management cluster
176+
make install-controlplane # install CRDs
177+
make install-bootstrap
178+
make CONTROLPLANE_IMG=${REGISTRY}/controlplane-controller CONTROLPLANE_IMG_TAG=dev deploy-controlplane # deploy the component
179+
make BOOTSTRAP_IMG=${REGISTRY}/bootstrap-controller BOOTSTRAP_IMG_TAG=dev deploy-bootstrap
180+
```
181+
182+
For easy development, please refer to [tilt-setup.md](docs/tilt-setup.md).
183+
184+
## Roadmap
55185

186+
- Support for External Databases
187+
- Fix Token Logic
188+
- Clean up Control Plane Provider Code
189+
- Post an issue!

0 commit comments

Comments
 (0)