stroom-kubernetes

This repository contains Helm charts that assist with the configuration and deployment of a Stroom stack to a Kubernetes cluster.

It is being developed against the Stroom v6.1 branch, with the intent of being forwards-compatible with v7 once that becomes stable.

This project uses gchq/stroom-resources as a reference guide, though it is not dependent on it.

1. Design goals

• Integrate Stroom into production Kubernetes environments using current and supported tooling (such as Helm)
• Replace the existing bespoke bootstrap scripts and resources, with a collection of Helm charts. This improves maintainability and simplifies on-boarding
• Simplify deployment, such as by removing the need to track and allocate port assignments for Stroom stack components
• Remove Nginx reverse proxies, replacing them with Kubernetes Ingress resources
• Simplify operations management, by allowing admins to use cluster management tools like Rancher to deploy and configure Stroom resources
• Support both production-ready deployments and development environments (e.g. using Minikube), while using the same architecture and code base
• Attain performance parity (or near to), compared to bare-metal installs
• Improve observability through Prometheus exporters
• Improve security by implementing Kubernetes hardening guidelines

2. Feature status

This project is currently in alpha.

The following Helm charts have been completed to the point of being functional:

• Core Stroom stack
  • Processing / UI node
  • Dedicated processing / UI nodes
  • MySQL (single instance, no replication)
  • Zookeeper
  • Kafka
  • Authentication service and UI
• Stroom services
  • Stroom stats
  • Stroom proxy
  • Solr
  • Log sender
  • HBase
  • HDFS
• Prometheus exporters
• Logging and audit transport
• Security hardening
  • TLS for all services
  • Non-root containers
  • Service accounts and cluster role bindings
• Documentation (integration with gchq/stroom-docs?)

3. Getting started

Prerequisites

• A Kubernetes cluster, such as Minikube or K3s
• Nginx Kubernetes ingress controller deployed. Other ingress controllers may be supported in future (via custom annotations)
• Helm installed
• A DNS entry pointing to the Kubernetes Ingress. Recommendation is for this DNS record to point to a high-availability load balancer, such as what's described in the Rancher docs
• A Kubernetes TLS secret for that DNS entry, created in the same namespace that Stroom will be deployed into
• Enough available (i.e. unbound) persistent volumes to support the number of replicas requested of each Helm chart. The choice of storage provider is entirely up to you. Ensure each persistent volume has at least the capacity configured for the relevant component, otherwise it won't be bound by the Persistent Volume Claim (PVC) created by the Helm chart, and the Stroom stack won't start up.

Stage files

Clone this repository to a directory on a machine that has kubectl and Kubernetes cluster admin rights

Deploy

1. cd into ./charts/stroom
2. Make a custom configuration file, using ./values/example.yaml as a guide
3. Ensure the following are set in your custom config file:
4. global.advertisedHost. FQDN of the Kubernetes ingress (e.g. stroom.example.com)
5. global.ingress.tls.secretName. Name of the TLS secret
6. Create a namespace for the Stroom deployment: kubectl create namespace stroom-dev
7. Deploy the Helm chart, applying your customisations: helm install -n stroom-dev -f <path to custom config YAML> stroom .

Stroom will now deploy to the namespace you created (in this case: stroom-dev). Cluster resources will be named using the release name stroom as a base.

To check on the status of the deployment, execute a command like: watch kubectl get all -n stroom-dev.

Upgrade

1. Clone the updated repository
2. Follow steps 1 - 3 as described above
3. Upgrade the chart: helm upgrade -n stroom-dev -f <path to custom config YAML> stroom .

Open Stroom

Open the following in your web browser: https://<advertised URL>

4. Contributing

As this project is still early in development, it is not yet open to public contributions.