diff --git a/CHANGELOG.md b/CHANGELOG.md index 6c277b5a..b6c1ad78 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ All notable changes to this project will be documented in this file. - Replace stackable-operator `print_startup_string` with `tracing::info!` with fields. - BREAKING: Inject the vector aggregator address into the vector config using the env var `VECTOR_AGGREGATOR_ADDRESS` instead of having the operator write it to the vector config ([#551]). +- Document that Spark Connect doesn't integrate with the history server ([#559]) ### Fixed @@ -28,6 +29,7 @@ All notable changes to this project will be documented in this file. [#551]: https://github.com/stackabletech/spark-k8s-operator/pull/551 [#553]: https://github.com/stackabletech/spark-k8s-operator/pull/553 [#554]: https://github.com/stackabletech/spark-k8s-operator/pull/554 +[#559]: https://github.com/stackabletech/spark-k8s-operator/pull/559 ## [25.3.0] - 2025-03-21 diff --git a/docs/modules/spark-k8s/pages/usage-guide/spark-connect.adoc b/docs/modules/spark-k8s/pages/usage-guide/spark-connect.adoc index d4a0307d..bf9bf1d1 100644 --- a/docs/modules/spark-k8s/pages/usage-guide/spark-connect.adoc +++ b/docs/modules/spark-k8s/pages/usage-guide/spark-connect.adoc @@ -45,11 +45,15 @@ spec: The example above adds a new endpoint for application metrics. +== Spark History Server + +Unforunately integration with the Spark History Server is not supported yet. +The connect server seems to ignore the `spark.eventLog` properties while also prohibiting clients to set them programatically. + == Notable Omissions The following features are not supported by the Stackable Spark operator yet -* Integration with the Spark History Server. * Authorization and authentication. Currently, anyone with access to the Spark Connect service can run jobs. * Volumes and volume mounts can be added only with pod overrides. * Job dependencies must be provisioned as custom images or via `--packages` or `--jars` arguments.