feat(nifi): Add NiFi OPA Authorizer (#1058)

* build nifi-opa-plugin from source with workaround

* add changelog entry

* add patch

* build nifi-opa-plugin from branch feat/reworked-opa-response

* Update nifi/Dockerfile

Co-authored-by: Malte Sander <[email protected]>

* chore(⏲): Use fast download mirrors (#1061)

* chore: Use fast download mirrors

* chore: Show download progress bar

* chore: Improve gpg key warning message

* chore: Show upload progress bar

Thanks to <https://bashupload.com/how_to_upload_progress_curl>

* Apply suggestions from code review

Co-authored-by: Techassi <[email protected]>

* chore: Make output more consistent

---------

Co-authored-by: Techassi <[email protected]>

* chore(⏲): Add missing tool update tasks to templates (#1062)

* chore(issue_templates): Use YY.M.X placeholders

* chore(issue_templates): Remove tasklists

:sob:

* chore(issue_templates): Add missing tool update tasks for cyclonedx and auditable

* feat: move patch apply logic to patchable (#1032)

* wip

* Update druid/Dockerfile

Co-authored-by: Natalie Klestrup Röijezon <[email protected]>

* fix: remove unnecessary check / shadow repo root var

* fix: druid src path

* fix: druid src path

* feat: introduce stackable-devel image

* fix: use PathBuf in ProductVersionContext

* chore: align zookeeper patch directory structure

* fix: stackable-devel dnf and shell config

* chore: switch patch process in other products

* fix: hive build

* fix: trino build

* fix: spark build

* chore: make hadolint happy

* fix: remove hbase intermediate sources / remove unnecessary and operator

* fix: permissions in patchable build process

* chore: remove unnecessary curl command in build process

* chore: move adding of JMX config and start-metastore script from builder stage to final stage

* chore: remove git repo in trino and hbase-operator-tools to avoid maven commit plugin bug

---------

Co-authored-by: Natalie Klestrup Röijezon <[email protected]>

* fix: add missing patchable config for Kafka 3.8.0 (#1065)

* fix(pr-template): Fix the rendering of the list (#1073)

* feat: make image namespace a workflow input (default sdp) (#1072)

* feat: make image namespace a workflow input (default sdp)

* feat: add registry namespace input to build workflows and remove default

* ci(mirror): Include image name and version in run-name (#1089)

* ci(mirror): Include image and version in run-name

* chore: Use colon

* fix: spark connect client Harbor credentials (#1088)

* fix: spark connect client Harbor credentials (#1092)

* feat: connect client image includes JupyterLab (#1071)

* feat: install demo dependencies

* spark-connect-client is now built directly off of spark-k8s

* run pre-commit hooks

* fix shellcheck sc2102

* chore: Bump tools ahead of 25.7.0 (#1090)

* chore(jmx_exporter): Bump products to use 1.2.0

* chore(tools): Bump kubectl (1.33.0 and yq (4.45.2)

* chore(cyclonedx-bom): Bump to 6.0.0

* chore: Update changelog

* chore: Changelog formatting

* chore(java): Add JDK 24 (#1097)

* chore(java): Add JDK 24

* chore: Update changelog

* chore(vector): Bump to 0.46.1 (#1098)

* chore(vector): Bump to 0.46.1

* chore(vector): Bump products to use 0.46.1

* chore(zookeeper): Remove 3.9.2 (#1093)

* chore: Changelog formatting

* chore(zookeeper): Remove 2.9.2

* chore: Update changelog

* chore: Update changelog

* chore(ubi-rust-builders): Update container images ahead of Stackable Release 25.7.0 (#1091)

* chore(ubi-rust-builders): Remove ubi8-rust-builder

* chore(ubi-rust-builders): Update base image and use protoc 30.2

* Apply suggestions from code review

Co-authored-by: Techassi <[email protected]>
Co-authored-by: Sebastian Bernauer <[email protected]>

---------

Co-authored-by: Techassi <[email protected]>
Co-authored-by: Sebastian Bernauer <[email protected]>

* chore(opa): Update versions ahead of 25.7.0 (#1103)

* chore(opa): Remove 0.67.1

* chore(opa): Remove legacy bundle-builder

* chore(opa): Bump ubi9 base image

* chore(opa): Add 1.4.2

* fix(opa): Manually install Go version

NOTE: The dnf package was 1.23.6 and OPA required 1.23.8
NOTE: I tried making the version paramaterized, but bake wouldn't allow
it (it worked fine with `docker build` and `docker build xbuild`).

* ci(mirror): Add golang

* chore(opa): Use mirrored golang image

* chore(opa): Parameterise golang container version

* chore: Update changelog

* fix: Bump vector to 0.46.1 for java-base:24 (#1104)

* chore(superset): Add 4.1.2 (#1102)

* chore(superset): Add 4.1.2

* fix(superset): Remove invalid parts of the file

* chore: Update changelog

* feat(nifi): Add nifi-iceberg-bundle (#1060)

* feat(nifi): Add nifi-iceberg-bundle

* changelog

* Add SBOM to final image

* Use version 0.0.1

* hadolint

* Bump to 0.0.2

* Bump to 0.0.3

* fix(nifi): Delete correct intermediate folder (#1106)

* fix(nifi): Delete correct intermediate folder

* changelog

* fix Dockerfile

* build nifi opa plugin from tag

* Update CHANGELOG.md

Co-authored-by: Malte Sander <[email protected]>

* fix hadolint

* Update nifi/Dockerfile

Co-authored-by: Malte Sander <[email protected]>

---------

Co-authored-by: Malte Sander <[email protected]>
Co-authored-by: Nick <[email protected]>
Co-authored-by: Techassi <[email protected]>
Co-authored-by: Lukas Krug <[email protected]>
Co-authored-by: Natalie Klestrup Röijezon <[email protected]>
Co-authored-by: Razvan-Daniel Mihai <[email protected]>
Co-authored-by: Sebastian Bernauer <[email protected]>

Author: 8 people

CHANGELOG.md

@@ -38,6 +38,8 @@ All notable changes to this project will be documented in this file.
   `check-permissions-ownership.sh` provided in stackable-base image ([#1025]).
 - zookeeper: check for correct permissions and ownerships in /stackable folder via
   `check-permissions-ownership.sh` provided in stackable-base image ([#1043]).
+- nifi: Build and add OPA authorizer plugin nar ([#1058]).
+- nifi: Add [nifi-iceberg-bundle](https://github.com/stackabletech/nifi-iceberg-bundle) for NiFi `2.2.0` ([#1060], [#1106]).
 - java: Add JDK 24 ([#1097]).
 - ci: Add golang image to mirror workflow ([#1103]).
 - omid: bump version to 1.1.3 ([#1105])
@@ -111,6 +113,7 @@ All notable changes to this project will be documented in this file.
 [#1054]: https://github.com/stackabletech/docker-images/pull/1054
 [#1055]: https://github.com/stackabletech/docker-images/pull/1055
 [#1056]: https://github.com/stackabletech/docker-images/pull/1056
+[#1058]: https://github.com/stackabletech/docker-images/pull/1058
 [#1060]: https://github.com/stackabletech/docker-images/pull/1060
 [#1090]: https://github.com/stackabletech/docker-images/pull/1090
 [#1091]: https://github.com/stackabletech/docker-images/pull/1091

nifi/Dockerfile

@@ -97,7 +97,7 @@ mkdir -p /stackable
 
 # NiFI 1.x natively supports Iceberg, no need to build an iceberg-bundle for it
 if [[ "${PRODUCT}" != 1.* ]] ; then
-    curl "https://github.com/stackabletech/nifi-iceberg-bundle/archive/refs/tags/${NIFI_ICEBERG_BUNDLE}.tar.gz" | tar -xzC .
+    curl -L "https://github.com/stackabletech/nifi-iceberg-bundle/archive/refs/tags/${NIFI_ICEBERG_BUNDLE}.tar.gz" | tar -xzC .
     cd nifi-iceberg-bundle-${NIFI_ICEBERG_BUNDLE} || exit
 
     sed -i -e "s/{{ NIFI_VERSION }}/${PRODUCT}/g" pom.xml
@@ -126,6 +126,35 @@ chmod g=u /stackable/*.cdx.json
 fi
 EOF
 
+FROM stackable/image/java-devel AS opa-authorizer-builder
+
+ARG NIFI_OPA_AUTHORIZER_PLUGIN
+ARG STACKABLE_USER_UID
+ARG PRODUCT
+
+USER ${STACKABLE_USER_UID}
+WORKDIR /build
+
+RUN <<EOF
+mkdir -p /stackable
+
+curl -L "https://github.com/DavidGitter/nifi-opa-plugin/archive/refs/tags/v${NIFI_OPA_AUTHORIZER_PLUGIN}.tar.gz" | tar -xzC .
+cd nifi-opa-plugin-${NIFI_OPA_AUTHORIZER_PLUGIN}/authorizer || exit
+
+mvn \
+    --batch-mode \
+    --no-transfer-progress \
+    clean package \
+    -DskipTests \
+    -Pnifi-${PRODUCT}
+
+cp ./target/opa-authorizer.nar /stackable/opa-authorizer.nar
+cp ../LICENSE /stackable/LICENSE
+
+# Set correct permissions
+chmod g=u /stackable/opa-authorizer.nar
+EOF
+
 FROM stackable/image/java-base AS final
 
 ARG PRODUCT
@@ -146,6 +175,8 @@ COPY --chown=${STACKABLE_USER_UID}:0 --from=nifi-iceberg-bundle-builder /stackab
 COPY --chown=${STACKABLE_USER_UID}:0 --from=nifi-iceberg-bundle-builder /stackable/*.cdx.json /stackable/nifi-${PRODUCT}/lib/
 COPY --chown=${STACKABLE_USER_UID}:0 --from=nifi-builder /stackable/git-sync /stackable/git-sync
 
+COPY --chown=${STACKABLE_USER_UID}:0 --from=opa-authorizer-builder /stackable/opa-authorizer.nar /stackable/nifi-${PRODUCT}/extensions/opa-authorizer.nar
+COPY --chown=${STACKABLE_USER_UID}:0 --from=opa-authorizer-builder /stackable/LICENSE /licenses/NIFI_OPA_PLUGIN_LICENSE
 COPY --chown=${STACKABLE_USER_UID}:0 nifi/stackable/bin /stackable/bin
 COPY --chown=${STACKABLE_USER_UID}:0 nifi/licenses /licenses
 COPY --chown=${STACKABLE_USER_UID}:0 nifi/python /stackable/python

nifi/versions.py

@@ -4,18 +4,21 @@
         "java-base": "11",
         "java-devel": "11",  # There is an error when trying to use the jdk 21 (since nifi 1.26.0)
         "git_sync": "v4.4.0",
+        "nifi_opa_authorizer_plugin": "0.1.0",
     },
     {
         "product": "1.28.1",
         "java-base": "11",
         "java-devel": "11",
         "git_sync": "v4.4.0",
+        "nifi_opa_authorizer_plugin": "0.1.0",
     },
     {
         "product": "2.4.0",
         "java-base": "21",
         "java-devel": "21",
         "git_sync": "v4.4.0",
         "nifi_iceberg_bundle": "0.0.4",
+        "nifi_opa_authorizer_plugin": "0.1.0",
     },
 ]