NETOBSERV-2225 - Deploy static plugin at operator startup #1345

jpinsonneau · 2025-04-02T09:38:40Z

Description

Create the console plugin when FlowCollector doesn't exists to expose the new forms.

~~Suggested alternatives: #1346 & #1374~~

See netobserv/network-observability-console-plugin#763 for the forms implementations

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

jpinsonneau · 2025-04-02T10:05:44Z

controllers/flowcollector_controller.go

+	// force reconcile at startup
+	go r.InitReconcile(ctx)
+
 	return nil
 }

+func (r *FlowCollectorReconciler) InitReconcile(ctx context.Context) error {
+	log := log.FromContext(ctx)
+	log.Info("Initializing resources...")
+
+	var err error
+	for attempt := range initReconcileAttempts {
+		// delay the reconcile calls to let some time to the cache to load
+		time.Sleep(5 * time.Second)
+		_, err = r.Reconcile(ctx, reconcile.Request{})
+		if err != nil {
+			log.Error(err, "Error while doing initial reconcile", "attempt", attempt)
+		} else {
+			break
+		}
+	}
+	return err
+}


☝️ I wonder if there is an out of box mechanism to trigger the loop after the cache loaded. That's why I'm using a sleep here and this will may work in all situations.

https://redhat-internal.slack.com/archives/C02939DP5L5/p1743518264445429

If I remember correctly, when a reconcile loop is failing, you can also return a time value to reschedule the reconciliation.

Yeah I gave a try with that without success.

I'm refactoring the code again to move the static content to another controller wich will be cleaner I guess. I will give another try with the reschedule time on the new controller 👍

jpinsonneau · 2025-04-17T08:24:43Z

controllers/static/static_controller.go

+	// force reconcile at startup
+	go r.InitReconcile(ctx)
+
+	return ctrl.NewControllerManagedBy(mgr).
+		For(&flowslatest.FlowCollector{}, reconcilers.IgnoreStatusChange).
+		Named("staticPlugin").
+		Complete(&r)
+}
+
+func (r *Reconciler) InitReconcile(ctx context.Context) {
+	log := log.FromContext(ctx)
+	log.Info("Initializing resources...")
+
+	for attempt := range initReconcileAttempts {
+		// delay the reconcile calls to let some time to the cache to load
+		time.Sleep(5 * time.Second)
+		_, err := r.Reconcile(ctx, ctrl.Request{})
+		if err != nil {
+			log.Error(err, "Error while doing initial reconcile", "attempt", attempt)
+		} else {
+			return
+		}
+	}
+}


@OlivierCazade I modified the PR to be in a dedicated controller.

As you can see, I force the Reconcile call during the Start function so OLM can't interpret the result containing Requeue / RequeueAfter here.

Since we don't create a dedicated CR for static plugin, I don't think we can rely on these here.

WDYT ?

github-actions · 2025-05-06T14:33:09Z

New images:

quay.io/netobserv/network-observability-operator:1c54f0e
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-sha-1c54f0e
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-sha-1c54f0e

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:1c54f0e make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-sha-1c54f0e

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-sha-1c54f0e
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

jpinsonneau · 2025-05-06T14:42:48Z

@memodi FYI I did some changes to address #1345 (comment) comment

Just tested and the behavior remains the same 😉
It avoid using an unecessary service account and role

memodi · 2025-05-13T15:23:02Z

/jira NETOBSERV-2225

jotak · 2025-05-14T05:41:53Z

controllers/static/static_controller.go

+	log := log.FromContext(ctx)
+	log.Info("Initializing resources...")
+
+	for attempt := range initReconcileAttempts {


btw, what happens if all 5 attempts fail? The static plugin wouldn't deploy, but the rest would work normally? Or does it make the controller CLBO or so ? (my understanding is it's the first, but just want to make sure)

If the attempts fails, the static plugin will not be there at controller startup.
As soon as a reconcile loop is triggered, it will appears (ie creating a FlowCollector for example)

jotak · 2025-05-14T05:44:53Z

controllers/static/static_controller.go

+
+	r.status.SetUnknown()
+	defer r.status.Commit(ctx, r.Client)
+


After setting "Unknown", I think this controller should return if openshift isn't detected, right?
We could check it in Kind

That's done in the static reconciler using HasConsolePlugin function:

network-observability-operator/controllers/consoleplugin/consoleplugin_static_reconciler.go

Lines 40 to 77 in 1080b69

func (r *CPReconciler) reconcileStatic(ctx context.Context, desired *flowslatest.FlowCollector) error {

l := log.FromContext(ctx).WithName("console-plugin")

ctx = log.IntoContext(ctx, l)

// Retrieve current owned objects

err := r.Managed.FetchAll(ctx)

if err != nil {

return err

}

if r.ClusterInfo.HasConsolePlugin() {

if err = r.checkAutoPatch(ctx, desired, constants.StaticPluginName); err != nil {

return err

}

}

if r.ClusterInfo.HasConsolePlugin() {

// Create object builder

builder := newBuilder(r.Instance, &desired.Spec, constants.StaticPluginName)

if err = r.reconcilePlugin(ctx, &builder, &desired.Spec, constants.StaticPluginName, "NetObserv static plugin"); err != nil {

return err

}

if err = r.reconcileDeployment(ctx, &builder, &desired.Spec, constants.StaticPluginName, ""); err != nil {

return err

}

if err = r.reconcileServices(ctx, &builder, constants.StaticPluginName); err != nil {

return err

}

} else {

// delete any existing owned object

r.Managed.TryDeleteAll(ctx)

}

return nil

}

If console plugin is not available, nothing is deployed and the status goes ready

The static controller could deploy something else than the console plugin in future so I think it's better to keeps things separated here. WDYT ?

ah ok, yes sounds good, thanks!

jotak

Just a comment when not running on openshift, other than that lgtm

jpinsonneau · 2025-05-14T08:05:25Z

/restest

jotak · 2025-05-14T12:20:33Z

/lgtm

memodi · 2025-07-14T15:47:01Z

@jpinsonneau - could you rebase this PR please? I tried locally using make commands but running into other issues.

openshift-ci · 2025-07-15T08:40:34Z

New changes are detected. LGTM label has been removed.

openshift-ci · 2025-07-15T08:40:38Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from jotak. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

jpinsonneau · 2025-07-15T08:41:28Z

Rebased without changes

memodi · 2025-07-15T20:10:50Z

/ok-to-test

github-actions · 2025-07-15T20:15:08Z

New images:

quay.io/netobserv/network-observability-operator:cdd8f5a
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-sha-cdd8f5a
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-sha-cdd8f5a

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:cdd8f5a make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-sha-cdd8f5a

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-sha-cdd8f5a
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

memodi · 2025-07-15T21:00:58Z

@jpinsonneau Here's quick feedback from my initial review of the form view:

- ebpf flow filtering
	- what does option 1 and 2 mean for port fields? [1]
- Deployment model
- Mode:
	- add a note which Loki mode is recommended for production use cases. 
- Prometheus Mode:
	- add a node what "Auto" mode mean? Or hide it if its not relevant 
- pipeline stage:
	- flow filtering.
	- some advanced configuration like "secondaryNetworks" could be exposed.

[1]

at the end it generated empty yaml [2] , I was trying with 4.20 OCP version

I'll add more feedback as I test more.

openshift-ci · 2025-07-15T23:31:20Z

@jpinsonneau: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-operator	`43c5e6c`	link	false	`/test e2e-operator`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

jpinsonneau force-pushed the 1942 branch from 0b47945 to efab0a0 Compare April 2, 2025 10:04

jpinsonneau commented Apr 2, 2025

View reviewed changes

jotak added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 2, 2025

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 2, 2025

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 2, 2025

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 2, 2025

jotak mentioned this pull request Apr 7, 2025

Create static console plugin in OLM bundle #1346

Closed

jpinsonneau force-pushed the 1942 branch from 5fb53db to c5da751 Compare April 14, 2025 09:04

netobserv deleted a comment from github-actions bot Apr 14, 2025

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 14, 2025

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 14, 2025

jpinsonneau force-pushed the 1942 branch 2 times, most recently from c4c57e1 to dbc4cbf Compare April 15, 2025 10:40

netobserv deleted a comment from github-actions bot Apr 15, 2025

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 15, 2025

jpinsonneau force-pushed the 1942 branch from dbc4cbf to 82efd81 Compare April 15, 2025 10:59

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 15, 2025

netobserv deleted a comment from github-actions bot Apr 15, 2025

openshift-merge-robot added the needs-rebase label Apr 15, 2025

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 15, 2025

jpinsonneau force-pushed the 1942 branch from 75f2aac to 82efd81 Compare April 15, 2025 12:39

openshift-merge-robot removed the needs-rebase label Apr 15, 2025

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 15, 2025

netobserv deleted a comment from github-actions bot Apr 15, 2025

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 15, 2025

jpinsonneau force-pushed the 1942 branch from 82efd81 to 0a42edc Compare April 17, 2025 08:21

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 17, 2025

jpinsonneau commented Apr 17, 2025

View reviewed changes

netobserv deleted a comment from github-actions bot Apr 17, 2025

netobserv deleted a comment from github-actions bot May 6, 2025

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label May 6, 2025

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label May 6, 2025

jpinsonneau added the needs-review Tells that the PR needs a review label May 13, 2025

jotak reviewed May 14, 2025

View reviewed changes

openshift-ci bot assigned jotak May 14, 2025

openshift-ci bot added the lgtm label May 14, 2025

jotak removed the needs-review Tells that the PR needs a review label May 14, 2025

openshift-merge-robot added the needs-rebase label Jun 5, 2025

always create console plugin

43c5e6c

jpinsonneau force-pushed the 1942 branch from 1080b69 to 43c5e6c Compare July 15, 2025 08:40

openshift-ci bot removed the lgtm label Jul 15, 2025

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Jul 15, 2025

openshift-merge-robot removed the needs-rebase label Jul 15, 2025

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Jul 15, 2025

	func (r CPReconciler) reconcileStatic(ctx context.Context, desired flowslatest.FlowCollector) error {
	l := log.FromContext(ctx).WithName("console-plugin")
	ctx = log.IntoContext(ctx, l)

	// Retrieve current owned objects
	err := r.Managed.FetchAll(ctx)
	if err != nil {
	return err
	}

	if r.ClusterInfo.HasConsolePlugin() {
	if err = r.checkAutoPatch(ctx, desired, constants.StaticPluginName); err != nil {
	return err
	}
	}

	if r.ClusterInfo.HasConsolePlugin() {
	// Create object builder
	builder := newBuilder(r.Instance, &desired.Spec, constants.StaticPluginName)

	if err = r.reconcilePlugin(ctx, &builder, &desired.Spec, constants.StaticPluginName, "NetObserv static plugin"); err != nil {
	return err
	}

	if err = r.reconcileDeployment(ctx, &builder, &desired.Spec, constants.StaticPluginName, ""); err != nil {
	return err
	}

	if err = r.reconcileServices(ctx, &builder, constants.StaticPluginName); err != nil {
	return err
	}
	} else {
	// delete any existing owned object
	r.Managed.TryDeleteAll(ctx)
	}

	return nil
	}

NETOBSERV-2225 - Deploy static plugin at operator startup #1345

Are you sure you want to change the base?

NETOBSERV-2225 - Deploy static plugin at operator startup #1345

Conversation

jpinsonneau commented Apr 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Dependencies

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented May 6, 2025

Uh oh!

jpinsonneau commented May 6, 2025

Uh oh!

memodi commented May 13, 2025 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jotak May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jotak left a comment

Choose a reason for hiding this comment

Uh oh!

jpinsonneau commented May 14, 2025

Uh oh!

jotak commented May 14, 2025

Uh oh!

memodi commented Jul 14, 2025

Uh oh!

openshift-ci bot commented Jul 15, 2025

Uh oh!

openshift-ci bot commented Jul 15, 2025

Uh oh!

jpinsonneau commented Jul 15, 2025

Uh oh!

memodi commented Jul 15, 2025

Uh oh!

github-actions bot commented Jul 15, 2025

Uh oh!

memodi commented Jul 15, 2025

Uh oh!

openshift-ci bot commented Jul 15, 2025

Uh oh!

Uh oh!

jpinsonneau commented Apr 2, 2025 •

edited

Loading

memodi commented May 13, 2025 •

edited by openshift-ci bot

Loading

jotak May 14, 2025 •

edited

Loading