Repository navigation
Conversation
No envtest could exercise the Grove pathway, so every Grove behaviour in the operator was verifiable only by hand on a real cluster. Two things were missing. The Grove types had no envtest fixtures: internal/controller/testing/grove.io carried only clustertopologybindings.yaml, so the API server rejected the PodCliqueSet the workloads reconciler creates. Add permissive stubs for podcliquesets, podcliques, and podcliquescalinggroups, matching the existing stub's style -- x-kubernetes-preserve-unknown-fields keeps the schema Grove's concern, so these carry no version coupling to Grove releases. The gate was also unreachable in practice. The defaulting webhook resolves a DGD's workload provider at Create from the gates its own manager was started with, and the controller package's shared Env starts from features.Defaults(), where Grove is off. A DGD admitted there is stamped "workload-provider: component" regardless of the gates the test's own controller carries. Options already supports a RuntimeConfig override, so the fix is an isolated Env for Grove tests via RunT rather than a change to operatorenv itself; flipping the shared Env would silently move every other test in the package onto Grove. The first test to use it covers the DGD controller's owned-Service watch: it deletes a Service the DGD owns and asserts a new one appears. It passes on this branch and fails without the watch, which is the coverage asked for in review on the Phase 3 Service PR. Two details worth keeping: the reconcile needs Namespace.Restricted set, because cluster-wide mode requires an RBAC manager no envtest runs, and it needs a ScaleClient, because the Grove scaler drives PodClique replicas through the scale subresource and fails closed without one. Signed-off-by: Tzu-Ling <tzulingk@nvidia.com>
tzulingk
marked this pull request as ready for review
August 17, 2026 16:05
Contributor
Author
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview:
Makes the operator's Grove pathway reachable from envtest, so Grove behaviour can be covered in CI instead of only by hand on a cluster. Stacked on #13178 — review that one first; this branch is based on it so its first test has a fix to prove.
Tracked as DYN-3987. Opened as a draft: it should land after #13178.
Details:
Raised by @julienmancuso in review on #13178, asking for an envtest covering
Owns(&corev1.Service{}). Writing it turned out to be blocked by test infrastructure rather than by the test.Blocker 1 — no Grove CRD fixtures.
internal/controller/testing/grove.io/held onlyclustertopologybindings.yaml, so the API server rejected thePodCliqueSetthe workloads reconciler creates and the reconcile died before reaching anything else. Added permissive stubs forpodcliquesets,podcliques, andpodcliquescalinggroups, in the same style as the existing stub. They usex-kubernetes-preserve-unknown-fields, so the schema stays Grove's concern and these fixtures carry no version coupling to Grove releases.Blocker 2 — the gate was unreachable in practice. The defaulting webhook resolves a DGD's
nvidia.com/workload-providerat Create from the gates its own manager was started with. The controller package's shared Env starts fromfeatures.Defaults(), where Grove is off, so a DGD admitted there is stampedcomponentregardless of what gates the test gives its own controller — andensureWorkloadProviderthen honours that annotation verbatim:operatorenv.Optionsalready supports aRuntimeConfigoverride (env.go:182-184), so no change tooperatorenvwas needed. Grove tests get their own isolated Env viaEnv.RunT. Flipping the shared Env instead would silently move every other test in the package onto Grove.Two further details this turned up, both recorded as comments in the test:
Namespace.Restricted, because cluster-wide mode requires an RBAC manager no envtest runs;ScaleClient, because the Grove scaler drives PodClique replicas through the scale subresource and fails closed withscale client is nil.Known limitation: envtest runs no Grove controller and no Grove webhooks, so a PodCliqueSet stays at
observedGeneration: 0and Grove's own admission rules (e.g. theminAvailable: 0rejection behind grove#676) do not reproduce. This covers our rendering and watch wiring, not Grove's acceptance. Cluster tests are still required for that.Where should the reviewer start?
deploy/operator/internal/controller/dgd_grove_envtest_test.go— thegroveEnvdeclaration and its comment explain why a separate Env is required rather than reusingsharedEnv; the test itself is the first consumer.deploy/operator/internal/controller/testing/grove.io/*.yaml— three stubs; confirm the permissive shape is what we want long term.Validation:
TestGroveDGDRecreatesAnOwnedServiceAfterDeletionpasses on this branch (8.1s) and fails onmain, which lacks feat(operator): emit a headless leader Service for elastic EP (Phase 3) #13178's watch:the deleted Service was never recreated, so the owned-Service watch is not enqueuing its DGD.make envtestgreen:internal/controllerandinternal/webhook/validation.make lintclean,go build ./...,gofmtclean.