syncer stuck on "the server could not find the requested resource"

[stevekuznetsov]

Reproduce:

git clone [email protected]:kcp-dev/controller-runtime-example.git
cd controller-runtime-example
git fetch origin refs/pull/19/head
git checkout FETCH_HEAD
make test-e2e

This gets stuck waiting on a Deployment to sync over to the pcluster, but I'm not doing anything special here ... after repro, logs are in .test/{kcp,syncer}.log. Attaching logs from my run.

Notably, syncer logs loop forever on:

I0812 15:02:43.085797       1 apiimporter.go:170] Importing APIs from location controller-runtime in logical cluster root:users:zu:yc:kcp-admin:controller-runtime-example (resources=[configmaps deployments.apps secrets serviceaccounts services])
I0812 15:02:45.620474       1 discovery.go:164] ignoring a resource since it is part of the core KCP resources: secrets (/v1, Kind=Secret)
I0812 15:02:45.620487       1 discovery.go:164] ignoring a resource since it is part of the core KCP resources: configmaps (/v1, Kind=ConfigMap)
I0812 15:02:45.620490       1 discovery.go:182] processing discovery for resource services (services.core)
I0812 15:02:45.621542       1 discovery.go:164] ignoring a resource since it is part of the core KCP resources: serviceaccounts (/v1, Kind=ServiceAccount)
I0812 15:02:45.621551       1 discovery.go:182] processing discovery for resource deployments (deployments.apps)
I0812 15:02:45.623041       1 apiimporter.go:204] Updating APIResourceImport root:users:zu:yc:kcp-admin:controller-runtime-example|services.controller-runtime.v1.core for SyncTarget controller-runtime
I0812 15:02:45.630944       1 apiimporter.go:204] Updating APIResourceImport root:users:zu:yc:kcp-admin:controller-runtime-example|deployments.controller-runtime.v1.apps for SyncTarget controller-runtime
E0812 15:03:14.565123       1 spec_process.go:269] Failed adding finalizer upstream on resource root:users:zu:yc:kcp-admin:controller-runtime-example|controller-runtime-example-system/controller-runtime-example-controller-manager: the server could not find the requested resource
E0812 15:03:14.565140       1 spec_controller.go:240] kcp-workload-syncer-spec failed to sync {{"apps" "v1" "deployments"} "controller-runtime-example-system/root:users:zu:yc:kcp-admin:controller-runtime-example|controller-runtime-example-controller-manager"}, err: the server could not find the requested resource
E0812 15:03:14.585550       1 status_process.go:174] Failed updating status of resource "/v1, Resource=services" root:users:zu:yc:kcp-admin:controller-runtime-example|controller-runtime-example-system/controller-runtime-example-controller-manager-metrics-service from pcluster namespace kcp-pk65pz51mpn3: the server could not find the requested resource
E0812 15:03:14.585566       1 status_controller.go:157] kcp-workload-syncer-status failed to sync {{"" "v1" "services"} "kcp-pk65pz51mpn3/controller-runtime-example-controller-manager-metrics-service"}, err: the server could not find the requested resource
I0812 15:03:45.657656       1 apiimporter.go:170] Importing APIs from location controller-runtime in logical cluster root:users:zu:yc:kcp-admin:controller-runtime-example (resources=[configmaps deployments.apps secrets serviceaccounts services])
I0812 15:03:48.191982       1 discovery.go:164] ignoring a resource since it is part of the core KCP resources: serviceaccounts (/v1, Kind=ServiceAccount)

kcp.log
syncer.log