Skip to content

feat(events): include actor and request metadata in webhook payloads#3959

Merged
rchincha merged 8 commits into
project-zot:mainfrom
cainydev:feat/event-actor-request-metadata
Jun 22, 2026
Merged

feat(events): include actor and request metadata in webhook payloads#3959
rchincha merged 8 commits into
project-zot:mainfrom
cainydev:feat/event-actor-request-metadata

Conversation

@cainydev

Copy link
Copy Markdown
Contributor

Summary

Closes #3859

  • Adds actor (username) and request (addr, method, useragent) fields to CloudEvents payloads for ImageUpdated, ImageDeleted, ImageLintFailed, and RepositoryCreated events
  • Introduces EventContext, ActorInfo, and RequestInfo types in pkg/extensions/events/common.go (no build tag — always compiled so the storage layer can reference them unconditionally)
  • The API layer (UpdateManifest, DeleteManifest) builds an EventContext from the live HTTP request and attaches it to the context via events.WithEventContext before passing it down to storage
  • The storage layer (imagestore) extracts the context at event-fire time via events.EventContextFromContext; nil context (internal/GC paths) cleanly omits both fields from the payload
  • Fields are omitted entirely when not present (unauthenticated requests, internal triggers), matching the issue's requested JSON shape

Example payload:

{
  "name": "space/my-image",
  "reference": "latest",
  "digest": "sha256:abc...",
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "actor": { "name": "john" },
  "request": { "addr": "192.168.0.1:54321", "method": "PUT", "useragent": "docker/24.0.5" }
}

@cainydev
cainydev force-pushed the feat/event-actor-request-metadata branch 4 times, most recently from fd33f76 to 45af4fd Compare April 10, 2026 20:31
@cainydev
cainydev marked this pull request as draft April 12, 2026 16:46
@cainydev cainydev changed the title feat(events): include actor and request metadata in webhook event payloads feat(events): include actor and request metadata in webhook payloads Apr 12, 2026
@cainydev
cainydev force-pushed the feat/event-actor-request-metadata branch from 8546997 to 15a705f Compare April 12, 2026 17:10
@rchincha
rchincha requested a review from Copilot April 12, 2026 23:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the events extension’s CloudEvents payloads to optionally include actor (authenticated username) and HTTP request metadata by propagating an EventContext from the API layer down into the storage layer where events are emitted.

Changes:

  • Introduces EventContext (ActorInfo, RequestInfo) plus context helpers in pkg/extensions/events, and teaches the event builder to inject actor/request into event data.
  • Plumbs context.Context through ImageStore APIs (InitRepo, PutImageManifest, DeleteImageManifest) and updates call sites so storage-emitted events can access the context at fire time.
  • Adds/updates tests and mocks across storage, extensions, and API to cover the new event payload fields.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
pkg/test/mocks/image_store_mock.go Updates mock ImageStore method signatures to accept context.Context.
pkg/test/image-utils/write.go Passes context.Background() into updated ImageStore APIs in test helpers.
pkg/test/image-utils/write_test.go Updates tests to match new mock function signatures with context.Context.
pkg/storage/types/types.go Adds context.Context to ImageStore interface methods used by API/storage.
pkg/storage/storage_test.go Updates storage tests and event-capture interface for new event context params.
pkg/storage/scrub_test.go Updates scrub tests to call InitRepo/PutImageManifest with context.
pkg/storage/s3/s3_test.go Updates S3 storage tests for context-bearing ImageStore APIs.
pkg/storage/local/local_test.go Updates local storage tests/fuzzers for context-bearing ImageStore APIs.
pkg/storage/local/local_elevated_test.go Updates elevated local test to call InitRepo with context.
pkg/storage/imagestore/imagestore.go Plumbs context into repo init / manifest put/delete and attaches EventContext to emitted events.
pkg/storage/gcs/gcs_test.go Updates GCS storage tests for new PutImageManifest/InitRepo/DeleteImageManifest signatures.
pkg/storage/gc/gc_test.go Updates GC tests to call PutImageManifest with context.
pkg/storage/gc/gc_internal_test.go Updates GC internal tests to call PutImageManifest with context.
pkg/storage/common/common_test.go Updates common storage tests to call PutImageManifest with context.
pkg/meta/parse_test.go Updates parse tests to call DeleteImageManifest with context.
pkg/meta/hooks.go Updates rollback/restore paths to call ImageStore APIs with context.
pkg/meta/hooks_test.go Updates test ImageStore implementation to match new Delete signature.
pkg/meta/hooks_internal_test.go Updates mock function types for new context-bearing ImageStore methods.
pkg/extensions/sync/sync_test.go Updates sync tests to call DeleteImageManifest with context.
pkg/extensions/sync/sync_internal_test.go Updates sync internal tests to call PutImageManifest with context.
pkg/extensions/sync/destination.go Uses context.Background() when syncing manifests into destination store.
pkg/extensions/search/search_test.go Updates search tests’ mock store functions for new context-bearing signatures.
pkg/extensions/events/events.go Extends recorder methods to accept *EventContext and inject it into payloads.
pkg/extensions/events/events_test.go Adds coverage asserting actor/request presence/omission, incl. HTTP sink.
pkg/extensions/events/common.go Adds EventContext types and context helpers; updates Recorder interface signatures.
pkg/extensions/events/builder.go Adds WithEventContext to conditionally include actor/request in event data.
pkg/api/routes.go Builds EventContext from HTTP request and attaches it to context passed into storage.
pkg/api/routes_test.go Updates route tests’ mocked store functions to accept context.Context.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/storage/imagestore/imagestore.go Outdated
Comment thread pkg/storage/imagestore/imagestore.go Outdated
Comment thread pkg/storage/gcs/gcs_test.go
Comment thread pkg/storage/gcs/gcs_test.go Outdated
@rchincha rchincha closed this Apr 13, 2026
@rchincha rchincha reopened this Apr 13, 2026
@rchincha

Copy link
Copy Markdown
Contributor

#3962

Take a look at this as well.

@cainydev

Copy link
Copy Markdown
Contributor Author

yes i saw it i will work on this PR a bit. I had it all figured out and then broke it on the rebase. copilot had a similar solution as this one.

@cainydev
cainydev force-pushed the feat/event-actor-request-metadata branch 3 times, most recently from b9a9589 to b655d8e Compare April 17, 2026 18:40
@codecov

codecov Bot commented Apr 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.33962% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.67%. Comparing base (ec0c724) to head (318998e).

Files with missing lines Patch % Lines
pkg/storage/imagestore/imagestore.go 90.32% 0 Missing and 3 partials ⚠️
pkg/extensions/events/common.go 71.42% 1 Missing and 1 partial ⚠️
pkg/extensions/search/cve/trivy/scanner.go 80.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3959      +/-   ##
==========================================
+ Coverage   91.64%   91.67%   +0.03%     
==========================================
  Files         200      200              
  Lines       29663    29704      +41     
==========================================
+ Hits        27184    27232      +48     
+ Misses       1596     1591       -5     
+ Partials      883      881       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cainydev
cainydev force-pushed the feat/event-actor-request-metadata branch from b655d8e to 1e81b7d Compare April 25, 2026 15:30
@rchincha

rchincha commented May 8, 2026

Copy link
Copy Markdown
Contributor

@cainydev are you planning on working on this?

@rchincha rchincha added this to the v2.1.17 milestone May 8, 2026
@cainydev

cainydev commented May 8, 2026

Copy link
Copy Markdown
Contributor Author

@cainydev are you planning on working on this?

yes - i just couldn't figure out why the ci failed tbh. maybe flaky?

@rchincha

rchincha commented May 8, 2026

Copy link
Copy Markdown
Contributor

The fix should target the failing blackbox test behind this workflow step:

Job definition: .github/workflows/ecosystem-tools.yaml
Failing step: make run-blackbox-ci
The failure itself is in the regclient image pull path, not in the workflow YAML. The relevant log lines are:

PullImage from image service failed
rpc error: code = Canceled desc = copying config: context canceled
level=fatal msg="pulling image: rpc error: code = Canceled desc = copying config: context canceled"
This happened immediately after the test:

ok 20 [release] push OCI artifact with regclient
So the likely broken case is the next regclient-related test that pulls an image through CRI-O/containerd after push.

Solution
Make the regclient blackbox test wait until the pushed manifest and its config blob are fully readable before calling the runtime pull.

The logs show many blob uploads completing with PUT, then the runtime pull fails while copying config. That points to a race: the test proceeds as soon as push returns, but CRI pulls before the config blob is consistently available to the image service.

Recommended code change
In the failing regclient test, add a readiness loop after push and before the runtime pull:

bash
manifest_digest="$(regctl manifest head localhost:${PORT}/test-regclient --format '{{.Digest}}')"

wait for manifest to be readable

for i in $(seq 1 30); do
if regctl manifest get "localhost:${PORT}/test-regclient@${manifest_digest}" >/dev/null 2>&1; then
break
fi
sleep 1
done

resolve and wait for config blob

config_digest="$(regctl manifest get "localhost:${PORT}/test-regclient@${manifest_digest}" --format '{{ .Config.Digest }}')"

for i in $(seq 1 30); do
if curl -fsI "http://localhost:${PORT}/v2/test-regclient/blobs/${config_digest}" >/dev/null 2>&1; then
break
fi
sleep 1
done

sudo crictl pull "localhost:${PORT}/test-regclient@${manifest_digest}"
If the test uses a tag instead of digest, prefer switching the runtime pull to the digest form after push. That removes any tag propagation timing issue.

Why this should fix it
The error is specifically:

copying config: context canceled
That usually means the runtime started pulling, requested the image config blob, and the request got canceled because the content was not yet stably available. Since the push had just completed, the most likely cause is eventual visibility/race in the local registry storage path rather than a bad image.

Additional hardening suggestion
If the test currently does something like:

bash
regctl image copy ...
crictl pull localhost:${PORT}/test-regclient
change it to:

bash
digest="$(regctl image digest localhost:${PORT}/test-regclient:latest)"
crictl pull "localhost:${PORT}/test-regclient@${digest}"
and keep the wait loop above.

Summary
The practical fix is:

Locate the failing regclient pull test in the blackbox suite run by make run-blackbox-ci.
After pushing the image/artifact, poll until:
the manifest is readable, and
the config blob returns success on HEAD.
Pull by digest instead of tag.
That directly addresses the race shown by the logs and is the most likely way to make this job stable.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

pkg/storage/imagestore/imagestore.go:945

  • Calling InitRepo(context.Background(), repo) here means that if a repository is first created via blob upload (a common push flow), initRepo() will emit RepositoryCreated with a nil EventContext, even when the trigger was an authenticated HTTP request. To preserve actor/request metadata, consider threading a request context through blob-upload APIs (or ensure higher layers call InitRepo(ctx, repo) before invoking blob upload methods).
// NewBlobUpload returns the unique ID for an upload in progress.
func (is *ImageStore) NewBlobUpload(repo string) (string, error) {
	if err := is.InitRepo(context.Background(), repo); err != nil {
		is.log.Error().Err(err).Msg("failed to initialize repo")

		return "", err
	}

Comment thread pkg/api/routes.go
@rchincha rchincha removed this from the v2.1.17 milestone May 12, 2026

@vrajashkr vrajashkr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cainydev thanks for the PR!

The changes look good to me overall. Left a couple of comments

I think there is a file where the context hasn't been added in yet:

Error: pkg/extensions/monitoring/monitoring_test.go:539:87: not enough arguments in call to imgStore.FullBlobUpload

Comment thread pkg/extensions/sync/destination.go
Comment thread test/blackbox/helpers_upgrade.bash Outdated
Comment thread pkg/extensions/events/common.go
Comment thread pkg/storage/imagestore/imagestore.go Outdated
Comment thread pkg/storage/imagestore/imagestore.go
Comment thread pkg/extensions/sync/destination.go
@cainydev
cainydev force-pushed the feat/event-actor-request-metadata branch from a9a7da2 to fc4ce28 Compare June 9, 2026 15:29
@cainydev

cainydev commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto latest main and pushed updates addressing the review feedback and the CI failure.

CI failure (lint + extensions tests). This was a build break, not flakiness. After main merged the trivy SBOM artifact feature and a new GC-metrics test, both call ImageStore methods whose signatures this PR changes to take context.

@andaaron

  • lint events: Good point. lint is a property of the manifest, not the tags. Reverted to a single ImageLintFailed per manifest instead of one per changed tag, and updated the test accordingly.
  • event-emission refactor: changedTags is now set to the path reference in the non-extra-tags path, so ImageUpdated emission collapses to a single loop over changedTags instead of the if/else fallback.
  • actor/request field shape: No third-party contract dictates these. The shape is our own, modeled on the events the CNCF distribution registry emits. Its not a 1:1 but close.

@vrajashkr

  • "system" actor for sync: Agree it'd be useful for tracking automated operations. Keeping it out of scope here as you suggested. can be a follow-up.

@cainydev
cainydev requested a review from Copilot June 9, 2026 15:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated 6 comments.

Comments suppressed due to low confidence (2)

pkg/meta/hooks.go:1

  • rollbackDigestManifestTags receives a ctx context.Context but uses context.Background() for store operations. This drops cancellation/deadline and also loses any request/event metadata that was attached to the incoming context. Use the provided ctx for DeleteImageManifest and PutImageManifest here so rollbacks behave consistently with the initiating request.
package meta

test/blackbox/helpers_zot.bash:1

  • This retry loop depends on external seq, which may not exist in minimal CI environments (and adds overhead). Prefer a native bash arithmetic loop (e.g., for ((i=1; i<=attempts; i++))) to avoid the dependency and improve portability. Consider also treating attempts as an integer (local -i attempts=...) to avoid surprising behavior when given non-numeric input.

Comment thread pkg/storage/imagestore/imagestore.go
Comment thread pkg/storage/imagestore/imagestore.go
Comment thread pkg/storage/imagestore/imagestore.go
Comment thread pkg/storage/imagestore/imagestore.go
Comment thread pkg/meta/hooks.go Outdated
Comment thread pkg/extensions/events/common.go
Comment thread pkg/extensions/events/common.go
Comment thread pkg/api/routes.go
@andaaron

andaaron commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Also please rebase.

@rchincha rchincha modified the milestones: v2.1.18, v2.1.19 Jun 19, 2026
@cainydev
cainydev force-pushed the feat/event-actor-request-metadata branch from fc4ce28 to 5ce21f7 Compare June 21, 2026 10:30
@andaaron
andaaron requested a review from Copilot June 21, 2026 11:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated 2 comments.

Comment thread pkg/api/routes.go
Comment thread test/blackbox/helpers_pushpull.bash Outdated
andaaron
andaaron previously approved these changes Jun 21, 2026
cainydev added 8 commits June 21, 2026 16:59
Signed-off-by: cainydev <wajo432@gmail.com>
Add ctx context.Context to NewBlobUpload, FullBlobUpload, PutBlobChunk,
PutBlobChunkStreamed and CheckBlob in the storage ImageStore interface.
HTTP handlers now thread events.WithEventContext through these calls so
RepositoryCreated events emitted on first push or cross-repo mount carry
actor and request metadata instead of context.Background().

Signed-off-by: cainydev <wajo432@gmail.com>
On busy CI runners, CRI-O occasionally cancels its own context during
the post-download "copying config" step even though zot has served
every blob with 200 OK. The pull-by-name test in pushpull, fips140 and
upgrade fails as a result. A fresh pull replays from cri-o's local
cache and almost always succeeds, so wrap crictl pull in a bounded
retry helper.

Signed-off-by: cainydev <wajo432@gmail.com>
Signed-off-by: cainydev <wajo432@gmail.com>
Upstream main added a FullBlobUpload call site in the GC metrics test
after this branch changed the method signature to accept context.Context.
Pass context.Background() so the package compiles after rebase.

Signed-off-by: cainydev <wajo432@gmail.com>
Lint is a property of the manifest, not the tag(s) it is applied under,
so emit a single ImageLintFailed event per manifest rather than one per
changed tag. Set changedTags to the path reference in the non-extra-tags
path so ImageUpdated emission collapses to a single loop over changedTags
instead of an if/else fallback.

Addresses review feedback from @andaaron.

Signed-off-by: cainydev <wajo432@gmail.com>
The trivy SBOM artifact generation (project-zot#4088) calls CheckBlob, FullBlobUpload
and PutImageManifest, whose signatures now take context.Context. Pass the
context already in scope through storeSBOMAsOCIArtifact so the search
extension compiles.

Signed-off-by: cainydev <wajo432@gmail.com>
Add a nil-context guard to EventContextFromContext so it returns nil
instead of panicking on a nil ctx, and use the request-scoped ctx in
rollbackDigestManifestTags for DeleteImageManifest/PutImageManifest so
rollbacks honor cancellation and carry event metadata.

Signed-off-by: cainydev <wajo432@gmail.com>
@cainydev
cainydev force-pushed the feat/event-actor-request-metadata branch from 5ce21f7 to 318998e Compare June 21, 2026 15:00
@rchincha rchincha modified the milestones: v2.1.19, v2.1.18 Jun 22, 2026
@rchincha
rchincha merged commit 49040c6 into project-zot:main Jun 22, 2026
45 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feat]: Include actor and request metadata in Webhook Events

5 participants