F-157: Widget pack install — OCI pull + cosign verification (#10)

* feat(pkl): full §15.2 PackManifest; relocate WidgetInstance into widgets.pkl

- Rewrites widgets.pkl with the complete §15.2 PackManifest schema
  (name, version, protocol, sdkVersion, bundle, bundleHash, classes,
  description?, homepage?, license?) with appropriate constraints.
- Relocates Position, Grid, and abstract WidgetInstance from dashboards.pkl
  into widgets.pkl so pack authors can extend WidgetInstance without
  importing dashboards.pkl.
- Updates dashboards.pkl to import switchyard:widgets and reference
  widgets.WidgetInstance, widgets.Grid, widgets.Position throughout.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(design): F-157 §3.1 — drop ContainerInstance from widgets.pkl re-export

Architecture table contradicted §4.1's explicit decision to keep
ContainerWidget in dashboards.pkl. Pack authors don't author containers
in v1.0 (only the builtin GroupCard is a container).

* feat(config): add widgetPackPolicy to Pkl + proto + decoder

- Adds `import "switchyard:widgets" as widgets` and
  `widgetPackPolicy: widgets.PackPolicy = new {}` to config.pkl
  (the top-level config module, where all aggregated fields live).
- Adds `WidgetPackPolicy` proto message with `allowed_signers` and
  `allow_unsigned` fields; adds `widget_pack_policy = 18` to
  `ConfigSnapshot`.
- Adds `widgetPackPolicyJSON` struct and decodes the field into
  `snap.WidgetPackPolicy` in `parseConfigJSON`.
- Regenerates `gen/switchyard/config/v1/snapshot.pb.go` via `task proto`.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(config): move widgetPackPolicyJSON to evaluator_decode.go

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(proto): add WidgetPackService

Define WidgetPackService with Install/List/Uninstall/Watch RPCs and all
associated message types (InstalledPack, UninstalledPack, WidgetPackEvent).
Reuses the existing SignatureStatus enum from dashboard.proto (same package).
Regenerate Go + Connect bindings.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(design): F-157 §5 — note SignatureStatus reuse + EXPIRED mapping

Task 3 implementation reused the existing SignatureStatus enum from
dashboard.proto rather than declaring a duplicate. The existing enum
has SIGNATURE_EXPIRED which the original spec didn't map to a Connect
error code; add that to §5.2.

* feat(widgetpack): on-disk Store with persistence + Subscribe

Replaces the in-memory Store stub with a fully-persistent implementation:
.registry.json written atomically on every Add/Remove, stale-entry pruning
on Load, multi-version keying (name@version), and non-blocking Subscribe
fan-out for install/uninstall WatchEvents.

Also updates install_test.go call sites to supply the required root arg.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(widgetpack): Store correctness — Remove rollback, event aliasing, stale log

- Remove now restores the in-memory entry if persistLocked fails (mirror of Add)
- Add no longer leaks the store's *InstalledPack into WatchEvent — a subscriber
  mutating the event payload could silently corrupt live state
- Load logs a warning when pruning stale registry entries (spec §6.4)

* feat(widgetpack): real cosign keyless verification via sigstore-go

Replaces the placeholder string-switch TrustPolicy.Verify with a real
sigstore-go-backed verifier. The new surface (Verifier, TrustPolicy,
VerificationResult) is what later F-157 tasks (install flow, daemon
wiring, integration test) will consume.

- TrustPolicy now stores allowed-signer globs and an AllowUnsigned flag
  behind an RWMutex; Set replaces both atomically. Glob matching uses
  path.Match against the cert SAN URI.
- NewVerifier accepts an injectable root.TrustedMaterial. Tests pass a
  ca.VirtualSigstore directly; production will use NewProductionVerifier
  (currently stubbed pending TUF wiring).
- Verify decodes a sigstore JSON bundle and runs sigstore-go's
  *verify.Verifier with WithTransparencyLog(1) + WithObserverTimestamps(1).
  Identity matching is done outside sigstore-go so we can apply our own
  glob policy against the SAN URI.

testutil_test.go provides newTestTrustRoot + signBlobEntity, shared with
the upcoming Task 15 OCI integration test. Unit tests feed entities to
the package-internal verifyEntity hook to exercise the full sigstore-go
pipeline (cert chain + Rekor + RFC3161 timestamp) without serialising to
JSON; the JSON path is covered by a garbage-bundle reject test plus the
forthcoming Task 15 integration test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(widgetpack): trust policy polish — pattern validation, doc, nil checks

- TrustPolicy.Set returns an error if any signer pattern is malformed
  (was: silently failing to match at verify time, looking like 'no signers')
- Verify godoc tightened: pol must be non-nil, nil pol rejects signed bundles
- NewVerifier returns an error on nil TrustedMaterial
- Comment on the unused ctx parameter clarifying it's reserved for TUF refresh

* feat(widgetpack): OCI artifact pull via oras-go

Adds Fetcher type that pulls widget pack OCI artifacts plus their cosign
signature artifacts (best-effort) into memory. Rejects multi-layer
artifacts and wrong media types. Tarball extraction and signature
verification are owned by Install (Task 9).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(widgetpack): OCI fetcher polish — retry, tidy, referrer doc

- Wire retry.DefaultClient into auth.Client so 429/Retry-After are honored
  (modern registries rate-limit unauthenticated pulls aggressively)
- go mod tidy: promote oras-go and image-spec to direct deps
- Document OCI 1.1 Referrers signature gap as a known limitation
- Add zero-layer manifest test case

* docs(design): F-157 §6.3 — note OCI 1.1 Referrers limitation (F-289)

* feat(config): expose SwitchyardSchemeReaderOption + add manifest property to widgets.pkl

Add `SwitchyardSchemeReaderOption()` to `internal/config` so external
packages can register the embedded switchyard: Pkl module reader without
importing unexported internals. Add a top-level `manifest: PackManifest?`
property to `widgets.pkl` so pack manifest.pkl files can amend the module
and be rendered to JSON by the Pkl evaluator.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(widgetpack): Pkl-evaluator-driven manifest validation

Add EvalManifest(ctx, path) which creates a fresh Pkl evaluator with the
switchyard: scheme reader, renders the manifest as JSON, and decodes into
a Manifest struct. Pkl constraints in PackManifest (protocol == "v1",
bundleHash startsWith "sha256:", name not empty, classes not empty) act as
the validation layer — constraint violations surface as evaluator errors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(widgetpack): sandbox EvalManifest for untrusted input

Manifest sources come from extracted-from-tarball Pkl files (Task 9
install flow). The Pkl evaluator must be sandboxed accordingly.

- Set RootDir to the manifest's directory (spec §6 step 4)
- Drop WithOsEnv so manifests can't read host environment variables
- Errcheck-clean ev.Close()
- Better error when 'manifest' property is null (vs. misleading
  "missing required field: name")
- Tighten test error handling and add coverage for null + optional
  fields

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(widgetpack): errcheck-clean rc.Close in readBlob

* feat(widgetpack): bundle HTTP handler with immutable cache

Serves /widgets/<pack>/<version>/<file> with immutable Cache-Control,
ETag from pack SHA256, correct Content-Type, 404 for unknown packs,
405 for non-GET/HEAD, 304 for If-None-Match, and two-layer path traversal
defence (path.Clean + post-Join prefix check).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(widgetpack): bundle handler polish — request context, test hygiene

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(widgetpack): full §15.4 install flow

Rewrites Installer.Install to chain pull → verify → stage → manifest
validate → hash → SDK check → collisions → atomic commit, with stable
FailureReason tokens and per-(name@version) install-mutex serialization.
Tarball extraction uses an Abs-prefix path-traversal defense.

The previous stub installer's tests are removed; Task 15's integration
test will exercise the real pipeline end-to-end.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(widgetpack): propagate Store.List error in checkCollisions

* feat(widgetpack): Uninstall with reference check

Add Installer.Uninstall (os.RemoveAll → store.Remove) with optional
DashboardLister guard that blocks removal when pack classes are in use.
Defaults to emptyDashboardLister (no-op) until F-156 lands.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(widgetpack): assert ErrPackNotFound identity in Uninstall test

* feat(widgetpack): WidgetPackService Connect handler

Implements the four-method Connect-RPC handler (Install, List, Uninstall,
Watch) together with proto-conversion helpers and full FailureReason→code
error mapping; includes two unit tests for the List and Uninstall paths.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(widgetpack): drop nil-installer Service contract

* feat(api): widget_pack procedure-catalog registrar (inert until F-184)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(daemon): wire WidgetPackService + bundle handler

Construct the F-157 widget pack subsystem (Store, Fetcher, TrustPolicy,
Installer, Service, BundleHandler) inside the daemon's Run flow, mount
the WidgetPackService Connect handler on the API listener, and pass the
bundle handler to listener.Deps.WidgetsHandler so /widgets/<pack>/...
serves real bundles.

The trust policy is initialised from the current Pkl ConfigSnapshot's
WidgetPackPolicy and hot-reloads via cfgManager.OnApplied; bad signer
patterns log a warning instead of crashing.

NewProductionVerifier is currently stubbed (the production TUF root is
not yet wired). The daemon tolerates a nil verifier: install.go's Step 2
treats it as "no verifier configured" and rejects signed packs with
ReasonSignatureInvalid while still permitting the policy.AllowUnsigned
path. This is the chosen v1 behaviour until the trust root lands.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(widgetpack): check err before resp.Body in serve_test

* feat(cli): wire switchyard widget {install,list,uninstall} to RPC

Replace no-op RunE stubs with real Connect-RPC client calls to
WidgetPackService; add --version/--force flags on uninstall and
styled output via existing helpers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(widgetpack): end-to-end integration against in-process registry

5 passing tests + 2 skipped (signed paths blocked on sigstore Bundle
inclusion-proof construction; signed verification is unit-tested via
verifyEntity in trust_test.go). Adds Fetcher.WithPlainHTTP option
(test-only) so the in-process registry can serve plain HTTP.

Acceptance: unsigned-rejected, unsigned-allowed (full happy path),
hash-mismatch, class-collision-with-builtin, already-exists.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(dashboard): wire widgetpack.Store into dashboard catalog

Adds Store.ClassesView() + PackView/PackClass snapshot types to widgetpack.
Updates dashboardBackend to accept *widgetpack.Store and rebuild the catalog
on each WidgetCatalog call, joining pack classes with builtins. Fixes F-157
acceptance criterion 5: catalog now reflects installed widget packs.
Adds TestStore_ClassesView and TestDashboardBackend_WidgetCatalog_ReflectsStore.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(widgetpack): drop unused dataDir + SetDashboardLister race

Removes Installer.dataDir (set but never read) and its NewInstaller parameter.
Drops SetDashboardLister (unsynchronized write); moves dl into the constructor
with nil defaulting to emptyDashboardLister{}, matching the rest of the
constructor pattern and eliminating the race condition. Updates all callers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(driverkit): go mod tidy — drop stale otel indirect dep

go.opentelemetry.io/otel is no longer a transitive requirement of the
driverkit module; go mod tidy removes it. Fixes driverkit-build-and-test
CI check.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Author: fdatoo

docs/design/specs/2026-05-04-f157-widget-pack-install-design.md

@@ -34,7 +34,7 @@ The full server-side install flow with admin authz and end-to-end CLI usability,
 
 | Path | New? | Purpose |
 |------|------|---------|
-| `internal/config/pkl/switchyard/widgets.pkl` | modified | Full §15.2 `PackManifest` + re-export of `WidgetInstance`/`ContainerInstance`; `widgetPackPolicy` instance |
+| `internal/config/pkl/switchyard/widgets.pkl` | modified | Full §15.2 `PackManifest` + re-export of `WidgetInstance` (and its helper classes); `widgetPackPolicy` instance |
 | `internal/config/pkl/switchyard/dashboards.pkl` | modified | Import `WidgetInstance` from `widgets.pkl` rather than declaring locally |
 | `internal/config/pkl/switchyard/policy.pkl` | modified | Add top-level `widgetPackPolicy: PackPolicy` |
 | `proto/switchyard/v1alpha1/widget_pack.proto` | new | `WidgetPackService { Install, List, Uninstall, Watch }` |
@@ -144,12 +144,10 @@ message InstalledPack {
   google.protobuf.Timestamp installed_at     = 11;
 }
 
-enum SignatureStatus {
-  SIGNATURE_STATUS_UNSPECIFIED = 0;
-  SIGNATURE_STATUS_VERIFIED    = 1;
-  SIGNATURE_STATUS_UNSIGNED    = 2;
-  SIGNATURE_STATUS_INVALID     = 3;
-}
+// SignatureStatus is reused from proto/switchyard/v1alpha1/dashboard.proto
+// (proto3 same-package enums must be unique). The existing enum's values
+// are SIGNATURE_UNKNOWN/VERIFIED/UNSIGNED/INVALID/EXPIRED — a strict superset.
+// SIGNATURE_EXPIRED maps to FAILED_PRECONDITION/signature_expired in §5.2.
 
 message WatchRequest {}
 message WatchEvent {
@@ -168,6 +166,7 @@ message UninstalledPack { string name = 1; string version = 2; }
 | Empty / malformed `ref` | `INVALID_ARGUMENT` | `bad_ref` |
 | Caller lacks `widget_pack.install` | `PERMISSION_DENIED` | (set by authz interceptor) |
 | Signature rejected by trust policy | `FAILED_PRECONDITION` | `signature_invalid` |
+| Signing certificate expired | `FAILED_PRECONDITION` | `signature_expired` |
 | `bundle.js` SHA256 ≠ `manifest.bundleHash` | `FAILED_PRECONDITION` | `hash_mismatch` |
 | `manifest.sdkVersion` major mismatch | `FAILED_PRECONDITION` | `sdk_incompatible` |
 | Class collision with builtin or installed pack | `FAILED_PRECONDITION` | `class_collision` |
@@ -244,6 +243,8 @@ Per-`(name@version)` mutex via `sync.Map`; concurrent installs of different pack
 - Anonymous by default; reads `~/.docker/config.json` for credentials so `docker login ghcr.io` flows through transparently.
 - Single-layer assumption checked explicitly; multi-layer artifacts rejected with `FAILED_PRECONDITION/bad_artifact`.
 
+**Known limitation — cosign signature lookup:** F-157 v1 reads cosign signatures only from the legacy tag-based layout (`<digest>.sig`). Cosign 2.x against OCI 1.1-capable registries (ghcr.io, AWS ECR, Docker Hub since 2024) defaults to attaching signatures as Referrers (manifest.subject), which this fetcher does not query. Modern-layout signed artifacts will appear unsigned to F-157. Tracked separately as **F-289** ("widget pack OCI 1.1 Referrers signature lookup").
+
 ### 6.4 Storage layout under `<DataDir>/widgets/`
 
 ```