fix: gracefully handle manifests missing from storage (prepare for sparse indexes)

[andaaron]

GC and scrub should not stop if a manifest or index is missing from storage. Other similar changes are also included.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[codecov]

Codecov Report

❌ Patch coverage is 96.94656% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.67%. Comparing base (2b6fba7) to head (f86d04c).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/meta/dynamodb/dynamodb.go	86.66%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3503      +/-   ##
==========================================
- Coverage   91.67%   91.67%   -0.01%     
==========================================
  Files         185      185              
  Lines       25755    25852      +97     
==========================================
+ Hits        23611    23699      +88     
- Misses       1385     1392       +7     
- Partials      759      761       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull Request Overview

This PR implements graceful handling of missing blobs throughout the zot registry system, allowing operations like garbage collection, integrity scrubbing, repository parsing, and syncing to continue processing when encountering missing blob files rather than failing immediately.

• Adds graceful missing blob handling in GC, scrub, parse, sync, and metaDB operations
• Updates tests to verify the new graceful handling behavior
• Preserves "affected" status in scrub operations when subjects/manifests are missing

Reviewed Changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
pkg/storage/storage_test.go	Updated test expectation: IsBlobReferenced returns nil error when multiarch image is missing
pkg/storage/scrub_test.go	Added comprehensive tests for scrub operations with missing manifests and permission errors
pkg/storage/scrub.go	Implemented graceful handling of missing blobs in scrub operations with status preservation logic
pkg/storage/local/local_test.go	Updated GC test to expect success when manifest blobs are missing
pkg/storage/gc/gc_internal_test.go	Added tests and updated expectations for missing blob handling in GC operations
pkg/storage/gc/gc.go	Added graceful missing blob handling in GC's removeIndexReferrers, identifyManifestsReferencedInIndex, and blob reference tracking
pkg/storage/common/common_test.go	Added tests for missing blob handling in PruneImageManifestsFromIndex, IsBlobReferencedInImageManifest, and GetBlobDescriptorFromIndex
pkg/storage/common/common.go	Implemented graceful missing blob handling in pruning, blob referencing, and descriptor retrieval operations
pkg/meta/redis/redis_test.go	Updated test to verify missing manifests are skipped gracefully; added corrupted data error test
pkg/meta/redis/redis.go	Skip missing manifests in getAllContainedMeta while propagating real errors
pkg/meta/parse_test.go	Added tests for missing manifest handling during repo parsing
pkg/meta/parse.go	Implemented graceful skip of missing manifests during repository parsing
pkg/meta/meta_test.go	Added test for GetFullImageMeta with nested index containing missing manifests
pkg/meta/dynamodb/dynamodb_test.go	Updated tests to verify missing manifests are skipped gracefully
pkg/meta/dynamodb/dynamodb.go	Refactored fetchImageMetaAttributesByDigest to skip missing items; added validation in FilterImageMeta; cleaned up indentation in createVersionTable
pkg/meta/boltdb/boltdb_test.go	Added comprehensive tests for missing and corrupted manifest handling
pkg/meta/boltdb/boltdb.go	Skip missing manifests in getAllContainedMeta while propagating real errors
pkg/extensions/sync/destination.go	Added graceful handling for missing manifest blobs during sync operations
pkg/extensions/search/search_test.go	Added test verifying search works correctly when manifests are missing from storage
pkg/extensions/search/cve/scan_test.go	Added check for empty Manifests slice when all manifests are missing

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Copilot reviewed 20 out of 20 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.