[perfectra1n]

Fixes #477. It covers all three root causes from the issue, one commit each so they can be reviewed and bisected on their own.

1. Incomplete kopia checkpoints are never treated as snapshots (c318ec23)

kopia snapshot list --json emits incomplete checkpoint manifests even without --incomplete. That flag filters only the text output (see cli/command_snapshot_list.go in kopia 0.23.1). SnapshotListEntry had no incomplete field, so serde dropped the marker. A single stray checkpoint then failed SnapshotReplication's post-verify on every run. It could also be picked as "latest" by restore, verify, catalog discovery and succeeded-snapshot resolution.

• SnapshotListEntry / DirSummary now carry incomplete, read through incomplete_reason(). It checks the top-level marker or rootEntry.summ, and treats any non-empty reason as incomplete.
• KopiaClient::snapshot_list / snapshot_list_all are complete-only by default. snapshot_list_all_with_incomplete() is used only for manifest-ID presence checks.
• Replication skips source checkpoints, never counts them as expected or missing, and reports them in the new status.lastRun.incompleteSkipped. The mover log names their IDs and the Ready message mentions them.
• The fixture was captured from a real kopia 0.23.1 repository.

2. One kopia invocation per delete batch (6f825bd2)

A snapdel-* Job ran one kopia snapshot delete <id> per member, so the repository was opened and its index loaded once per member. It now does one snapshot list --all and one kopia snapshot delete <id>... --delete per 500 IDs.

• Partitioning first is mandatory. I confirmed against a real 0.23.1 repository that a multi-ID delete is all-or-nothing, because the write session only flushes on success. One absent ID aborts the whole call, and kopia still logs "Deleting snapshot …" for IDs it then never commits.
• batch_delete::plan_batch_delete is pure and keeps every legacy per-member rule: absent means already done, and the stale-ID self-heal still sits behind the start_time data-loss gate. Its matcher excludes the member's own recorded ID, which reproduces the post-delete view the legacy path re-listed. Without that exclusion, a mid-rewrite pin pair is ambiguous and the live manifest is orphaned.
• A retryable bulk failure fails the chunk and leaves the retry to the controller's backoff. It does not fan out one repository open per ID. A non-retryable failure retries each ID alone, and a failed listing fails the Job outright.
• classify(): the issue's error (failed to open repository … error loading indexes … unexpected EOF) was Unknown. It is now RepositoryUnavailable, and the more specific arms still win.

3. Per-repository cap on delete batch Jobs (dbd0a95f)

Delete batches were documented as single-flight per repository, but nothing enforced it. That is how #477 ended up with about 33 concurrent Jobs on one B2 repository.

• New field spec.concurrency.maxConcurrentDeleteJobs on Repository / ClusterRepository. Absent means 1. The minimum is 1, enforced by both the schema and the webhook, and there is no unlimited value.
• pool::DeleteAdmissionLedger does observe → decide → reserve under one lock (the same pattern as AdmissionLedger), per repository and optionally cluster-wide. It replaces the racy throttle_verdict. A reservation never parks its own Job, so a sibling reconcile of the same wave still dedups on the 409.
• Batch Jobs now have a 30-minute activeDeadlineSeconds. With a cap of 1, the inherited 48 h deadline would let one wedged Job stall a repository's deletions for two days.
• Batches are truncated to an 80 KiB serialized-member budget, under the 100 KiB work-spec env cap. MAX_BATCH_MEMBERS goes from 200 to 1000. With long source paths, the old 200 cap could already produce a Job that could not be built.
• Every "already single-flight" claim is corrected, the helm-docs README is regenerated, and ADR-0006 gets an amendment note.

Verification

• mise run fmt-check, clippy, test (89 test binaries), gen-check, complexity-check (29/29, no growth) and docs are all green.
• Every commit was built and tested on its own.
• I mutation-tested the planner and the ledger by breaking each key line on purpose, and every mutation made a test fail.
• The fake-kopia shim tests count kopia processes: a batch of N costs exactly one list and one delete.
• e2e on fresh kind clusters, mass_deletion plus snapshot_replication, 17/17 passing:
  • Scenario 6 now hard-asserts single-flight. The slow-mover fixture holds wave 1 live, and wave 2's Job appears only after wave 1 is done.
  • The new scenario 10 checks that maxConcurrentDeleteJobs: 2 really lets two batches run at once, and never more than two.
  • All existing batch and breaker scenarios still pass, including the global throttle and outage retry.

Reporter: happy to have you run the preview images against your repository. They include the bulk delete, which should drain a backlog like yours in seconds per batch.