Overhaul workflow verification

[stephanos]

What was changed

(1) Added new flag --continue-on-error to run scenario.
(2) Extracted workflow completion check into reusable component.
(3) Moved state for completion check into executor.
(4) Consolidated Search Attribute for completion checking.
(5) Added WorkflowExecutionAlreadyStarted handling for completion checking.
(6) Consolidated post-scenario verification.

Why?

(1) It is very useful to move past any non-transient errors during a scenario run and keep going; reporting the errors at the end. This helps with runs where a single workflow fails to complete and would block further execution.

(2-6) Checking workflow completion is a crucial verification step; this change make it easier to re-use across scenarios.

Checklist

1. Closes

2. How was this tested:

3. Any docs updates needed?

[stephanos]

This was technically a good idea, but easily causes one-off issues. For example, when the all iterations were created but the run crashes and resumes, it would error since the start iteration cannot be larger than the target number of iterations. And now with the WorkflowExecutionAlreadyStarted handling, this is safe.

[stephanos]

Moved into ExecutorState.

[stephanos]

Moved into WorkflowCompletionChecker

[stephanos]

So it inherits the Resumable and Verifyable behavior.

[semgrep-managed-scans]

Semgrep found 1 missing-explicit-permissions finding:

• .github/workflows/docker-images.yml
  • L45-111 - Triage

No explicit GITHUB_TOKEN permissions found at the workflow or job level. Add a permissions: block at the workflow root (applies to all jobs) or per job with least privilege (e.g., contents: read and only specific writes like pull-requests: write if needed).