Skip to content

Tests: a stateful fake GitHub and multi-run release scenarios - #116

Open
lukasIO wants to merge 2 commits into
playground/0.31.0-pre.3from
claude/verwalter-scenario-harness
Open

lukasIO wants to merge 2 commits into
playground/0.31.0-pre.3from
claude/verwalter-scenario-harness

Conversation

@lukasIO

@lukasIO lukasIO commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator

A scenario over several runs of the tool, such as "a release fails, the branch moves, a recovery run starts", is long to write today. Each test in tests/e2e.rs scripts the answers of gh by hand with a closure. So these scenarios were run by hand on verwalter-playground, where each one needs merges and several minutes per run.

This PR is based on the test branch playground/0.31.0-pre.3, because the scenarios need #105, #109, #110 and #111. It can go to main after those merge.

Change

tests/e2e.rs only. Nothing under src/ changes, and no existing test changes.

  • A stateful fake GitHub, FakeGitHub. It keeps a release per tag with its draft flag and its asset names. It answers the gh calls of release, publish and status: view, create, upload, edit and delete of a release, and an empty PR list. A call that it does not know fails, and the fake panics at the end of the test if one occurred, because status ignores some gh errors.
  • scenario_a_release_that_lacks_an_asset_is_repaired_from_its_tag, four steps on one repository:
    • A release run builds one of two required assets. The release stays a draft, the publish command does not run, and status names the missing file.
    • The branch moves. A run with both assets is refused and names ref=<tag>. Nothing is uploaded.
    • A run without assets: publish fails with the changed-source reason. The command does not run.
    • The repair from the tag: both assets are uploaded, the command runs once, the release is public, and status is complete.
  • scenario_a_version_edited_by_hand_is_not_an_incomplete_release.

Verified

  • cargo fmt --all -- --check, cargo clippy --all-targets -- -D warnings and cargo test pass (85 unit, 2 main, 38 end-to-end).
  • Each step matches the run of the same scenario on verwalter-playground with builds 0.31.0-pre.1 and 0.31.0-pre.2.

Not done yet

  • The fake has no registry over several runs. status and release call the registry request directly, and only publish takes a replacement. A scenario with "the registry has the version now" needs that parameter in both.
  • The fake cannot inject a failure and has no PR state. The closure client stays the tool for such tests.
  • No existing test is converted. Candidates: release_with_assets_then_abandon, release_publishes_a_draft_only_with_its_required_assets, publish_leaves_a_draft_without_a_required_asset.

🤖 Generated with Claude Code

lukasIO and others added 2 commits November 18, 2026 14:30
FakeGitHub keeps one release per tag (draft, asset names) behind the
Client trait, logs the write calls, and fails the test on a gh call it
has no answer for. Two scenarios use it: a release that lacks a
required asset and is repaired from its tag over four runs, and a
version edited by hand that status does not report as incomplete.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…asset

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@lukasIO lukasIO self-assigned this Nov 18, 2026
@lukasIO
lukasIO marked this pull request as ready for review November 19, 2026 10:35
@lukasIO
lukasIO requested a review from ladvoc as a code owner November 19, 2026 10:35

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant