Skip to content

fix(python-mutation): apply detect-state pattern with single-line install - #172

Merged
williaby merged 3 commits into
mainfrom
chore/python-mutation-detect-state
May 26, 2026
Merged

williaby merged 3 commits into
mainfrom
chore/python-mutation-detect-state

Conversation

@williaby

@williaby williaby commented May 26, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Apply the Wave 1C strengthened detect-state pattern (per PR #166 / PR #170) to python-mutation.yml. Wave 2B of the CI Repair Sprint Phase 2.

Changes

  • Added Detect repo state step (skip / poetry-not-supported / uv-locked / uv-no-lock).
  • Split single Install dependencies step into two if:-guarded single-line run: steps:
    • Install dependencies (uv with lockfile): literal --frozen, inline NOSONAR(S8541).
    • Install dependencies (uv without lockfile): no --frozen, inline NOSONAR(S8541,S8544).
  • Replaced the bare Skip notice step with Set skip outputs and notice: emits deterministic score=0, passed=true, skipped=true to $GITHUB_OUTPUT so downstream callers no longer see empty strings on the skip path (and exposes a new skipped workflow_call output).
  • Gated Run mutation testing, Analyze mutation results, Upload mutation reports, Post PR comment, and Mutation Testing Summary on state == 'uv-locked' || state == 'uv-no-lock'.
  • Replaced the bare # NOSONAR (no rule list) on the previous install line with explicit comma-separated rule lists per the strengthened patterns doc.
  • Fixed the nested uv run mutmut results --json subprocess inside the analyze step's Python heredoc to pass --frozen and $NO_BUILD_FLAG, restoring the lockfile/no-build guarantee on the 4th uv run invocation.

Trigger preserved (CI-053)

The on: block remains workflow_call: only. No pull_request: trigger added. Downstream callers must continue to use schedule + workflow_dispatch per manifest CI-053.

Pattern notes

  • Install steps use Pattern A (single-line run: + inline NOSONAR). Confirmed reliable per docs/sonarcloud-nosonar-patterns.md.
  • The Run mutation testing step retains run: | because mutmut invocations use multi-line continuations. Literal --frozen is on every uv run line, including the Python subprocess inside the Analyze mutation results step (S8544 won't fire structurally). Preceding-line # NOSONAR(S8541) is best-effort per the strengthened doc. If SonarCloud flags S8541 on these lines, the follow-up is restructuring to single-line per Option 3.

Test plan

  • actionlint passes (with -shellcheck= per feedback_actionlint_shellcheck_embed.md).
  • CI on .github passes.
  • SonarCloud quality gate: OK, 0 open vulns.
  • After merge, manual gh workflow run on ByronWilliamsCPA/maester-tests Mutation Testing workflow succeeds.

Review response (post-/pr-review)

After /pr-review (#172 comment trail), two Copilot findings and one CLAUDE.md compliance gap were addressed in commits b08e83f and e30cad7:

  1. Skip-path workflow_call outputs. mutation-score / passed no longer evaluate to empty strings when state=skip. Added a set-skip-outputs step plus a coalescing job-output expression. New skipped output exposed for sophisticated callers.
  2. Nested uv run in the Python heredoc. Subprocess now builds the command with --frozen and $NO_BUILD_FLAG from env, matching the surrounding install/run steps.
  3. CHANGELOG.md updated with a ### Fixed entry under [Unreleased] summarizing the detect-state pattern application and the two follow-up fixes.

Branch was also rebased onto origin/main (was 1 commit BEHIND #170) to clear the merge gate.

Related

Copilot AI review requested due to automatic review settings May 26, 2026 01:22
@coderabbitai

coderabbitai Bot commented May 26, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

@williaby, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 27 minutes and 47 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 68e72421-318b-4bd7-bd96-fd9bf96f187f

📥 Commits

Reviewing files that changed from the base of the PR and between 92dd50c and e30cad7.

📒 Files selected for processing (2)
  • .github/workflows/python-mutation.yml
  • CHANGELOG.md
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/python-mutation-detect-state

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the reusable python-mutation.yml workflow to follow the strengthened Wave 1C detect-state pattern, so it can safely handle repos with no pyproject.toml, Poetry-managed repos, and uv repos without a committed uv.lock.

Changes:

  • Add a Detect repo state step that classifies the repo as skip, poetry-not-supported, uv-locked, or uv-no-lock.
  • Split dependency installation into two single-line uv sync steps gated by detect-state to keep SonarCloud-friendly literal flag usage.
  • Gate mutation run, analysis, artifact upload, PR comment, and summary steps to only run on uv states.

Comment thread .github/workflows/python-mutation.yml Resolved
Comment on lines 227 to 231
- name: Analyze mutation results
id: analyze
if: steps.detect.outputs.state == 'uv-locked' || steps.detect.outputs.state == 'uv-no-lock'
run: |
echo "Analyzing mutation test results..."
@williaby

Copy link
Copy Markdown
Collaborator Author

PR Review

Status: ✅ All 24 CI checks pass · ✅ SonarCloud Quality Gate passed · Copilot completed · CodeRabbit rate-limited (no review) · ⚠️ Branch BEHIND main

No Critical findings. 3 Important items below; full report (incl. Suggested + Informational tiers) in the local review session.

Important (should fix)

  1. Skip-path leaves workflow_call outputs empty. python-mutation.yml L100-L122 — mutation-score and passed outputs are sourced from steps.analyze.outputs.*, but analyze is gated if: state == 'uv-locked' || state == 'uv-no-lock'. On the skip path the outputs evaluate to empty strings; downstream callers using if: needs.mutation.outputs.passed == 'true' silently fall through to false rather than receiving a deterministic sentinel. Fix: add a small step gated on state == 'skip' that writes score=0 / passed=true (or another agreed sentinel + a skipped=true flag) to $GITHUB_OUTPUT. (Source: Copilot, validated locally.)

  2. Nested uv run inside Python heredoc bypasses --frozen and $NO_BUILD_FLAG. python-mutation.yml L239-L260 — subprocess.run(['uv', 'run', 'mutmut', 'results', '--json'], ...) omits both --frozen and $NO_BUILD_FLAG, contradicting the PR body's stated invariant "Literal --frozen is on every uv run line." Pre-existing code, but this PR was a good opportunity to tighten it while consolidating install patterns. Fix: add the flags to the subprocess argv, or sidestep uv by parsing the already-emitted mutmut-results.txt. (Source: Copilot, validated locally.)

  3. CHANGELOG.md not updated for a fix: commit. Repo maintains CHANGELOG.md with an [Unreleased] section (Keep-a-Changelog format). OpenSSF baseline + global CLAUDE.md expects CHANGELOG entries for feat:/fix:/perf:/breaking commits. PR fix(python-sonarcloud): detect repo state #166 (Wave 1C exemplar) also did not update CHANGELOG, so if there is an established practice of batching workflow-template fixes, that should be documented in .claude/rules/. Otherwise add an ### Fixed entry under [Unreleased]. (Source: local agent — CLAUDE.md compliance.)

Bookkeeping

  • PR body Test plan: [ ] CI on .github passes. is unchecked but all 24 checks are green.
  • Branch is BEHIND main (mergeStateStatus = BEHIND). Rebase before merge.

🤖 Generated with Claude Code

…tall

Apply the Wave 1C strengthened detect-state pattern (see PR #166): split the
install step into two if-guarded single-line `run:` steps (uv-locked vs
uv-no-lock), each carrying Pattern A inline NOSONAR. Gate downstream steps
on detect.outputs.state so skip and poetry-not-supported paths exit cleanly.

The pre-existing single-line install (line 169) used a bare `# NOSONAR`
without a rule list; replace with explicit S8541 (and S8541,S8544 on the
no-lock path) rule lists per the strengthened patterns doc.

Trigger unchanged: workflow_call only (CI-053 enforced; mutation testing
must remain schedule + workflow_dispatch in downstream callers).

The `Run mutation testing` step retains its `run: |` block because the
shell logic uses multi-line uv run calls with literal `--frozen`. Per the
strengthened guidance, preceding-line NOSONAR inside `run: |` is fragile
but the literal `--frozen` keeps S8544 from firing structurally. Verify
the SonarCloud gate per PR; if S8541 fires here, restructure to single-
line shape per Option 3.
williaby and others added 2 commits May 25, 2026 18:41
…ted uv run

Address two findings from PR #172 review (Copilot, local pr-review):

1. Skip path leaves workflow_call outputs empty.
   The job's score/passed outputs sourced from steps.analyze.outputs.*,
   which evaluate to empty strings when analyze is gated off on the skip
   path. Downstream callers using `if: needs.mutation.outputs.passed ==
   'true'` silently fell through to false. Add a `set-skip-outputs`
   step that writes deterministic score=0, passed=true, and skipped=true,
   and coalesce the job-output expressions across analyze and the new
   step. Expose `skipped` as a new workflow_call output so sophisticated
   callers can distinguish "skipped" from "tested and passed".

2. Nested `uv run mutmut results --json` subprocess inside the analyze
   step's Python heredoc omitted --frozen and \$NO_BUILD_FLAG, breaking
   the PR body's stated invariant that literal --frozen is on every
   `uv run` line. Build the command in Python so both flags propagate,
   restoring the lockfile/no-build guarantee on this invocation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Document the python-mutation.yml changes from PR #172 in the
[Unreleased] Fixed section, covering the detect-state pattern
application, deterministic skip-path outputs (with new `skipped`
output), and the nested subprocess --frozen/\$NO_BUILD_FLAG fix.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@williaby

Copy link
Copy Markdown
Collaborator Author

PR Fix Summary

Addressed all 3 review findings (2 from Copilot, 1 from local pr-review) in two commits on the rebased branch:

Commit Concern
b08e83f Workflow: skip-path outputs + nested uv run
e30cad7 CHANGELOG entry under [Unreleased] → ### Fixed

1. Copilot: skip path leaves workflow_call outputs empty

On the skip path (no pyproject.toml), the workflow skips the analyze step, but the job and workflow_call outputs (mutation-score, passed) are still sourced from steps.analyze.outputs.*, which will be empty.

Fixed in b08e83f. Added a set-skip-outputs step (gated on state == 'skip') that writes score=0, passed=true, skipped=true to $GITHUB_OUTPUT, plus a coalescing job-output expression (steps.analyze.outputs.X || steps.set-skip-outputs.outputs.X) so downstream callers always receive deterministic values. Also exposed skipped as a new workflow_call output so sophisticated callers can distinguish "skipped" from "tested and passed". Chose passed=true (not false) because the workflow's default is fail-under-threshold: false (advisory, not a hard gate), so the skip path should pass through downstream if: needs.mutation.outputs.passed == 'true' checks rather than block them.

2. Copilot: nested uv run mutmut results --json bypasses --frozen and $NO_BUILD_FLAG

The Analyze mutation results step runs uv run mutmut results --json from the embedded Python without --frozen and without honoring the NO_BUILD_FLAG derived from the no-build input.

Fixed in b08e83f. The Python heredoc now builds the subprocess argv with --frozen and $NO_BUILD_FLAG from env: cmd = ['uv', 'run', '--frozen']; cmd.append(no_build_flag) if no_build_flag else None; cmd.extend(['mutmut', 'results', '--json']). Restores the lockfile/no-build guarantee that the other three uv run invocations in the file already carry. Updated the PR body's "Pattern notes" so the invariant claim now genuinely holds across all four uv run invocations.

3. CLAUDE.md compliance: CHANGELOG.md not updated for a fix: commit

Fixed in e30cad7. Added a ### Fixed entry under [Unreleased] covering the detect-state pattern application and both Copilot fixes.

Branch state

  • Was 1 commit BEHIND origin/main (docs(sonarcloud): strengthen NOSONAR rule with Wave 1C empirical finding #170 had merged after this branch). Rebased clean (no conflicts) and force-pushed before applying fixes, so the GitHub Actions merge-gate no-op behavior on BEHIND PRs would not strand the new fix commits.
  • PR body Test plan checkboxes for CI and SonarCloud Quality Gate updated to [x] (both were already passing pre-rebase).

Pre-commit passing locally (yamllint, markdownlint, detect-secrets, trufflehog, no-em-dash, commitizen). CI re-run triggered by the push.

Note on thread replies: the project's .claude/settings.json denies gh api -X POST (memory: PR #108 hardening), so individual review-thread replies could not be auto-posted. Per-thread resolutions are documented inline above. The threads on lines 201 and 252 can be marked resolved in the GitHub UI.

🤖 Generated with Claude Code

@williaby
williaby merged commit 0b9f8e5 into main May 26, 2026
28 checks passed
@williaby
williaby deleted the chore/python-mutation-detect-state branch May 26, 2026 01:47
williaby added a commit that referenced this pull request May 26, 2026
PR #170 strengthened the NOSONAR rule based on Wave 1C (PR #166), which
found that comma-separated NOSONAR and inline-trailing NOSONAR inside
run: | blocks are not honored. The strengthened rule was framed as "any
NOSONAR inside run: | is unreliable; treat Pattern B as a lucky
non-failure."

Subsequent Waves 2B (PR #172, python-mutation) and 2C (PR #173, python-
performance-regression) re-validated Pattern B with passing SonarCloud
gates. Both PRs kept multi-line run: | blocks with literal --frozen on
every uv line and preceding-line single-rule # NOSONAR(S8541). Both
achieved gate=OK with 0 vulns on first push.

The evidence supports a more nuanced rule: Pattern B is reliable when
three conditions hold simultaneously.

1. Every uv invocation in the run: | block uses literal --frozen
   (not dynamic $FROZEN_FLAG).
2. NOSONAR is single-rule S8541 only, not comma-separated.
3. NOSONAR is on the preceding line, not inline-trailing.

Updates to docs/sonarcloud-nosonar-patterns.md:

- TL;DR table: split the prior "Preceding-line with literal --frozen"
  row into two rows that distinguish the working condition (single-rule
  S8541) from the failure modes (comma-separated or inline-trailing).
- TL;DR Rule of thumb: replace "single-line run: only" framing with
  "two predictable shapes" framing, listing both Pattern A and the
  conditioned Pattern B.
- Recommended Patterns: replace "Pattern B (fragile)" framing with
  "Pattern B (acceptable under specific conditions)" plus an explicit
  three-condition reliability list. Note 3-for-3 empirical track record
  when conditions hold.
- Date line: add "nuanced 2026-05-26" with PR pointer.

The earlier strengthened rule was over-pessimistic. This nuance does not
contradict it; both PRs #166 and #170 documented failure modes correctly,
and PRs #172 and #173 demonstrated the working conditions. Future Phase 2
or 3 implementers should default to Pattern A but may use Pattern B when
the three conditions hold.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
williaby added a commit that referenced this pull request May 26, 2026
…ce (#174)

* docs(sonarcloud): nuance Pattern B reliability with Wave 2B/2C evidence

PR #170 strengthened the NOSONAR rule based on Wave 1C (PR #166), which
found that comma-separated NOSONAR and inline-trailing NOSONAR inside
run: | blocks are not honored. The strengthened rule was framed as "any
NOSONAR inside run: | is unreliable; treat Pattern B as a lucky
non-failure."

Subsequent Waves 2B (PR #172, python-mutation) and 2C (PR #173, python-
performance-regression) re-validated Pattern B with passing SonarCloud
gates. Both PRs kept multi-line run: | blocks with literal --frozen on
every uv line and preceding-line single-rule # NOSONAR(S8541). Both
achieved gate=OK with 0 vulns on first push.

The evidence supports a more nuanced rule: Pattern B is reliable when
three conditions hold simultaneously.

1. Every uv invocation in the run: | block uses literal --frozen
   (not dynamic $FROZEN_FLAG).
2. NOSONAR is single-rule S8541 only, not comma-separated.
3. NOSONAR is on the preceding line, not inline-trailing.

Updates to docs/sonarcloud-nosonar-patterns.md:

- TL;DR table: split the prior "Preceding-line with literal --frozen"
  row into two rows that distinguish the working condition (single-rule
  S8541) from the failure modes (comma-separated or inline-trailing).
- TL;DR Rule of thumb: replace "single-line run: only" framing with
  "two predictable shapes" framing, listing both Pattern A and the
  conditioned Pattern B.
- Recommended Patterns: replace "Pattern B (fragile)" framing with
  "Pattern B (acceptable under specific conditions)" plus an explicit
  three-condition reliability list. Note 3-for-3 empirical track record
  when conditions hold.
- Date line: add "nuanced 2026-05-26" with PR pointer.

The earlier strengthened rule was over-pessimistic. This nuance does not
contradict it; both PRs #166 and #170 documented failure modes correctly,
and PRs #172 and #173 demonstrated the working conditions. Future Phase 2
or 3 implementers should default to Pattern A but may use Pattern B when
the three conditions hold.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(sonarcloud): resolve internal contradictions in Pattern B framing

Addresses PR #174 review findings:

Critical
- Anti-patterns heading at line 145 narrowed from "any NOSONAR inside
  run: | blocks" to "unreliable NOSONAR forms inside run: | blocks",
  since the body of the same PR now endorses Pattern B (preceding-line
  single-rule NOSONAR with literal --frozen) inside run: | blocks.
- Hypothesis section (lines 88-107) rewritten: the absolute Wave 1C
  claim that "no NOSONAR placement inside a run: | block scalar is
  reliable" was the surviving prior framing that contradicted the new
  Pattern B framing. Reframed as historical Wave 1C conclusion now
  superseded by Wave 2B/2C re-validation.

Important
- "3-for-3" reliability claim qualified: PR #173 is open at time of
  writing with a passing pre-merge gate; reframed as "2 merged
  applications plus 1 pre-merge gate pass" so readers do not treat all
  three legs as production-validated.

Suggested
- Pattern B uv-line count corrected from "11 uv lines across 4 blocks"
  to "11 uv lines across 5 blocks" (actual count of run: | blocks
  added in PR #173).
- TL;DR rule-of-thumb sentence deduplicated against the identical
  sentence in the Pattern B detail section, now references the
  Verification protocol section instead.

Informational
- "Lucky non-failure" jargon at line 141 rewritten to be
  self-contained for first-time readers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants