Skip to content

fix: resolve GitHub Actions workflow compatibility issues - #3

Merged
williaby merged 2 commits into
mainfrom
workflow-fixes-pr
Dec 6, 2025
Merged

williaby merged 2 commits into
mainfrom
workflow-fixes-pr

Conversation

@williaby

@williaby williaby commented Dec 6, 2025 •

Copy link
Copy Markdown
Collaborator

Summary

This PR fixes critical GitHub Actions workflow compatibility issues that were preventing reusable workflows from executing correctly.

Changes

1. python-compatibility.yml - Fixed workflow output reference

  • Added job-level outputs section to compatibility-summary job
  • Created step to capture needs.test-matrix.result as a step output
  • Workflow output now correctly references job output via jobs.compatibility-summary.outputs.result

Why this matters: GitHub Actions doesn't allow workflow outputs to directly reference job results (jobs.test-matrix.result). They must be captured as job outputs first.

2. python-ci.yml - Fixed string vs number comparison

  • Added fromJSON() to convert step outputs before numeric comparison
  • Prevents type mismatch issues in conditional expressions
  • Affects RAD tag checking and LLM tag validation

3. python-release.yml - Standardized if expressions

  • Updated all if expressions to use ${{ }} syntax consistently
  • Ensures proper parsing and evaluation across all GitHub Actions contexts

4. python-codecov.yml - Improved token compatibility

  • Changed from secrets.GITHUB_TOKEN to github.token
  • Better compatibility in reusable workflow context

5. python-mutation.yml - Restored visual feedback

  • Restored emoji characters in PR comment template
  • Improves readability of mutation testing results

6. python-slsa.yml - Converted to template format

  • Changed from reusable workflow to template documentation
  • GitHub Actions doesn't support nested reusable workflow calls for SLSA generator

Validation

✅ YAML Syntax: All workflow files validated with Python yaml parser
✅ ShellCheck: All shell scripts pass linting
✅ Compatibility: Follows GitHub Actions best practices

Testing

These fixes resolve workflow execution errors related to:

  • Output referencing in reusable workflows
  • Type conversion in conditional expressions
  • Token authentication in cross-repository contexts

Additional Changes

  • Fixed ShellCheck issues in sync_org_files.sh (SC2016, SC1112)

Test Plan

  • Trigger workflows in a test repository
  • Verify workflow outputs are properly captured
  • Confirm conditional expressions evaluate correctly
  • Validate SLSA generator integration works

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

  • Chores
    • Standardized CI/CD workflow configurations and expression syntax across Python testing, release, and provenance pipelines for improved reliability.
    • Enhanced mutation testing feedback with visual status indicators (✅ for pass, ⚠️ for issues).
    • Converted SLSA artifact provenance workflow from automatic execution to manual trigger mode for greater control.

✏️ Tip: You can customize this high-level summary in your review settings.

- python-slsa.yml: Convert from reusable workflow to template since
  GitHub Actions doesn't support nested reusable workflow calls. The
  SLSA generator must be called directly from consuming workflows.

- python-ci.yml: Fix string vs number comparison in if expressions
  by using fromJSON() to properly convert step outputs before comparison.

- python-codecov.yml: Use github.token instead of secrets.GITHUB_TOKEN
  for better compatibility in reusable workflow context.

- python-release.yml: Standardize all if expressions to use ${{ }}
  syntax for consistency and to avoid potential parsing issues.

- python-mutation.yml: Restore missing emoji characters in PR comment
  template for proper visual feedback.
The workflow-level output was incorrectly referencing jobs.test-matrix.result
directly, which is not allowed. Job results are only available in the needs
context of dependent jobs.

Fixed by:
- Adding a 'result' output to the compatibility-summary job
- Using a step to capture needs.test-matrix.result as a step output
- Referencing the job output in the workflow-level output
@coderabbitai

coderabbitai Bot commented Dec 6, 2025 •

Copy link
Copy Markdown

Walkthrough

GitHub Actions workflows updated across multiple pipelines: conditional expressions wrapped with fromJSON for JSON awareness, token source changed, matrix result propagation modified, mutation testing summary output altered, release workflow expressions corrected, and SLSA provenance workflow converted from reusable to template-driven.

Changes

Cohort / File(s) Summary
Expression & Condition Fixes
.github/workflows/python-ci.yml, .github/workflows/python-release.yml
python-ci.yml: Wrapped numeric outputs with fromJSON() for conditional comparisons (rad_tags and llm_tags). python-release.yml: Converted conditional expressions to proper GitHub Actions syntax using ${{ ... }} wrapper format (e.g., if: inputs.X → if: ${{ inputs.X }}).
Authentication & Token Updates
.github/workflows/python-codecov.yml
Changed secrets.GITHUB_TOKEN to github.token in the "Download coverage artifact" step for improved token source handling.
Matrix Result Propagation
.github/workflows/python-compatibility.yml
Added compatibility-summary job with new Set result output step and result output. Updated workflow matrix-result output to reference jobs.compatibility-summary.outputs.result instead of jobs.test-matrix.result.
Mutation Testing Output Changes
.github/workflows/python-mutation.yml
Changed emoji assignment from static empty string to conditional logic ('✅' on pass, '⚠️' on fail) in PR comment step. Removed the "Mutation Testing Summary" step entirely.
Workflow Type & Input Conversion
.github/workflows/python-slsa.yml
Renamed from "SLSA Provenance (Reusable)" to "SLSA Provenance (Template)". Converted trigger from workflow_call to workflow_dispatch. Added new inputs: base64-subjects, upload-assets, upload-tag-name, provenance-name. Updated provenance job to propagate these inputs to SLSA generator.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Areas requiring extra attention:

  • .github/workflows/python-ci.yml and .github/workflows/python-release.yml: Verify that fromJSON() wrapping and expression syntax changes produce correct conditional evaluations in all edge cases
  • .github/workflows/python-compatibility.yml: Confirm that result propagation through the new compatibility-summary job output correctly preserves test-matrix status
  • .github/workflows/python-slsa.yml: Review workflow conversion from reusable to template-driven to ensure all downstream consumers are updated and input propagation is correct

Poem

🐰 Workflows now gleam with JSON-wrapped precision,
Templates bloom where reusables once stood tall,
Result cascades flow through compatibility's decision,
Emojis dance on mutations' win or fall,
Hopping onward, our pipelines streamline! ✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main objective of the PR: resolving GitHub Actions workflow compatibility issues.
Description check ✅ Passed The description comprehensively covers all required sections: clear summary, detailed changes across six files, validation steps performed, and a test plan. All key modifications are well-explained with context.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch workflow-fixes-pr

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3bf8bf5 and 65922ab.

📒 Files selected for processing (6)
  • .github/workflows/python-ci.yml (2 hunks)
  • .github/workflows/python-codecov.yml (1 hunks)
  • .github/workflows/python-compatibility.yml (2 hunks)
  • .github/workflows/python-mutation.yml (1 hunks)
  • .github/workflows/python-release.yml (5 hunks)
  • .github/workflows/python-slsa.yml (3 hunks)
🔇 Additional comments (10)
.github/workflows/python-codecov.yml (1)

124-124: Using github.token for reusable workflow token compatibility.

Replacing secrets.GITHUB_TOKEN with github.token is the recommended approach for reusable workflows. This change improves token handling in the download-artifact action and aligns with GitHub Actions best practices for workflow composition.

.github/workflows/python-mutation.yml (2)

235-235: Emoji indicators improve mutation testing result visibility.

The conditional emoji logic (✅ for pass, ⚠️ for below threshold) provides clear visual feedback in PR comments. This enhances readability and makes mutation testing status immediately apparent.


226-291: AI summary inconsistency: "Mutation Testing Summary" step is not removed.

The AI summary states that the "Mutation Testing Summary" step (lines 293–307) was removed entirely, but the code shows this step still exists in the workflow. Both the PR comment (lines 226–291) and the step summary now display mutation results, providing dual visibility channels. This is not an issue, but the summary was inaccurate about the scope of changes.

.github/workflows/python-ci.yml (2)

328-328: JSON-aware numeric comparison for RAD tag count.

Wrapping steps.check-tags.outputs.rad_tags with fromJSON() ensures type-safe comparison. Since shell outputs are always strings, the fromJSON() conversion is necessary to compare the string value against the numeric zero correctly. This prevents potential string-vs-number type mismatches.


339-339: JSON-aware numeric comparison for LLM tag count.

The fromJSON() wrapper on steps.check-tags.outputs.llm_tags ensures proper numeric comparison in the combined conditional. This prevents type mismatch issues and aligns with the fix applied to the RAD tag check on line 328.

.github/workflows/python-release.yml (2)

113-113: Expression syntax standardization for consistency.

Wrapping the condition with ${{ inputs.run-tests }} ensures consistent YAML expression parsing across the workflow. This standardization improves readability and aligns all conditionals with modern GitHub Actions syntax conventions.


198-198: Consistent expression wrapper syntax across all conditional statements.

All conditional statements in the release workflow have been standardized to use explicit ${{ }} wrappers. This ensures uniform parsing behavior, improves maintainability, and prevents subtle YAML evaluation issues. The changes span multiple jobs (SBOM generation, semantic release, signing, GitHub release upload, and PyPI publishing) and are applied consistently throughout.

Also applies to: 215-215, 225-225, 232-232, 240-240, 252-252, 298-298

.github/workflows/python-slsa.yml (2)

1-53: Correct conversion from reusable workflow to template documentation.

Converting SLSA provenance to a template-driven approach is the right solution. GitHub Actions currently doesn't support nested reusable workflow calls, and since SLSA provenance requires calling the official SLSA generator (itself a reusable workflow), direct invocation is the only viable pattern. The documentation clearly explains the limitation and provides a straightforward copy-paste example for integration into downstream workflows.


60-83: Workflow dispatch inputs enable template usage and manual testing.

The workflow_dispatch trigger with inputs (base64-subjects, upload-assets, upload-tag-name, provenance-name) allows both manual triggering and serves as a reference for the provenance job inputs that users must pass when copying the job into their own workflows. The default values and fallback logic (e.g., upload-tag-name || github.ref_name) are sensible and user-friendly.

.github/workflows/python-compatibility.yml (1)

83-83: Result propagation through compatibility-summary job enables proper output handling.

Introducing the compatibility-summary job as an intermediary to capture and propagate needs.test-matrix.result is a solid pattern. This enables the job to always run (via if: always()) while still exposing the test-matrix result to calling workflows via a step output. The workflow's public output now references the propagated result, ensuring downstream jobs receive the correct test status even when the matrix job completes with a non-success status.

Also applies to: 182-188


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@williaby
williaby merged commit 000bb33 into main Dec 6, 2025
5 checks passed
@williaby
williaby deleted the workflow-fixes-pr branch December 10, 2025 03:07
williaby added a commit that referenced this pull request Dec 11, 2025
Fixes multiple reusable workflow issues causing failures in downstream projects:

1. **FIPS Compatibility (python-fips-compatibility.yml)**
   - Update astral-sh/setup-uv from invalid SHA to stable v5.1.1
   - Resolves "action could not be found" startup failure

2. **Python Compatibility Matrix (python-compatibility.yml)**
   - Add -c flag to jq for compact single-line JSON output
   - Fixes "Invalid format" error in $GITHUB_OUTPUT processing

3. **ClusterFuzzLite Fuzzing (python-fuzzing.yml)**
   - Update google/clusterfuzzlite actions from invalid SHA to @v1
   - Fixes startup failures for build_fuzzers, run_fuzzers, and prune actions

4. **SonarCloud Integration (python-ci.yml)**
   - Add SONAR_TOKEN check to job condition
   - Prevents execution failure when token not configured
   - Gracefully skips analysis if token missing

**Impact**: Resolves all 4 workflow startup failures identified in
rag-processor PR #3. Workflows will now execute successfully or skip
gracefully when optional secrets are missing.

**Testing**: Validated against rag-processor/pull/3 failure scenarios

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
williaby added a commit that referenced this pull request Dec 11, 2025
Fixes multiple reusable workflow issues causing failures in downstream projects:

1. **FIPS Compatibility (python-fips-compatibility.yml)**
   - Update astral-sh/setup-uv from invalid SHA to stable v5.1.1
   - Resolves "action could not be found" startup failure

2. **Python Compatibility Matrix (python-compatibility.yml)**
   - Add -c flag to jq for compact single-line JSON output
   - Fixes "Invalid format" error in $GITHUB_OUTPUT processing

3. **ClusterFuzzLite Fuzzing (python-fuzzing.yml)**
   - Update google/clusterfuzzlite actions from invalid SHA to @v1
   - Fixes startup failures for build_fuzzers, run_fuzzers, and prune actions

4. **SonarCloud Integration (python-ci.yml)**
   - Add SONAR_TOKEN check to job condition
   - Prevents execution failure when token not configured
   - Gracefully skips analysis if token missing

**Impact**: Resolves all 4 workflow startup failures identified in
rag-processor PR #3. Workflows will now execute successfully or skip
gracefully when optional secrets are missing.

**Testing**: Validated against rag-processor/pull/3 failure scenarios

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
williaby added a commit that referenced this pull request May 26, 2026
…n-main, revert Pattern A churn

Revert commit 46e462b. Per docs/sonarcloud-nosonar-patterns.md (nuanced
in PR #174, merged 2026-05-25), inline-trailing NOSONAR inside `run: |`
block scalars is NOT honored by SonarCloud's githubactions rule set, and
the preceding-line Pattern B form IS reliable when three conditions hold:
literal --frozen on every uv line, single-rule S8541 only, no dynamic
$FROZEN_FLAG. The collapse-to-single-line approach in 46e462b violated
condition #3 (inline-trailing) and triggered 21 new MAJOR vulnerabilities
on the SonarCloud gate, up from 11. This revert restores Pattern B at
all original locations.

Restructure the `Install dependencies on main` step (still incorrect
even at the fda25ca state) to satisfy SonarCloud:

- Replace the `run: |` block + `[ -f uv.lock ]` shell check (which used
  comma-separated NOSONAR(S8541,S8544) for the uv-no-lock branch,
  violating Pattern B condition #2) with two single-line `run:` steps.
- Gate each step with `hashFiles('uv.lock')` instead of the stale
  `steps.detect.outputs.state`. hashFiles() re-evaluates per step
  against the current worktree, so after `Checkout Main Branch` it
  correctly reflects main's tree (the original Copilot finding).
- Each single-line step uses Pattern A (inline NOSONAR), which IS the
  most reliable form per the docs and accepts comma-separated rule
  lists (the uv-no-lock branch carries NOSONAR(S8541,S8544)).

This combines the correctness fix for the Copilot install-on-main
finding with the SonarCloud-compatible suppression shape, replacing
the previous flawed attempt.
williaby added a commit that referenced this pull request May 26, 2026
…coding uv sync --frozen (#173)

* fix(python-performance-regression): detect repo state instead of hardcoding uv sync/run --frozen

Apply established NOSONAR detect-state pattern (see python-mutation.yml:159-169)
to the largest reusable workflow in the org (642 lines, 17 unguarded uv
invocations across 8 install/run steps).

Changes:
- Add `Detect repo state` step that branches on pyproject.toml + lockfile
  presence: skip when no pyproject, fail fast on poetry repos with actionable
  error, install with --frozen on uv-locked repos, install without --frozen
  on uv-no-lock repos (uv sync creates the lockfile on that path).
- Add `Resolve extra dependency flags` helper step (Option 3 from PR #166)
  that builds a flat "--extra dev --extra ml" string output, so the per-state
  install steps can be single-line Pattern A NOSONAR placement.
- Split the two install steps (PR-branch and main-branch) into uv-locked /
  uv-no-lock variants each, four total. Pattern A inline NOSONAR(S8541) on
  uv-locked (literal --frozen visible); NOSONAR(S8541,S8544) on uv-no-lock
  where --frozen is structurally absent by design.
- Apply Pattern B (preceding-line NOSONAR(S8541) inside `run: |`) to the
  three multi-line shell blocks (Run PR Benchmarks, Run Baseline Benchmarks,
  Compare Performance). Every uv invocation in these blocks carries literal
  --frozen, so only S8541 needs suppression. Per docs/sonarcloud-nosonar-
  patterns.md strengthened 2026-05-26, Pattern B is acceptable when literal
  --frozen is structurally present and only S8541 fires. The blocks are
  genuinely multi-line (capability detection branching with three argv
  variants, Python heredoc) so collapsing to single-line is not feasible
  without significant readability loss.
- Add `Skip notice` step for the no-pyproject path so downstream callers
  see a clear summary entry instead of a silent green.
- Guard all downstream benchmark / summary / PR-comment / fail steps with
  `if: steps.detect.outputs.state == 'uv-locked' || == 'uv-no-lock'` so
  the workflow exits cleanly when no pyproject is present.

S8541 suppression rationale: `$NO_BUILD_FLAG` (from inputs.no-build, default
'true' -> '--no-build') is dynamic by design so consumers with PEP 517 build
backends like hatchling can pass `no-build: false`. SonarCloud cannot resolve
the env var. The lockfile-based `uv sync --frozen` (uv-locked path) constrains
installable versions to the consumer's audited uv.lock, bounding residual
setup-script risk.

Wave 2C of CI Repair Sprint Phase 2. Largest single-file change in the sprint.

* fix(python-performance-regression): address Copilot review findings on PR #173

Resolves four issues raised in Copilot's review and one related design
smell surfaced by the /pr-review pass.

Critical:
- Install-on-main now re-detects uv.lock presence inline via [ -f uv.lock ]
  rather than trusting steps.detect.outputs.state. The original gating
  used the PR-branch detection after `git checkout origin/main`, which
  misclassifies main when the PR adds or removes uv.lock. Collapses the
  two install-on-main steps into one with a shell-level branch.
- Validate Benchmark Script now carries the same detect-state guard as
  adjacent steps, so the skip path (no pyproject.toml) actually skips
  instead of failing on a missing benchmark script.

Important:
- Compare Performance no longer calls sys.exit(1) on regression. The
  dedicated `Fail on Regression` step at the end of the job is now the
  single source of failure-on-regression behavior. Post PR Comment and
  Performance Summary now run on regression, which is the case where
  they matter most. Removes the unused `import sys` and the redundant
  FAIL_ON_REGRESSION env var on the Compare step.
- Resolve extra dependency flags validates each extra against the PEP
  508 allowlist (^[A-Za-z0-9._-]+\$) before emitting it; caller-controlled
  bytes can no longer reach the downstream `uv sync \$EXTRA_FLAGS`
  expansion. Whitespace separators (space, tab, or newlines from YAML
  block scalars) are normalized via `tr -s`.

Verification:
- actionlint -shellcheck= passes
- yamllint passes (existing line-length warnings unchanged)
- pre-commit run --files passes (no-em-dash hook included)

* fix(python-performance-regression): use same-line NOSONAR (Pattern A) for SonarCloud

SonarCloud quality gate failed on the previous push with 11 new S8541
(and one S8544) issues because the workflow's prior Pattern B (preceding-
line `# NOSONAR(...)` inside `run: |`) is not recognized by SonarCloud's
standard NOSONAR mechanism, which is documented as same-line only.

Conversion:
- Single-line invocations get NOSONAR appended at end of line (Generate
  Synthetic Test Data, Install dependencies on main x2, Compare
  Performance heredoc opener).
- Multi-line capability checks (`uv run ... --help | grep ...`) get
  NOSONAR appended at end of the existing single line.
- Multi-line benchmark execution branches in Run PR Benchmarks and Run
  Baseline Benchmarks have their `uv run` argv collapsed onto one line
  (no more `\`-continuations) so a same-line NOSONAR can sit on the same
  line as the `uv run --frozen $NO_BUILD_FLAG` invocation. The `|| {
  ... }` fallback brace groups stay multi-line for readability.

Stale comments about "Pattern B (preceding-line NOSONAR inside run: |)
is acceptable here because..." removed; replaced with brief notes
explaining the same-line NOSONAR rationale.

Verification:
- actionlint -shellcheck= passes
- yamllint passes (8 new line-length warnings from collapsed `uv run`
  lines; warnings only, not errors)
- pre-commit run --files passes

* fix(python-performance-regression): use hashFiles() to gate install-on-main, revert Pattern A churn

Revert commit 46e462b. Per docs/sonarcloud-nosonar-patterns.md (nuanced
in PR #174, merged 2026-05-25), inline-trailing NOSONAR inside `run: |`
block scalars is NOT honored by SonarCloud's githubactions rule set, and
the preceding-line Pattern B form IS reliable when three conditions hold:
literal --frozen on every uv line, single-rule S8541 only, no dynamic
$FROZEN_FLAG. The collapse-to-single-line approach in 46e462b violated
condition #3 (inline-trailing) and triggered 21 new MAJOR vulnerabilities
on the SonarCloud gate, up from 11. This revert restores Pattern B at
all original locations.

Restructure the `Install dependencies on main` step (still incorrect
even at the fda25ca state) to satisfy SonarCloud:

- Replace the `run: |` block + `[ -f uv.lock ]` shell check (which used
  comma-separated NOSONAR(S8541,S8544) for the uv-no-lock branch,
  violating Pattern B condition #2) with two single-line `run:` steps.
- Gate each step with `hashFiles('uv.lock')` instead of the stale
  `steps.detect.outputs.state`. hashFiles() re-evaluates per step
  against the current worktree, so after `Checkout Main Branch` it
  correctly reflects main's tree (the original Copilot finding).
- Each single-line step uses Pattern A (inline NOSONAR), which IS the
  most reliable form per the docs and accepts comma-separated rule
  lists (the uv-no-lock branch carries NOSONAR(S8541,S8544)).

This combines the correctness fix for the Copilot install-on-main
finding with the SonarCloud-compatible suppression shape, replacing
the previous flawed attempt.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants