Repository navigation
feat(ci): add opt-in parallel test-bucket jobs to python-ci.yml - #269
Conversation
quality-checks currently runs unit, integration, and security pytest buckets as sequential steps in a single job; for repos with a large integration suite this puts several minutes of serial pytest on the critical path. Add an opt-in `parallel-tests` input (default false) that splits those buckets into three independent jobs (test-unit, test-integration, test-security) plus a coverage-combine job that merges their coverage data and republishes the same coverage-reports artifact downstream callers already consume, so no caller needs to change to pick up the artifact. Default-off: when parallel-tests is unset, quality-checks behaves byte-identically to before (5 step-level `if:` conditions gain `&& !inputs.parallel-tests`, which is a no-op when the input is false), and the new jobs report skipped, which ci-gate already treats as acceptable for optional jobs (matching the existing enable-matrix-testing pattern). Signed-off-by: Claude <noreply@anthropic.com>
|
Warning Review limit reached
Next review available in: 53 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR adds an opt-in parallel-tests mode to the reusable python-ci.yml workflow to reduce CI wall clock time by splitting unit, integration, and security pytest buckets into parallel jobs, then recombining coverage into the same coverage-reports artifact consumers already expect.
Changes:
- Added a new
parallel-testsboolean input (defaultfalse) to enable parallelized test execution. - Introduced
test-unit,test-integration,test-securityjobs that run pytest buckets independently and publish per-bucket coverage data artifacts. - Added a
coverage-combinejob to download per-bucket artifacts, runcoverage combine, and re-upload the mergedcoverage-reportsartifact, plus updatedci-gateto include the new jobs.
| if: | | ||
| always() && inputs.parallel-tests && | ||
| needs.test-unit.result == 'success' && | ||
| (needs.test-integration.result == 'success' || needs.test-integration.result == 'skipped') && | ||
| (needs.test-security.result == 'success' || needs.test-security.result == 'skipped') | ||
| permissions: |
| - name: Install dependencies | ||
| run: uv sync --all-extras --frozen $NO_BUILD_FLAG | ||
| shell: bash |
| parallel-tests: | ||
| description: >- | ||
| Opt-in: run the unit/integration/security test buckets as separate | ||
| parallel jobs (test-unit, test-integration, test-security) instead | ||
| of sequential steps inside quality-checks, with a coverage-combine | ||
| job producing the same coverage-reports artifact downstream callers | ||
| already expect. Default false preserves the existing single-job | ||
| behavior for all other callers. | ||
| required: false |
Qlty/actionlint (shellcheck SC2086) and SonarCloud (S8541) both flagged the new-code $NO_BUILD_FLAG/$GITHUB_OUTPUT usages introduced in test-unit/test-integration/test-security/coverage-combine: these are verbatim copies of quality-checks' own patterns, but the accompanying suppression comments only carried over for the branches that already had them there, not for every new occurrence. Quote $GITHUB_OUTPUT (always safe) and add the shellcheck disable + NOSONAR(S8541) comments the un-quoted $NO_BUILD_FLAG word-splitting requires, verified against a real shellcheck binary rather than guessed. Also fixes two real bugs Copilot's review caught in the new jobs: - coverage-combine could run even when the caller repo has no pyproject.toml: test-unit's own steps all no-op in that case (state= skip), so it reports success with no artifact uploaded, and coverage-combine's download-artifact step would fail trying to fetch it. test-unit now exposes its detected repo-state as a job output; coverage-combine's `if:` gates on it not being 'skip', matching quality-checks' existing graceful-skip behavior. - coverage-combine always ran `uv sync --frozen`, which requires a committed uv.lock; the workflow explicitly supports uv-managed callers without one (uv-no-lock). coverage-combine now runs its own lockfile detection and branches the install the same way every other job here does. Also documents the new `parallel-tests` input and its `relative_files = true` caller requirement in docs/workflows/python-ci.md (previously undocumented per review). Signed-off-by: Claude <noreply@anthropic.com>
The previous commit put NOSONAR on its own line preceding the flagged command (to sit alongside the shellcheck disable directive, which must precede). SonarCloud's githubactions plugin evidently only recognizes NOSONAR as a trailing same-line comment on the flagged line itself: the one pre-existing pattern in this file that is genuinely clean on main (quality-checks' "Install dependencies (uv without lockfile)" one-liner) uses a trailing comment, and every occurrence I added with a preceding comment was still flagged as a fresh vulnerability on this PR's diff despite the suppression text being present. Keep `# shellcheck disable=SC2086` on its own preceding line (shellcheck requires this) and move `# NOSONAR(...)` back onto the same line as the command it annotates, matching the codebase's only confirmed-working example. Verified against a real shellcheck binary that this ordering still satisfies SC2086 with zero warnings across all 15 affected lines. Signed-off-by: Claude <noreply@anthropic.com>
PR ReviewPREMISE QUESTION: Feature is justified and opt-in, but triplicates ~480 lines of job boilerplate that this file's own Critical (must fix before merge)
Important (should fix)
SonarQube: 16 issues queued for auto-fix (all resolved by Critical item 1). Copilot review received (3 comments); all were already addressed at 0d02c92. CodeRabbit check is green but no review ran (rate-limited). 🤖 Generated with Claude Code |
The org's own docs/sonarcloud-nosonar-patterns.md (surfaced via review) documents this exact failure precisely: SonarCloud's githubactions ruleset only reliably honors NOSONAR as a trailing comment on a genuine single-line YAML `run:` scalar (Pattern A), not a preceding-line comment inside a `run: |` block (Pattern B is reliable only for multi-line backslash-continued commands with a literal --frozen, e.g. the pytest steps, which is why those were never flagged across any of this feature's prior commits). The previous commit used a preceding-line comment for the single-line install/coverage commands, an anti-pattern the doc traces back to PR #166 commit bb31ec1. Rebuild every affected `uv sync`/`uv run` line as a bash array (NO_BUILD_ARGS=(); [ -n "$NO_BUILD_FLAG" ] && NO_BUILD_ARGS+=(...); ... "${NO_BUILD_ARGS[@]}") on a single physical `run:` line with a trailing NOSONAR comment. This satisfies Pattern A for SonarCloud while staying shellcheck-clean without needing a `# shellcheck disable=SC2086` comment (which itself requires its own preceding line and so cannot coexist with a single-line run:). Verified against a real shellcheck binary, and that the array construction survives `set -e` when the flag is empty (GitHub Actions bash steps run with -e by default). Also addresses the review's other findings: - Fix the literal em-dash in docs/workflows/python-ci.md (hard project rule). - Add a CHANGELOG.md entry under [Unreleased] for the new input. - Add `fetch-depth: 0` and a "Validate source layout" step to the three new test-* jobs, matching quality-checks (previously only coverage-combine and quality-checks had these). - coverage-combine's coverage-combine step now uses --keep and || true, matching quality-checks' own combine step. - coverage-combine now runs its own "Verify relative_files is configured" preflight (parses pyproject.toml with tomllib) and fails fast with an actionable error instead of silently under-counting coverage when a caller opts in without relative_files = true. - Drop coverage-combine's final upload step's `if: always()` to match quality-checks' upload step (no always(); only runs on success). Not addressed here (raised as a design question, not a bug): the review's premise question about collapsing test-unit/test-integration/ test-security into one `strategy.matrix` job. That's a real simplification but changes the job/check-name surface this feature exposes to callers; deferring to a follow-up discussion rather than folding into this already-large diff. Signed-off-by: Claude <noreply@anthropic.com>
|
Pushed 0241ecf addressing all Critical/Important items:
On the premise question (collapsing test-unit/test-integration/test-security into one Generated by Claude Code |
|
Was pinned to a commit on the feature branch to trial parallel-tests before ByronWilliamsCPA/.github#269 merged (per the #VERIFY note it carried). Now points at 24c0593, the squash-merge commit on main, so CI no longer depends on the feature branch continuing to exist.



Description
quality-checkscurrently runs unit, integration, and security pytest buckets as sequential steps inside a single job. For a repo with a sizable integration suite (e.g. cyo-adventure: ~5.5 min of integration tests alone), this puts several minutes of serial pytest on the CI critical path even though the three buckets are independent and could run concurrently.This adds an opt-in
parallel-testsinput (defaultfalse) that splits those buckets into three parallel jobs —test-unit,test-integration,test-security— plus acoverage-combinejob that merges their coverage data and republishes the samecoverage-reportsartifact (same name, same file layout) thatquality-checksproduces today, so callers that already download that artifact need no changes.Related Issue
N/A — internal CI performance investigation for cyo-adventure.
Type of Change
Changes Made
parallel-testsboolean input (defaultfalse).test-unit,test-integration,test-security: each does its own checkout/install and runs one pytest bucket, writing coverage to a job-scoped.coverage.<type>data file.coverage-combinejob: downloads all three coverage-data artifacts, runscoverage combine+coverage report --fail-under, and uploads the merged result as thecoverage-reportsartifact (matching the non-parallel path's name and file set exactly).quality-checks's own unit/integration/security/coverage-combine/upload steps are now gated&& !inputs.parallel-tests, so it becomes lint/format/typecheck/Vulture/Bandit/pip-audit only when the flag is on.ci-gatenow also depends on the four new jobs, using the samesuccess or skippedacceptance already used formatrix-testing— whenparallel-testsis unset, they reportskippedand the gate is unaffected.Testing
Test Details
if:expression decodes to the intended string (python3 -c "yaml.safe_load(...)").if:gates gaining&& !inputs.parallel-tests, plusci-gate'sneeds:list) — everything else is additive, so existing callers with the input unset get byte-identical behavior.actionlintwasn't available in the sandbox this was authored in. cyo-adventure is opting in as the first real-world test; see companion PR there.coverage combinestep depends on the caller'spyproject.tomlsetting[tool.coverage.run] relative_files = trueso coverage data recorded on three separate runners merges correctly. cyo-adventure already sets this. Callers that opt in without it may see an inaccurate combined coverage percentage.Checklist
git commit --signoff(DCO)Reviewer Notes
Recommend merging this once cyo-adventure's opt-in run confirms the coverage-combine step produces an accurate merged coverage number and the wall-clock improvement materializes as expected (~11 min → ~6-7 min was the estimate). Happy to iterate on the job split if review turns up a cleaner structure.
Generated by Claude Code