Skip to content

Don't unblock run-level-concurrency-blocked runs in the resolver#37461

Merged
silverwind merged 6 commits into
go-gitea:mainfrom
silverwind:fix-blocked-run-resolver
May 4, 2026
Merged

Don't unblock run-level-concurrency-blocked runs in the resolver#37461
silverwind merged 6 commits into
go-gitea:mainfrom
silverwind:fix-blocked-run-resolver

Conversation

@silverwind
Copy link
Copy Markdown
Member

@silverwind silverwind commented Apr 28, 2026

Fixes #37446.

The job-status resolver in checkJobsOfCurrentRunAttempt only considered needs and job-level concurrency when transitioning jobs out of Blocked. When something drove the resolver against a run blocked solely by workflow-level concurrency — for example, a sibling run in the same group entering the queue and triggering EmitJobsIfReadyByRun — the run's job silently became Waiting while another run still held the concurrency group, and the runner could pick it up, defeating the concurrency guarantee.

The fix bails out of the resolver when the run's latest attempt is still blocked by run-level concurrency. checkRunConcurrency re-evaluates when the holding run finishes.

Covered by a unit test (Test_checkJobsOfCurrentRunAttempt_RunLevelConcurrencyKeepsJobsBlocked in services/actions/job_emitter_test.go) that sets up a Running holder attempt and a Blocked sibling attempt in the same concurrency group directly in the DB, calls checkJobsOfCurrentRunAttempt, and asserts the blocked job stays Blocked. Fails on master, passes with the fix.


This PR was written with the help of Claude Opus 4.7

checkJobsOfCurrentRunAttempt's resolver only considered needs and
job-level concurrency when transitioning jobs out of Blocked. When
something drove the resolver against a run blocked solely by
workflow-level concurrency (for example, a sibling run in the same
group entering the queue and triggering EmitJobsIfReadyByRun), the
run's job silently became Waiting while another run still held the
group, and the runner could pick it up.

Bail out of the resolver when the run's latest attempt is still
blocked by run-level concurrency. checkRunConcurrency re-evaluates
when the holding run finishes.

Fixes go-gitea#37446

Co-Authored-By: Claude (Opus 4.7) <noreply@anthropic.com>
@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Apr 28, 2026
@silverwind silverwind requested a review from Zettat123 April 28, 2026 00:09
@silverwind silverwind added topic/gitea-actions related to the actions of Gitea type/bug backport/v1.26 This PR should be backported to Gitea 1.26 labels Apr 28, 2026
@silverwind silverwind requested a review from Copilot April 28, 2026 00:54
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Prevents the job-status resolver from transitioning jobs out of Blocked when the overall run is still blocked by workflow-level (run-level) concurrency, closing a gap where queued resolver runs could defeat concurrency guarantees.

Changes:

  • Add an early-bail in checkJobsOfCurrentRunAttempt when the latest attempt is still blocked by run-level concurrency.
  • Add an integration test ensuring schedule-triggered runs remain blocked (and don’t emit runnable jobs) while a sibling run holds the concurrency group.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
tests/integration/actions_concurrency_test.go Adds an integration test covering the run-level concurrency-blocked resolver regression scenario.
services/actions/job_emitter.go Bails out of the job resolver when the run is blocked due to run-level concurrency, preventing jobs from becoming runnable prematurely.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/integration/actions_concurrency_test.go Outdated
Comment thread services/actions/job_emitter.go
Comment thread tests/integration/actions_concurrency_test.go Outdated
silverwind and others added 2 commits April 28, 2026 03:29
Per review feedback on go-gitea#37461.

Co-Authored-By: Claude (Opus 4.7) <noreply@anthropic.com>
The Len(blockedRuns, 1) assertion already proves the surviving schedule
run is not Waiting, so the runner could not pick anything up.

Co-Authored-By: Claude (Opus 4.7) <noreply@anthropic.com>
@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Apr 28, 2026
@wxiaoguang
Copy link
Copy Markdown
Contributor

I believe it can be tested in a unit test.

TestScheduleConcurrencyBlockedRunStaysBlocked is extremely slow, and it is not clear about the test details.

@wxiaoguang wxiaoguang marked this pull request as draft April 28, 2026 04:28
@silverwind
Copy link
Copy Markdown
Member Author

TestScheduleConcurrencyBlockedRunStaysBlocked is extremely slow, and it is not clear about the test details.

Took 2.7s locally. Slow yes, but not extremely.

@wxiaoguang
Copy link
Copy Markdown
Contributor

TestScheduleConcurrencyBlockedRunStaysBlocked is extremely slow, and it is not clear about the test details.

Took 2.7s locally. Slow yes, but not extremely.

These tests will be extremely slow in CI, the more added, the slower.

@wxiaoguang
Copy link
Copy Markdown
Contributor

wxiaoguang commented Apr 28, 2026

TestScheduleConcurrencyBlockedRunStaysBlocked is extremely slow, and it is not clear about the test details.

Took 2.7s locally. Slow yes, but not extremely.

These tests will be extremely slow in CI, the more added, the slower.

Share some findings I have got about CI time: Operating a git repo via API/web is slow, due to it needs to execute the Gitea's git hook. It might be slower in CI due to limited resources.

To keep CI fast and optimize the speed: avoid unnecessary git repo operations via Gitea's API or web (avoid the Gitea's git hook) as much as possible

  • Some tests can be clearly written in unit tests
  • Some git repo operation can be done via git fast-import

I think it can save at least many minutes if git fast-import can be correctly used.

@silverwind
Copy link
Copy Markdown
Member Author

Yes, with ~4 times slower CI, we are looking at 12s+, which is borderline.

@wxiaoguang
Copy link
Copy Markdown
Contributor

Another concern is that the integration tests are abused.

Actually, for a function level logic, it should be clearly tested in unit tests, including various edge cases.

Integration test should focus on "the whole thing overall works together", it's difficult to use it to cover edge cases, and usually it is not informative when writing a integration test for a speical case (a lot of unrelated code, maintenance burden)

@silverwind
Copy link
Copy Markdown
Member Author

silverwind commented Apr 28, 2026

Yeah, what can be asserted in a unit test should be. I didn't tell Claude to write integration test, it decided itself. Likely a good pointer to add to AGENTS.md.

My attack points for fast CI are:

Per @wxiaoguang's feedback on go-gitea#37461: the run-level concurrency guard
in checkJobsOfCurrentRunAttempt is function-level logic and is better
covered by a unit test. The unit test sets up a Running holder
attempt and a Blocked sibling attempt in the same concurrency group
directly in the DB, calls checkJobsOfCurrentRunAttempt, and asserts
the blocked job stays Blocked. ~0.3s vs ~3.7s for the integration
version, and no API/git-hook overhead.

Co-Authored-By: Claude (Opus 4.7) <noreply@anthropic.com>
@silverwind
Copy link
Copy Markdown
Member Author

@wxiaoguang good call — replaced the integration test with a unit test in services/actions/job_emitter_test.go (commit dd83ccf). It sets up the Running holder + Blocked sibling attempts in the same concurrency group directly via db.Insert and calls checkJobsOfCurrentRunAttempt on the blocked run. ~0.3s vs ~3.7s for the integration version, no git-hook traffic. Verified it fails on master and passes with the fix.


This response was written with the help of Claude Opus 4.7

@silverwind silverwind marked this pull request as ready for review April 28, 2026 05:04
@bircni bircni requested a review from wxiaoguang May 1, 2026 08:53
@wxiaoguang
Copy link
Copy Markdown
Contributor

The new test looks good to me

@GiteaBot GiteaBot removed the lgtm/need 1 This PR needs approval from one additional maintainer to be merged. label May 3, 2026
@GiteaBot GiteaBot added the lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. label May 3, 2026
@silverwind silverwind added the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label May 4, 2026
@silverwind silverwind enabled auto-merge (squash) May 4, 2026 10:48
@silverwind silverwind merged commit 198ef50 into go-gitea:main May 4, 2026
22 checks passed
@silverwind silverwind deleted the fix-blocked-run-resolver branch May 4, 2026 11:10
@GiteaBot GiteaBot added this to the 1.27.0 milestone May 4, 2026
@GiteaBot GiteaBot removed the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label May 4, 2026
@GiteaBot GiteaBot added the backport/done All backports for this PR have been created label May 4, 2026
lunny pushed a commit that referenced this pull request May 4, 2026
) (#37538)

Backport #37461 by @silverwind

Fixes #37446.

The job-status resolver in `checkJobsOfCurrentRunAttempt` only
considered `needs` and job-level concurrency when transitioning jobs out
of `Blocked`. When something drove the resolver against a run blocked
solely by workflow-level concurrency — for example, a sibling run in the
same group entering the queue and triggering `EmitJobsIfReadyByRun` —
the run's job silently became `Waiting` while another run still held the
concurrency group, and the runner could pick it up, defeating the
concurrency guarantee.

The fix bails out of the resolver when the run's latest attempt is still
blocked by run-level concurrency. `checkRunConcurrency` re-evaluates
when the holding run finishes.

Covered by a unit test
(`Test_checkJobsOfCurrentRunAttempt_RunLevelConcurrencyKeepsJobsBlocked`
in `services/actions/job_emitter_test.go`) that sets up a Running holder
attempt and a Blocked sibling attempt in the same concurrency group
directly in the DB, calls `checkJobsOfCurrentRunAttempt`, and asserts
the blocked job stays `Blocked`. Fails on master, passes with the fix.

---
This PR was written with the help of Claude Opus 4.7

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Claude (Opus 4.7) <noreply@anthropic.com>
silverwind added a commit to McMichalK/gitea that referenced this pull request May 5, 2026
* origin/main: (49 commits)
  ci: lint PR titles with commitlint (go-gitea#37498)
  Make ServeSetHeaders default to download attachment if filename exists (go-gitea#37552)
  fix(actions): validate workflow param to prevent 500 error (go-gitea#37546)
  Fix various problems (go-gitea#37547)
  docs: fix 4 typos in CHANGELOG.md (go-gitea#37549)
  [skip ci] Updated translations via Crowdin
  chore(deps): update action dependencies (go-gitea#37540)
  fix: Fix `nolyfill` for renovate (go-gitea#37537)
  Refactor pull request view (7) (go-gitea#37524)
  Update go js py dependencies (go-gitea#37525)
  Don't unblock run-level-concurrency-blocked runs in the resolver (go-gitea#37461)
  Refactor pull request view (6) (go-gitea#37522)
  Refactor pull request view (5) (go-gitea#37517)
  fix: persist mirror repository metadata (go-gitea#37519)
  fix(packages): use file names for generic web downloads (go-gitea#37514)
  fix: merge autodetect can't close other PRs but only the last one when multiple PRs are pushed at once (go-gitea#37512)
  Fix update branch protection order (go-gitea#37508)
  Refactor "flex-list" to "flex-divided-list" (go-gitea#37505)
  fix: redirect early CLI console logger to stderr (go-gitea#37507)
  Fix mCaptcha broken after Vite migration (go-gitea#37492)
  ...

# Conflicts:
#	templates/repo/diff/box.tmpl
zjjhot added a commit to zjjhot/gitea that referenced this pull request May 6, 2026
* main: (36 commits)
  refactor(deps): migrate from `nektos/act` fork to `gitea/runner` (go-gitea#37557)
  ci: lint PR titles with commitlint (go-gitea#37498)
  Make ServeSetHeaders default to download attachment if filename exists (go-gitea#37552)
  fix(actions): validate workflow param to prevent 500 error (go-gitea#37546)
  Fix various problems (go-gitea#37547)
  docs: fix 4 typos in CHANGELOG.md (go-gitea#37549)
  [skip ci] Updated translations via Crowdin
  chore(deps): update action dependencies (go-gitea#37540)
  fix: Fix `nolyfill` for renovate (go-gitea#37537)
  Refactor pull request view (7) (go-gitea#37524)
  Update go js py dependencies (go-gitea#37525)
  Don't unblock run-level-concurrency-blocked runs in the resolver (go-gitea#37461)
  Refactor pull request view (6) (go-gitea#37522)
  Refactor pull request view (5) (go-gitea#37517)
  fix: persist mirror repository metadata (go-gitea#37519)
  fix(packages): use file names for generic web downloads (go-gitea#37514)
  fix: merge autodetect can't close other PRs but only the last one when multiple PRs are pushed at once (go-gitea#37512)
  Fix update branch protection order (go-gitea#37508)
  Refactor "flex-list" to "flex-divided-list" (go-gitea#37505)
  fix: redirect early CLI console logger to stderr (go-gitea#37507)
  ...
eleboucher pushed a commit to eleboucher/apoci that referenced this pull request May 20, 2026
This PR contains the following updates:

| Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| [code.gitea.io/gitea](https://github.com/go-gitea/gitea) | `v1.26.1` → `v1.26.2` | ![age](https://developer.mend.io/api/mc/badges/age/go/code.gitea.io%2fgitea/v1.26.2?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/code.gitea.io%2fgitea/v1.26.1/v1.26.2?slim=true) |

---

### Release Notes

<details>
<summary>go-gitea/gitea (code.gitea.io/gitea)</summary>

### [`v1.26.2`](https://github.com/go-gitea/gitea/releases/tag/v1.26.2)

[Compare Source](go-gitea/gitea@v1.26.1...v1.26.2)

- SECURITY
  - fix(permissions): Fix reading permission ([#&#8203;37769](go-gitea/gitea#37769))
  - fix(actions): make artifact signature payloads unambiguous ([#&#8203;37707](go-gitea/gitea#37707))
  - fix: Unify public-only token filtering in API queries and repo access checks ([#&#8203;37118](go-gitea/gitea#37118))
  - fix: Add missed token scope checking ([#&#8203;37735](go-gitea/gitea#37735))
  - fix(oauth): bind token exchanges to the original client request ([#&#8203;37704](go-gitea/gitea#37704))
  - fix(oauth): strengthen PKCE validation and refresh token replay protection ([#&#8203;37706](go-gitea/gitea#37706))
  - fix(web): enforce token scopes on raw, media, and attachment downloads ([#&#8203;37698](go-gitea/gitea#37698))
  - fix(security): enforce wiki git writes and LFS token access at request time ([#&#8203;37695](go-gitea/gitea#37695))
  - feat(api): encrypt AWS creds ([#&#8203;37679](go-gitea/gitea#37679))
  - fix(deps): update dependency mermaid to v11.15.0 \[security], add e2e test
  - fix(packages): Add label for private and internal package and fix composor package source permission check ([#&#8203;37610](go-gitea/gitea#37610))
  - fix(git): Fix smart http request scope bug ([#&#8203;37583](go-gitea/gitea#37583))
  - Fix basic auth bug ([#&#8203;37503](go-gitea/gitea#37503))
  - Fix allow maintainer edit permission check ([#&#8203;37479](go-gitea/gitea#37479)) ([#&#8203;37484](go-gitea/gitea#37484))
  - Fix URL sanitization to handle schemeless credentials ([#&#8203;37440](go-gitea/gitea#37440)) ([#&#8203;37471](go-gitea/gitea#37471))
  - Fix attachment Content-Security-Policy ([#&#8203;37455](go-gitea/gitea#37455)) ([#&#8203;37464](go-gitea/gitea#37464))
  - chore(deps): bump go-git/go-git/v5 to 5.19.0 ([#&#8203;37608](go-gitea/gitea#37608))

- BUGFIXES
  - fix(pull): handle empty pull request files view to allow reviews ([#&#8203;37783](go-gitea/gitea#37783))
  - fix(markup): make RenderString never fail ([#&#8203;37779](go-gitea/gitea#37779))
  - fix: add natural sort to sortTreeViewNodes ([#&#8203;37772](go-gitea/gitea#37772))
  - fix: package creation unique conflict ([#&#8203;37774](go-gitea/gitea#37774))
  - fix!: add DEFAULT\_TITLE\_SOURCE setting for pull request title default behavior ([#&#8203;37465](go-gitea/gitea#37465))
  - fix: Allow direct commits for unprotected files with push restrictions ([#&#8203;37657](go-gitea/gitea#37657))
  - fix(actions): wrong assumption that run id always >= job id ([#&#8203;37737](go-gitea/gitea#37737))
  - fix(auth): set User-Agent on avatar fetch and sync avatar on link-account register ([#&#8203;37564](go-gitea/gitea#37564)) ([#&#8203;37588](go-gitea/gitea#37588))
  - fix(actions): deadlock between PrepareRunAndInsert and UpdateTaskByState ([#&#8203;37692](go-gitea/gitea#37692))
  - fix(repo): /generate must sync the branch table for the new repo ([#&#8203;37693](go-gitea/gitea#37693))
  - build: Fix snap build (1.26)
  - fix(actions): run TransferLogs on UpdateLog{Rows:\[], NoMore:true} ([#&#8203;37631](go-gitea/gitea#37631))
  - fix show correct mergebase
  - fix: make clone URL respect public URL detection setting ([#&#8203;37615](go-gitea/gitea#37615))
  - fix: "run as root" check ([#&#8203;37622](go-gitea/gitea#37622))
  - chore(deps): update dependency go to v1.26.3 ([#&#8203;37601](go-gitea/gitea#37601))
  - Compare dropdown fails when selecting branch with no common merge-base ([#&#8203;37470](go-gitea/gitea#37470))
  - fix: treat email addresses case-insensitively ([#&#8203;37600](go-gitea/gitea#37600))
  - fix(actions): fix blank lines after ::endgroup:: ([#&#8203;37597](go-gitea/gitea#37597))
  - fix(actions): report individual step status in workflow job API response ([#&#8203;37592](go-gitea/gitea#37592))
  - fix: Invalid UTF-8 commit messages in JSON API responses ([#&#8203;37542](go-gitea/gitea#37542))
  - fix: use consistent GetUser family functions ([#&#8203;37553](go-gitea/gitea#37553))
  - fix(api): return 409 message instead of empty JSON for wrong commit id ([#&#8203;37572](go-gitea/gitea#37572))
  - fix(actions): prevent panic when workflow contains null jobs ([#&#8203;37570](go-gitea/gitea#37570))
  - Make ServeSetHeaders default to download attachment if filename exists ([#&#8203;37552](go-gitea/gitea#37552)) ([#&#8203;37555](go-gitea/gitea#37555))
  - Fix(actions): validate workflow param to prevent 500 error ([#&#8203;37546](go-gitea/gitea#37546)) ([#&#8203;37554](go-gitea/gitea#37554))
  - Don't unblock run-level-concurrency-blocked runs in the resolver ([#&#8203;37461](go-gitea/gitea#37461)) ([#&#8203;37538](go-gitea/gitea#37538))
  - Fix(packages): use file names for generic web downloads ([#&#8203;37514](go-gitea/gitea#37514)) ([#&#8203;37520](go-gitea/gitea#37520))
  - Fix merge autodetect can't close other PRs but only the last one when multiple PRs are pushed at once ([#&#8203;37512](go-gitea/gitea#37512)) ([#&#8203;37516](go-gitea/gitea#37516))
  - Fix update branch protection order ([#&#8203;37508](go-gitea/gitea#37508)) ([#&#8203;37513](go-gitea/gitea#37513))
  - Fix mCaptcha broken after Vite migration ([#&#8203;37492](go-gitea/gitea#37492)) ([#&#8203;37509](go-gitea/gitea#37509))
  - Fix review submission from single-commit PR view ([#&#8203;37475](go-gitea/gitea#37475)) ([#&#8203;37485](go-gitea/gitea#37485))
  - Fix scheduled action panic with null event payload ([#&#8203;37459](go-gitea/gitea#37459)) ([#&#8203;37466](go-gitea/gitea#37466))
  - Make GetPossibleUserByID can handle deleted user ([#&#8203;37430](go-gitea/gitea#37430)) ([#&#8203;37431](go-gitea/gitea#37431))
  - Remove excessive quote from terraform instructions ([#&#8203;37424](go-gitea/gitea#37424)) ([#&#8203;37426](go-gitea/gitea#37426))
  - Fix color regressions, add `priority` color ([#&#8203;37417](go-gitea/gitea#37417)) ([#&#8203;37421](go-gitea/gitea#37421))

- MISC
  - Add CurrentURL template variable back ([#&#8203;37444](go-gitea/gitea#37444)) ([#&#8203;37449](go-gitea/gitea#37449))

Instances on **[Gitea Cloud](https://cloud.gitea.com)** will be automatically upgraded to this version during the specified maintenance window.

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMDEuMSIsInVwZGF0ZWRJblZlciI6IjQzLjEwMS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJ0eXBlL3BhdGNoIl19-->

Reviewed-on: https://git.erwanleboucher.dev/eleboucher/apoci/pulls/47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport/done All backports for this PR have been created backport/v1.26 This PR should be backported to Gitea 1.26 lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. topic/gitea-actions related to the actions of Gitea type/bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky test TestScheduleConcurrency

6 participants