Skip to content

ci: add nightly failure issue alerts#3293

Merged
serrrfirat merged 2 commits into
mainfrom
ci/nightly-alert-issues
May 6, 2026
Merged

ci: add nightly failure issue alerts#3293
serrrfirat merged 2 commits into
mainfrom
ci/nightly-alert-issues

Conversation

@serrrfirat
Copy link
Copy Markdown
Collaborator

Summary

  • add a reusable nightly alert script that dedupes GitHub issue alerts by workflow
  • extract failed job metadata and high-signal log snippets into the issue body
  • wire scheduled E2E and Nightly Deep CI runs to create/update failure issues and close them on recovery

Verification

  • bash -n .github/scripts/nightly-alert-issue.sh .github/scripts/nightly-alert-issue.test.sh
  • .github/scripts/nightly-alert-issue.test.sh
  • git diff --check
  • ruby -e 'require "yaml"; ARGV.each { |f| YAML.load_file(f); puts "parsed #{f}" }' .github/workflows/e2e.yml .github/workflows/nightly-deep-ci.yml

Notes

  • Alert jobs only run for scheduled workflow runs.
  • Repeated failures update the existing issue body instead of adding comments.
  • The next scheduled success closes the open alert issue with a recovery note.

@github-actions github-actions Bot added scope: ci CI/CD workflows size: L 200-499 changed lines risk: medium Business logic, config, or moderate-risk modules contributor: core 20+ merged PRs labels May 6, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a bash script and a corresponding test suite to automate the management of GitHub issues for nightly workflow alerts. The script handles creating, updating, or closing issues based on workflow results and includes logic to extract failure excerpts from logs. Feedback suggests increasing the search limit and using quoted titles in the gh issue list command to more reliably identify existing alert issues and prevent duplicates in repositories with many open issues.

Comment on lines +59 to +64
gh issue list \
--repo "$REPO" \
--state open \
--search "${title} in:title" \
--json number,title \
--jq 'map(select(.title == env.ALERT_ISSUE_TITLE))[0].number // empty'
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The default limit for gh issue list is 30. In repositories with a high volume of open issues, the existing alert issue might be missed if it was created some time ago and is no longer among the 30 most recently created or updated issues. Increasing the limit (e.g., to 100) and using a quoted search term for the title will improve the reliability of finding the correct issue to update, preventing duplicate alerts.

Suggested change
gh issue list \
--repo "$REPO" \
--state open \
--search "${title} in:title" \
--json number,title \
--jq 'map(select(.title == env.ALERT_ISSUE_TITLE))[0].number // empty'
gh issue list \
--repo "$REPO" \
--state open \
--limit 100 \
--search "\"$title\" in:title" \
--json number,title \
--jq 'map(select(.title == env.ALERT_ISSUE_TITLE))[0].number // empty'

@serrrfirat serrrfirat merged commit cde13e5 into main May 6, 2026
14 checks passed
@serrrfirat serrrfirat deleted the ci/nightly-alert-issues branch May 6, 2026 12:01
This was referenced May 7, 2026
@ironclaw-ci ironclaw-ci Bot mentioned this pull request May 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor: core 20+ merged PRs risk: medium Business logic, config, or moderate-risk modules scope: ci CI/CD workflows size: L 200-499 changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant