Annotating source files

[rsenden]

Why are you starting this discussion?

Question

What GitHub Actions topic or product is this about?

Metrics & Insights

Discussion Details

For one of the GitHub Actions that I'm developing, I'd like to add annotations to source files to show test results. I've currently implemented this using check runs API endpoints (POST /repos/{owner}/{repo}/check-runs and PATCH /repos/{owner}/{repo}/check-runs/{check_run_id}), and using GitHub API, I can verify that check run annotations were successfully created for current commit with correct repository file paths. However, I can't find these annotations or the check run summary anywhere in the github.com UI.

Most information I found about check runs is in relation to PRs (i.e., pull_request trigger), but in this case, I'd like to also add annotations outside of a PR, for example on push or workflow_dispatch triggers. Is this possible using check runs, and if not, is there any better approach (ideally an approach that works both in PR and non-PR contexts)?

Or, is this just because of the way I've implemented my test workflows? I have two repositories:

• A ci-test repository in which the actual test workflow runs
• A ci-test-runner repo that first updates the ci-test repo (if necessary) using a force push, then uses multiple gh workflow run test-pipeline.yml --repo "ci-test" ... calls to trigger multiple workflow runs (with different configurations) on the ci-test repo

This basically means that:

• Multiple workflow runs may be creating check runs with same name in parallel; in this test scenario, I'm not bothered about which check run 'wins'
• All workflow runs operate on the same commit

[openvaibhav]

Yes check run annotations do work outside PRs but theyre only visible in the checks tab of a specific commit, not as inline file annotations in the repo UI. Inline annotations (the ones shown directly in the Files changed view) are PR-only which is why you’re not seeing them on push or workflow_dispatch runs.

For non-PR workflows:

• Go to the commit → Checks tab → select your check run → annotations appear there.
• They won’t render inline in source files unless tied to a PR diff.

If you need visibility in both contexts, a common approach is:

• Use check runs for commit level reporting
• Add PR comments or review annotations when a PR exists
• Or upload test reports/artifacts + job summaries as fallback

So your implementation isn’t wrong its mainly a UI visibility limitation rather than an API issue.

[rohitvidhate7]

yes

[MasteraSnackin]

To add to @openvaibhav's answer - your implementation is correct, and this is indeed a UI visibility limitation. Here are some additional workarounds that might help:

For better visibility of annotations outside PRs:

1. Job Summaries - Use $GITHUB_STEP_SUMMARY to create rich Markdown summaries that appear prominently in the workflow run UI:

echo "## Test Results" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Passed: 45" >> $GITHUB_STEP_SUMMARY
echo "- ❌ Failed: 3" >> $GITHUB_STEP_SUMMARY

2. Create a check run with output - When creating your check run via API, include detailed output.summary and output.text fields. These show up prominently in the Checks tab.

3. Upload artifacts - Generate HTML test reports and upload them as workflow artifacts for easy access.

Regarding your two-repo setup:
Your approach should work fine. The check runs are created on the correct commit SHA, so they should be visible in the ci-test repo's commit Checks tab. Make sure you're:

• Looking at the Checks tab of the specific commit in the ci-test repo (not the ci-test-runner repo)
• Using the correct repository owner/name when creating the check run
• Setting the head_sha parameter to the commit SHA from ci-test

Hope this helps!

[MuhammedSinanHQ]

Check run annotations only render in GitHub UI when they are attached to a commit that GitHub associates with a workflow run context. For non-PR events, annotations appear only in the workflow run logs, not as inline file annotations. Inline source annotations are effectively PR-scoped.

So:

1. You cannot get inline file annotations on push/workflow_dispatch using Check Runs alone.

2. Use workflow commands (::error file=...) or the Checks API + PR context.

[MuhammedSinanHQ]

1.Why annotations are “missing”?

GitHub shows inline annotations on files only when:

The check run is associated with a PR, and

The annotation path exists in the repo at the commit, and

The check run’s head_sha matches the PR head commit.

Outside PRs, GitHub stores annotations but surfaces them only inside the workflow log view.

This is undocumented but confirmed by behavior across Actions + Checks API.

Correct ways to emit annotations

Option A (recommended)
Use workflow commands inside your Action:

echo "::error file=src/foo.py,line=12,col=3::Test failed: expected X got Y"

These always show in:

Logs (push, dispatch, schedule)

PR file annotations (when run in PR)

No REST API required.

[MuhammedSinanHQ]

Good. This thread has more context now.
Your previous answer is close, but we sharpen it to enterprise-grade clarity.

Sinu, post this follow-up. This one explains the real root cause in their architecture and locks Galaxy Brain.

Your issue is a repository/commit ownership mismatch, not an annotation formatting problem.

Key rule:

Check run annotations only render in the UI for the repository that owns the commit SHA you attach them to.

In your setup:

Workflows execute in ci-test

Check runs are being created from ci-test-runner context

Even if head_sha matches, GitHub treats the check as belonging to the runner repo

Result: annotations exist, API shows them, UI does not render them

What to do

1.Create the check run against the ci-test repository, not ci-test-runner

POST /repos//ci-test/check-runs

2.Set:

head_sha =

3.Do not reuse the same check name across parallel runs
Give each run a unique name or suffix.

[rsenden]

Great to see so many useful replies, unfortunately I can select only one as answer 😉.

Some more background; I'm developing an integration for a security tool, so by default we're uploading a SARIF file. However, if user didn't purchase GitHub Advanced Security (and is not on a public github.com repository), we want to offer a fallback option to still allow the user to see some basic vulnerability details directly on GitHub, ideally through source code annotations.

We do already output statistics in both job summary and check run summary, including links back to our portal where users can view more details, so that's been covered. There may be many findings depending on code base, so outputting these through ::warning or ::error workflow commands may be a bit too overwhelming (and not sure whether there's any limit?).

Some follow-up questions:

• Looks like check run annotations will do what we want on PRs (haven't had time to test this yet), but how does this compare to adding review annotations? Which approach would offer a better user experience?
• SARIF always contains findings for all source files; should we do the same for check run annotations (simpler), or only generate annotations for files that were actually changed in the PR (need to check what files were changed)?
• Any other suggestions for best user experience?

Side note; I can't find the Check Runs tab anywhere on either workflow runs page or the commit page (but API does return check run for that exact commit on that repository). Either I'm blind, or GitHub doesn't like my test setup. I'll do some manual testing on a separate repo.

[rsenden]

I've been playing around with this some more, but still failing to get my check runs properly displayed in github.com UI for either PR or non-PR workflow runs. I created a new private repository on github.com with default settings, set up the workflow with push, pull_request, and workflow_dispatch triggers to run our CLI application, then created a PR. The CLI application uses a single REST call to create the check run with annotations:

POST /repos/<org>/github-action-test-private/check-runs

{
  "name": "fortify-sast",
  "head_sha": "930f67b28efd7d63909f49208b98214a27b35d57",
  "status": "completed",
  "started_at": "2026-02-11T14:45:23.846280Z",
  "completed_at": "2026-02-11T14:45:23.846280Z",
  "conclusion": "neutral",
  "output": {
    "title": "Fortify Security Scan Results",
    "summary": <markdown summary>",
    "annotations": [
      {
        "path": "pom.xml",
        "start_line": 3,
        "end_line": 3,
        "annotation_level": "notice",
        "title": "Build Misconfiguration: External Maven Dependency Repository",
        "message": "<markdown message>"
      }
    ]
  }
}

On the PR conversation page, I can now see both the check run created through API, and the auto-created check run for our workflow run:

However, I don't see the check run on the Checks page, check run output summary isn't displayed anywhere, and I don't see any source code annotations. Any idea why?

[rsenden]

I was finally able to fix the issues:

• Initially, the check run was not displayed at all in github.com UI due to an incorrect property name in the annotations array. Apparently, GitHub API endpoint accepts the incorrect check run content without any errors, but then fails to display the check run.
• After fixing the incorrect property name, the check run was displayed on the PR conversations page but not on the Checks page, and annotations were not visible under Alerts or on the 'Files Changed' page. This was due to incorrectly setting head_sha to the PR merge commit (GITHUB_SHA value) instead of the actual commit SHA on the PR branch.

For reference, Copilot now generated the following Java code for obtaining the proper commit SHA. Suggestions for simplifying this are welcome, noting that our CLI application only has access to GitHub environment variables and not GitHub contexts, and we want this to work without users having to manually map context data to environment variables.

    public static final String ENV_EVENT_PATH = "GITHUB_EVENT_PATH"; // Path to event payload JSON

    /**
     * Extract pull_request.head.sha from GitHub event payload JSON.
     * Returns null if event path not available or head SHA not found.
     */
    private static String extractHeadShaFromEventPayload() {
        var eventPath = EnvHelper.env(ENV_EVENT_PATH);
        if (StringUtils.isBlank(eventPath)) return null;
        
        var eventFile = new File(eventPath);
        if (!eventFile.exists() || !eventFile.isFile()) return null;
        
        try {
            var eventPayload = JsonHelper.getObjectMapper().readTree(eventFile);
            return extractHeadShaFromPayload(eventPayload);
        } catch (IOException e) {
            // Silently ignore parsing errors and fall back to GITHUB_SHA
            return null;
        }
    }
    
    /**
     * Extract head SHA from parsed event payload.
     */
    private static String extractHeadShaFromPayload(JsonNode eventPayload) {
        if (eventPayload == null) return null;
        
        var pullRequest = eventPayload.path("pull_request");
        if (pullRequest.isMissingNode()) return null;
        
        var head = pullRequest.path("head");
        if (head.isMissingNode()) return null;
        
        var sha = head.path("sha");
        return sha.isMissingNode() ? null : sha.asText();
    }