githubv4.Query: Resource not accessible by integration

[fmarier]

I'm using the default GitHub Action workflow, though with tags instead of commit hashes in https://github.com/brave/simplepadding/blob/master/.github/workflows/scorecards.yml and I'm seeing this error now that I've upgraded to v2:

Error: one or more checks had a runtime error: internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration

Is there a permission missing from the example workflow?

I see it's also been reported in ossf/scorecard#1097, but I'm not sure I'm seeing the same issue since it was working fine before I upgraded from v1.1.2 to v2.

[azeemshaikh38]

Thanks for the report. I can reproduce this on my end. Seems like switching on specific branch-protection settings (like Enable signed commits) can make the action throw up this error. I haven't tested exhaustively to see what all settings lead to this error, but for now I'll try to send out a fix to ensure Scorecard exists gracefully in such cases.

[fmarier]

Here are the branch protection settings I have enabled in the above repo:

[nijel]

It happens for https://github.com/WeblateOrg/weblate/ as well, the only branch protection we have enabled is "Require status checks to pass before merging". Failing workflow run: https://github.com/WeblateOrg/weblate/runs/8262284657

PS: It also started to happen with upgrade to version 2.

[dirien]

Not adding additional noise, but happens in https://github.com/dirien/minectl-sdk too.

https://github.com/dirien/minectl-sdk/runs/8267086312?check_suite_focus=true

Only change is the update to 2.0.0.

[dirien]

I compared a the output from cosign (Ping @cpanato) and my and see following difference by same pipline file:

cosign:

Repository: sigstore/cosign
Fork repository: false
Private repository: false
Publication enabled: false
Format: json
Policy file: 
Default branch: main
Using payload from: results.json

the failed one:

Private repository: false
Publication enabled: false
Format: sarif
Policy file: /policy.yml
Default branch: main
Error: one or more checks had a runtime error: internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
2022/09/09 09:35:31 error during command execution: one or more checks had a runtime error: internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration

Why is there a /policy.yaml?

[azeemshaikh38]

Found the issue - seems to have started after we upgraded to scorecard from v4.5.0 -> v4.6.0. Confirmed that #894 fixes this. Will cut a release for v2.0.1 with this PR to fix the breakages while I investigate the real issue.

[varunsh-coder]

@azeemshaikh38 I think the root cause is due to this line
https://github.com/ossf/scorecard/blob/8de962e91d860f81916dbeb3b9a1a943fcf8a80f/cmd/root.go#L163

When Scorecard is run with the GITHUB_TOKEN, some checks (e.g. for branch protection) are expected to error out. Due to the above change, that causes the overall run to fail.

You can search for this string one or more checks had a runtime error in the output for the Action runs that failed.
e.g. https://github.com/WeblateOrg/weblate/runs/8262284657#step:4:168

[azeemshaikh38]

fyi - https://github.com/ossf/scorecard-action/releases/tag/v2.0.1

@varunsh-coder - good point, I think that's it. Will look into it.

[laurentsimon]

strange that the policy file is not set for the cosign repo. This should be set, otherwise we will be reporting results in the scanning dashboard that we should not.

[dirien]

strange that the policy file is not set for the cosign repo. This should be set, otherwise we will be reporting results in the scanning dashboard that we should not.

Works for me now! https://github.com/dirien/minectl-sdk/runs/8274045473?check_suite_focus=true

@laurentsimon now I have the policy.yaml also not anymore like cosign!

Repository: dirien/minectl-sdk
Fork repository: false
Private repository: false
Publication enabled: false
Format: json
Policy file: 
Default branch: main
Using payload from: results.json
Generating ephemeral keys...

[spencerschrock]

@azeemshaikh38 I think the root cause is due to this line https://github.com/ossf/scorecard/blob/8de962e91d860f81916dbeb3b9a1a943fcf8a80f/cmd/root.go#L163

When Scorecard is run with the GITHUB_TOKEN, some checks (e.g. for branch protection) are expected to error out. Due to the above change, that causes the overall run to fail.

You can search for this string one or more checks had a runtime error in the output for the Action runs that failed. e.g. https://github.com/WeblateOrg/weblate/runs/8262284657#step:4:168

Yes I made the change as part of ossf/scorecard#2124, unaware it would spill over to scorecard-action. Fixing it would probably involve using a specific error here that we check for using errors.Is here and here (or better yet, doing something in the entrypoint to wrap it for all uses)