Skip to content

ci: a label-driven Jibril release gate covering every recorded pnpm failure mode - #43

Merged
jadoonf merged 7 commits into
mainfrom
devin/1788889870-jibril-release-gate
Sep 9, 2026
Merged

jadoonf merged 7 commits into
mainfrom
devin/1788889870-jibril-release-gate

Conversation

@jadoonf

@jadoonf jadoonf commented Sep 8, 2026 •

Copy link
Copy Markdown

Summary

One gate for pnpm's Garnet health, on this fork only. It runs on every pull request labelled garnet-release-testing (and on a schedule for each new Jibril tag), reproduces the exact shape that has kept upstream dark since 2026-09-02 (a reusable workflow called with the $/ self-repository form, on the blacksmith-8vcpu-ubuntu-2404 runner class), and passes only when the whole evidence chain lands. A green job is not a pass.

Ten legs, evaluated by .github/scripts/garnet-release-gate-verify.sh:

leg pass
L1 startup jibril unit active, /var/run/jibril/jibril.status.json with .status = ok and GitHub step discovery ok
L2 profile control plane returns a profile for this run, job and attempt (403/404 polled as "not yet", other codes are errors)
L3 workload registry.npmjs.org in the profile's egress, attributed to the step Workload with egress
L4 App comment garnet-runtime-review[bot] comment bound to the PR head, final marker present. If pending, the leg names the cause it can see: no profile, or a merge commit GitHub regenerated after the sensor recorded
L5 permalink public HTML and API forms both HTTP 200
L6 integration shape ci.yml/test.yml/release.yml as pnpm reviewers read them: full-SHA action pin, explicit api_token, no id-token: write, secrets: inherit or new pull-requests: write added by the PR. macOS steps and the Dependabot empty-token case are printed as disclosures
L7 integration run pnpm's own TS CI job on this head: profile present for the instrumented cell, or the leg says why not
L8 reviewers zizmor (regular and pedantic) on the workflow files the PR touched, plus zero unresolved bot review threads under .github/
L9 timing sensor start time and workload time against a no-sensor baseline job, overhead budget 120 s
L10 coverage instrumented vs uninstrumented CI matrix cells, stated plainly (today 1 of 6)

Every leg maps to a row in .github/garnet-gate/FAILURE_LEDGER.md: every Garnet failure seen on pnpm so far, where it was seen, and the check that catches it. A new failure gets a ledger row and a check in the same PR.

Each run writes a JSON record (base, head, merge-at-run and current merge SHA, action SHA, requested and resolved Jibril version, profile id, comparison scope, timing, coverage, every leg's state) as an artifact, renders it as the Step Summary, and keeps one comment per tag on the PR, updated in place.

Standing rule this PR encodes: every pull request on this fork that tests Garnet product or release work for pnpm carries garnet-release-testing. Unlabelled test PRs have not been accepted. The label is on every open Garnet test PR now.

The caller job in garnet-jibril-release-gate.yml is named reproduce, the same id as the job it calls, because ci.yml's test calls test.yml's test. Jibril resolves GITHUB_JOB against the caller file; with differing ids the status file reports step discovery degraded and the egress lands on an unknown step (ledger F21).

What this PR does not do: touch ci.yml or any upstream-facing workflow, add id-token: write, change permissions beyond the gate's own pull-requests: write for its comment and actions: read for listing the head's TS CI run, or claim coverage for macOS, Windows, fork PRs or Dependabot runs.

Expected diagnostics:

  • uses: $/.github/workflows/garnet-jibril-release-gate-job.yml is deliberate. It is the production failure shape. actionlint and Greptile flag it; GitHub accepts it, and upstream ci.yml runs this form on every PR.
  • actionlint does not know the blacksmith-8vcpu-ubuntu-2404 label; that is the runner class upstream uses for the Garnet-enabled cell.
  • zizmor: no findings at regular or pedantic.

Live results on this PR, one comment per tag:

  • v2.17.0-rc.9: FAIL on L1, L2, L4. The sensor exits on the $/ reference; no profile, comment pending.
  • v2.17.0-rc.10 (run 34309875655): the sensor stays active, the profile uploads, the App comment finalizes, the permalink answers 200, overhead 15 s. FAIL on L1 (steps.status=degraded, job reproduce not found) and L3 (egress on step <unknown>), both caused by the caller job id gate in the gate itself, and L7 read "no TS CI run" because the verify token had no actions: read. This push fixes the gate for both; the next labelled run is the reading that counts.

The verifier was also run locally against a known good fork run (33660546962) → PASS.

Related: garnet-org/jibril#754 (the fix), garnet-org/action#147 (api_token-first auth and start-failure disclosure).

Squash Commit Body

Add a Jibril release gate that runs on every pull request labelled
garnet-release-testing and on a schedule for each new Jibril tag. It
reproduces the self-repository reusable workflow reference
(uses: $/.github/workflows/...) on the Blacksmith runner class upstream
uses, and passes only on the full evidence chain.

Ten legs: sensor active with a status file that reports GitHub step
discovery ok; profile uploaded for this run, job and attempt; workload
egress attributed to the named step; Runtime Review App comment
finalized against the PR head, with the visible cause named when it
stays pending; public permalink 200 in HTML and API form; integration
shape as pnpm reviewers read it (full-SHA pin, explicit token, no new
permissions); pnpm's own TS CI run on the head; zizmor and unresolved
bot review threads on touched workflow files; sensor overhead against a
no-sensor baseline; instrumented share of the CI matrix.

Each leg maps to a row in .github/garnet-gate/FAILURE_LEDGER.md, which
lists every Garnet failure seen on pnpm and the check that catches it.
Disclosures a machine cannot decide (macOS steps, Dependabot runs, fork
pull requests) print under the verdict instead of failing it.

The reproduce and baseline jobs never fail on their own; the verify
job writes the record, Step Summary and per-tag PR comment first, then
fails only on a non-PASS verdict. The $/ form and the Blacksmith label
diagnostics from actionlint are expected.

The caller job id matches the callee job id, as ci.yml's test calls
test.yml's test: Jibril resolves GITHUB_JOB against the caller file and
reports step discovery degraded otherwise. The verify job holds
actions: read so the integration leg can list the head's TS CI run; a
listing error fails the leg instead of reading as an absent run.

Checklist

  • Added or updated tests. (The gate is the test; the verifier was exercised locally against a known passing run, a missing-profile run, and a regenerated-merge-commit PR.)
  • Live FAIL on rc.9 and rc.10 recorded with full records; rc.10 reaches profile, App comment and permalink.
  • Live PASS on all ten legs pending the next labelled run against v2.17.0-rc.10.

Link to Devin session: https://app.devin.ai/sessions/3ab2e708c38e478dad8aa472a81c8fcc
Open in Devin Desktop: https://app.devin.ai/desktop/session/3ab2e708c38e478dad8aa472a81c8fcc?variant=devin
Requested by: @jadoonf


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Runtime evidence (Garnet)

Kernel-recorded execution record for head a4abb18faa6447ed6c22b7fb3a8285eae3efae08, mirrored verbatim from
the sticky Garnet Runtime Review comment on this PR so reviewers that read only
the description ground in the same bytes. Facts only. Judgment stays with the
reviewer. Cite grounded findings as:

Runtime evidence (Garnet, head a4abb18): <execution chain> → <destination> (<workflow>/<job>) —

Execution record (verbatim mirror)

Execution Profiles recorded for 1 job, triggered by a4abb18

1 job unchanged · compared with f76a862
recorded at the kernel by Garnet · 2026-09-09 04:38 UTC

Garnet Jibril Release Gate / reproduce ↗ · 2 destinations
@@ f76a862 (previous) vs a4abb18 (current) @@
  Runner.Worker
  └─ bash
     └─ node (step: "Start the clock")
        ├─ ○ localhost (dns resolver)
        └─ ○ registry.npmjs[.]org
 
- systemd (runner background · −1)
- └─ systemd-resolved
-    └─ ○ 8.8.8.8

View this job's Execution Profile in Garnet →


💡 How to read this
Runner.Worker          ← process on a path
└─ npm
   └─ node             ← process that acted
      └─ ○ npmjs[.]org ← observed action

follow a path downward to see what ran and what it did — each path to an observed action is an execution chain

names on the path = processes · ○ = observed action · (…) = context

+ only in the current record · − only in the previous record · runner background = the runner's infrastructure, not your workflow

Workload behaviour (Garnet)

Garnet Jibril Release Gate / reproduce · head 1a06ba6 · 3 workload chains · 1 runner-platform chains

Workload behaviour vs ba67148: +0 −8 destinations
− github[.]com
− codeload.github[.]com
− nodejs[.]org
− node-registry.bit[.]cloud
− pnpm[.]io
− release-assets.githubusercontent[.]com
− unofficial-builds.nodejs[.]org
− npm.jsr[.]io
runner platform: +0 −20 destinations · not workload behaviour

3 workload chains · 2 destinations at head, 8 gone
destination chains Δ vs base reached by
localhost 2 19 → 2 node, sudo
registry.npmjs[.]org 1 72 → 1 node
codeload.github[.]com 0 gone node
github[.]com 0 gone git-remote-http, node
node-registry.bit[.]cloud 0 gone node
nodejs[.]org 0 gone node
npm.jsr[.]io 0 gone node
pnpm[.]io 0 gone node
release-assets.githubusercontent[.]com 0 gone node
unofficial-builds.nodejs[.]org 0 gone node
runner platform · 1 chains · 1 destinations at head, 20 gone · no recorded workflow step
destination chains Δ vs base reached by
localhost 1 7 → 1 sudo
140.82.112.22 0 gone nginx
140.82.113.21 0 gone nginx
140.82.114.21 0 gone nginx
169.254.169.123 0 gone chronyd
185.125.190.56 0 gone chronyd
185.125.190.57 0 gone chronyd
185.125.190.58 0 gone chronyd
192.168.127.1 0 gone blacksmithd, sudo
198.137.202.56 0 gone chronyd
23.143.196.204 0 gone chronyd
23.186.168.128 0 gone chronyd
5.161.94.12 0 gone chronyd
8.8.8.8 0 gone nginx, systemd-resolve
91.189.91.157 0 gone chronyd
api.snapcraft[.]io 0 gone snapd
cloudfront.cdn.snapcraftcontent[.]com 0 gone snapd
nodejs[.]org 0 gone pnpm
productionresultssa15.blob.core[.]windows 0 gone node
registry.npmjs[.]org 0 gone pnpm
unofficial-builds.nodejs[.]org 0 gone pnpm

recorded at the kernel by Garnet · profile view →

Summary by CodeRabbit

  • New Features
    • Added automated release gating for Garnet Jibril releases across scheduled, manual, and labeled pull request runs.
    • Added validation of startup health, workload attribution, generated profiles, public links, security checks, and performance overhead.
    • Added detailed gate results, evidence records, workflow summaries, and tracking comments.
    • Added a failure ledger documenting known release-gate failure scenarios and their corresponding checks.

Greptile Summary

This PR adds a label-driven and scheduled Jibril release gate, a ten-leg verifier, persisted evidence artifacts and comments, and a ledger mapping recorded pnpm/Garnet failures to checks.

  • Runs a candidate Jibril release through matching instrumented and baseline Blacksmith jobs.
  • Verifies startup, profile upload, workload attribution, Runtime Review output, links, integration configuration and execution, reviewer state, timing, and coverage.
  • Runtime evidence for head a4abb18 confirms the gate’s reproduce job uploaded profile 01a08476-307d-7456-9c96-4bcf02108314 and reached registry.npmjs.org; the captured chain attributed that traffic to “Start the clock,” providing relevant evidence for the attribution failure mode this gate is designed to detect.
  • Two verifier fail-open paths still allow PASS without completing required L7 or L8 checks.

Confidence Score: 3/5

The PR is not yet safe to merge because required integration-run and reviewer checks can fail open while the release gate reports PASS.

L7 exits its polling loop on the first empty workflow-run response and records no failure, while L8 maps scanner and review-thread query failures to clean results; either behavior breaks the gate’s core contract that PASS represents a complete evidence chain. All previous review findings are no longer outstanding: three were fixed and resolved, and the $/ reference finding was correctly withdrawn after live verification.

Files Needing Attention: .github/scripts/garnet-release-gate-verify.sh

Important Files Changed

Filename Overview
.github/scripts/garnet-release-gate-verify.sh Implements all ten evidence legs, but L7 and L8 contain fail-open paths that can produce PASS without completing required checks.
.github/workflows/garnet-jibril-release-gate-job.yml Defines baseline, instrumented, and verification jobs with scoped permissions, explicit secret propagation, artifacts, and verdict reporting.
.github/workflows/garnet-jibril-release-gate.yml Adds label, schedule, and manual orchestration with per-ref concurrency and per-tag tracking.
.github/garnet-gate/FAILURE_LEDGER.md Documents recorded Garnet failure modes and maps each to its intended gate leg.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  T[Schedule, dispatch, or labeled PR] --> D[Detect release tag]
  D --> B[Baseline workload]
  D --> R[Instrumented reproduce job]
  B --> V[Ten-leg verifier]
  R --> V
  V --> E[JSON artifact and step summary]
  V --> C[Per-tag PR comment]
  V -->|All required evidence established| P[PASS]
  V -->|Any failing leg| F[FAIL]
Loading

Reviews (2): Last reviewed commit: "ci: record the step numbering skew behin..." | Re-trigger Greptile

Greptile also left 2 inline comments on this PR.

@devin-ai-integration

Copy link
Copy Markdown

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@coderabbitai

coderabbitai Bot commented Sep 8, 2026 •

Copy link
Copy Markdown

Review Change StackReview Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 017c1751-33e6-48c3-b2b4-0a289548cc68

📥 Commits

Reviewing files that changed from the base of the PR and between 0d0bbd1 and a4abb18.

📒 Files selected for processing (4)
  • .github/garnet-gate/FAILURE_LEDGER.md
  • .github/scripts/garnet-release-gate-verify.sh
  • .github/workflows/garnet-jibril-release-gate-job.yml
  • .github/workflows/garnet-jibril-release-gate.yml

📝 Walkthrough

Walkthrough

Adds a Garnet Jibril release gate with release selection, sensor reproduction, baseline timing, evidence verification, artifact recording, tracking comments, and a failure ledger covering 25 failure modes.

Changes

Garnet release gate

Layer / File(s) Summary
Release selection and dispatch
.github/workflows/garnet-jibril-release-gate.yml
The workflow runs on schedules, manual dispatches, and labeled pull requests. It selects a Jibril tag, skips completed scheduled gates, and calls the reusable gate workflow.
Reproduction and verification jobs
.github/workflows/garnet-jibril-release-gate-job.yml
The reusable workflow times a baseline workload, runs the sensor workload, checks service status, executes verification, uploads records, updates the tracking comment, and fails when the verdict is not PASS.
Evidence verification and failure mapping
.github/scripts/garnet-release-gate-verify.sh, .github/garnet-gate/FAILURE_LEDGER.md
The script evaluates ten verification legs, queries Garnet and GitHub evidence, writes JSON and Markdown records, and emits workflow outputs. The ledger maps failures F1–F25 to gate legs.

Estimated code review effort: 4 (Complex) | ~60 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant GateWorkflow
  participant ReusableGateJob
  participant GarnetControlPlane
  participant VerifyScript
  participant GitHubAPI
  GateWorkflow->>ReusableGateJob: dispatch selected Jibril tag
  ReusableGateJob->>ReusableGateJob: run baseline and sensor workload
  ReusableGateJob->>GarnetControlPlane: produce profile and status evidence
  ReusableGateJob->>VerifyScript: pass job outputs and baseline timing
  VerifyScript->>GarnetControlPlane: verify profile and attribution
  VerifyScript->>GitHubAPI: verify comments, workflows, and review threads
  VerifyScript->>ReusableGateJob: return verdict and records
  ReusableGateJob->>GitHubAPI: upload artifacts and update tracking comment
Loading

Suggested reviewers: zkochan

✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch devin/1788889870-jibril-release-gate

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

github-advanced-security[bot]

This comment was marked as resolved.

greptile-apps[bot]

This comment was marked as resolved.

…nt and permalink

The gate used to stop at the systemd unit. It now runs a second job after the sensor's
post step and checks the whole evidence chain: profile uploaded to the control plane for
this run and attempt, the workload destination attributed to the workload step, the
Runtime Review App comment final for the PR head, and the profile-selected public
permalink answering 200. Each run writes one record (base, head, merge and profile SHAs,
comparison scope, action SHA, requested and resolved Jibril version) to the step summary,
an artifact and a per-tag comment that is updated in place instead of duplicated.

A pull_request trigger scoped to the gate files runs the chain in PR context, the only
context where the App comment exists. The caller grants pull-requests: write to the called
workflow; the workload step is non-fatal so a registry outage reads as an untested leg
rather than a failed gate.
@garnet-runtime-review

garnet-runtime-review Bot commented Sep 8, 2026 •

Copy link
Copy Markdown

Execution Profiles recorded for 1 job, triggered by a4abb18

1 job unchanged · compared with f76a862
recorded at the kernel by Garnet · 2026-09-09 04:38 UTC

Garnet Jibril Release Gate / reproduce ↗ · 2 destinations
@@ f76a862 (previous) vs a4abb18 (current) @@
  Runner.Worker
  └─ bash
     └─ node (step: "Start the clock")
        ├─ ○ localhost (dns resolver)
        └─ ○ registry.npmjs[.]org
 
- systemd (runner background · −1)
- └─ systemd-resolved
-    └─ ○ 8.8.8.8

View this job's Execution Profile in Garnet →


💡 How to read this
Runner.Worker          ← process on a path
└─ npm
   └─ node             ← process that acted
      └─ ○ npmjs[.]org ← observed action

follow a path downward to see what ran and what it did — each path to an observed action is an execution chain

names on the path = processes · ○ = observed action · (…) = context

+ only in the current record · − only in the previous record · runner background = the runner's infrastructure, not your workflow

@devin-ai-integration devin-ai-integration Bot changed the title ci: add a self-running Jibril release gate against the $/ reusable-workflow form ci: add a Jibril release gate that passes only on profile, App comment and permalink Sep 8, 2026
@github-actions

github-actions Bot commented Sep 8, 2026 •

Copy link
Copy Markdown

Jibril release gate · v2.17.0-rc.9 · FAIL

leg result
L1 startup FAIL — status file absent — sensor never reached main loop
L2 profile absent (control plane HTTP 403) · 0 egress peers, 0 destinations
L3 workload untested
L4 App comment pending · #43 (comment)
L5 permalink html untested · api untested
L6 integration shape action 3d47f4a9; sensor version left to the action; explicit token
L7 integration run untested: no TS CI run found for head 0a0750a
L8 reviewers zizmor: 0 finding(s) at the regular persona, 0 pedantic, on 2 touched workflow file(s) · unresolved bot threads on .github/: 0
L9 timing sensor start 12s · workload 1s with sensor vs 0s without · overhead 13s (budget 120s)
L10 coverage 1 of 6 CI cells instrumented: ubuntu/node 24 · not instrumented: ubuntu/node 22, ubuntu/node 26, windows 1/3/node 22, windows 2/3/node 22, windows 3/3/node 22 · Linux x86_64 only; macOS, Windows, fork PRs and Dependabot runs record nothing
record value
jibril requested v2.17.0-rc.9 · resolved unknown
action garnet-org/action@e546567a72e4fede11ec39d6e9f75b539adef22c
run 34288923777 attempt 1 · blacksmith-01m21jtfr76yr1xk6cg2vtbsff-8vcpu · pull_request
workflow ref seen by sensor absent
base → head 0d0bbd1 → 0a0750a
merge sha at run 6665a7d · now 6665a7d · profile recorded none
comparison in comment n/a
synthetic merge no

Failing legs:

  • L1 startup: status file absent — sensor never reached main loop
  • L2 profile: absent (control plane HTTP 403) after 600s (run 34288923777)
  • L4 app comment: pending after 600s for head 0a0750a · no profile reached the control plane, so the placeholder has nothing to finalize from

Disclosed, not failing:

  • L6 shape: release.yml keeps 2 Garnet step(s) on macOS jobs; the sensor never records there (CodeRabbit and Qodo flagged this on fork PRs)
  • L6 shape: ci.yml passes the token explicitly, so Dependabot-actor runs get an empty value and record nothing (upstream pull 13270)

Legs map to past failures in .github/garnet-gate/FAILURE_LEDGER.md. Full record in the run's garnet-release-gate artifact.

…record

When no profile exists the permalink leg is never probed; show it as
untested instead of a curl 000 code, and print the recorded profile SHA
as none instead of an empty code span.
… pnpm failure mode

The gate now runs on any pull request labelled garnet-release-testing and
reports on that pull request. The verifier grows from startup, profile,
workload, App comment and permalink to ten legs: integration shape as pnpm
reviewers read it, pnpm's own TS CI run on the candidate, zizmor and open
bot review threads on touched workflow files, sensor overhead against a
no-sensor baseline, and the instrumented share of the CI matrix.

Each leg maps to a row in .github/garnet-gate/FAILURE_LEDGER.md, which
lists every Garnet failure seen on pnpm so far and the check that catches
it. Disclosures that a machine cannot decide (macOS steps, Dependabot
runs, fork pull requests) print under the verdict instead of failing it.
@devin-ai-integration devin-ai-integration Bot added the garnet-release-testing Garnet product/release test PR; the Jibril release gate runs and reports here label Sep 8, 2026
@devin-ai-integration devin-ai-integration Bot changed the title ci: add a Jibril release gate that passes only on profile, App comment and permalink ci: a label-driven Jibril release gate covering every recorded pnpm failure mode Sep 8, 2026
@devin-ai-integration devin-ai-integration Bot added garnet-release-testing Garnet product/release test PR; the Jibril release gate runs and reports here and removed garnet-release-testing Garnet product/release test PR; the Jibril release gate runs and reports here labels Sep 9, 2026
@github-actions

github-actions Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

Jibril release gate · v2.17.0-rc.10 · FAIL

leg result
L1 startup PASS — sensor active, status ok, github steps resolution ok
L2 profile uploaded after 1s · 01a08476-307d-7456-9c96-4bcf02108314 · 2 egress peers, 2 destinations
L3 workload destination present, but attributed to steps ["0. Start the clock"] instead of "Workload with egress"
L4 App comment final after 0s · #43 (comment)
L5 permalink html 200 · api 200 · https://app.garnet.ai/public/runs/34311733947?profile=01a08476-307d-7456-9c96-4bcf02108314
L6 integration shape action 3d47f4a9; sensor version left to the action; explicit token
L7 integration run TS CI run 34311733945 finished success with no profile (no profile is bound to that run): the job was green, the sensor recorded nothing
L8 reviewers zizmor: 0 finding(s) at the regular persona, 0 pedantic, on 2 touched workflow file(s) · unresolved bot threads on .github/: 5
L9 timing sensor start 16s · workload 0s with sensor vs 0s without · overhead 16s (budget 120s) · profile visible 1s after the job · comment final 0s after the profile
L10 coverage 1 of 6 CI cells instrumented: ubuntu/node 24 · not instrumented: ubuntu/node 22, ubuntu/node 26, windows 1/3/node 22, windows 2/3/node 22, windows 3/3/node 22 · Linux x86_64 only; macOS, Windows, fork PRs and Dependabot runs record nothing
record value
jibril requested v2.17.0-rc.10 · resolved v2.17.0-rc.10
action garnet-org/action@e546567a72e4fede11ec39d6e9f75b539adef22c
run 34311733947 attempt 1 · blacksmith-01m226xjw08tkpf4gx4yytjnhp-8vcpu · pull_request
workflow ref seen by sensor garnet-labs/pnpm/.github/workflows/garnet-jibril-release-gate.yml@refs/pull/43/merge
base → head 0d0bbd1 → a4abb18
merge sha at run 1a06ba6 · now 1a06ba6 · profile recorded 1a06ba6 (refs/pull/43/merge)
comparison in comment immediate parent → head (previous f76a862)
synthetic merge yes · sensor recorded merge commit 1a06ba6, PR head is a4abb18

Failing legs:

Disclosed, not failing:

  • L6 shape: release.yml keeps 2 Garnet step(s) on macOS jobs; the sensor never records there (CodeRabbit and Qodo flagged this on fork PRs)
  • L6 shape: ci.yml passes the token explicitly, so Dependabot-actor runs get an empty value and record nothing (upstream pull 13270)

Legs map to past failures in .github/garnet-gate/FAILURE_LEDGER.md. Full record in the run's garnet-release-gate artifact.

…ob list workflow runs

Jibril resolves GITHUB_JOB against the caller workflow file. With the caller
job named gate and the callee job named reproduce, v2.17.0-rc.10 reported
github.steps.status=degraded and attributed the workload egress to an unknown
step (run 34309875655). pnpm's ci.yml calls test.yml with matching job ids, so
the gate now does the same.

The verify job's token had no actions: read, so listing the PR head's TS CI
run failed and the integration leg read the error as an absent run. The leg
now fails on a listing error and reports an absent run only on an empty list.

Ledger rows F21 and F22 record both.
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

/agentic_review

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

@coderabbitai review

@qodo-code-review

qodo-code-review Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (9) 📘 Rule violations (1) 📜 Skill insights (0)

Grey Divider


Action required

1. The gate rejects its own workload evidence 🐞 Bug ≡ Correctness
Description
The reproduction adds two id-bearing run steps before Workload with egress, while the verifier
requires the registry destination to be attributed to that workload name. Because Jibril numbers
id-less run steps differently from GitHub and the recorded failure shows this shape shifts
attribution to an earlier step, the gate can fail L3 on its deliberately reproduced workload even
when the sensor records the egress.
Code

.github/workflows/garnet-jibril-release-gate-job.yml[R104-106]

+      # step with an id records as `<unknown>` (ledger F23). The two
+      # id-bearing run steps above stay: pnpm's job has one before its test
+      # step, and Jibril counts them where GitHub does not (ledger F25).
Evidence
The reproduction's Start the clock and Stop the action clock steps both have ids and use run,
then the workload step is intentionally id-less; the verifier fails whenever the destination is
attributed to another step. The ledger documents that id-bearing steps before an id-less workload
shift Jibril attribution backward, so this reproduction does not actually preserve the claimed
one-step pnpm shape and can make the gate self-fail.

.github/workflows/garnet-jibril-release-gate-job.yml[71-95]
.github/workflows/garnet-jibril-release-gate-job.yml[101-115]
.github/scripts/garnet-release-gate-verify.sh[119-130]
.github/garnet-gate/FAILURE_LEDGER.md[46-50]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The reproduction places two id-bearing `run` steps before the id-less workload step, while the verifier requires attribution to `Workload with egress`. Jibril's step numbering can therefore attribute the registry egress to an earlier step and make L3 fail for a healthy reproduction.

## Issue Context
The focused change documents that the reproduced shape should match pnpm's test job, which has one id-bearing run step before the workload. The verifier treats any attribution other than the configured workload name as a failure.

## Fix Focus Areas
- .github/workflows/garnet-jibril-release-gate-job.yml[71-107]
- .github/scripts/garnet-release-gate-verify.sh[115-130]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Missing CI runs still pass the gate 🐞 Bug ≡ Correctness
Description
L7 immediately breaks its polling loop when no matching TS CI run exists and never calls
leg_fail. When the run is absent rather than merely still starting, the final verdict can remain
PASS even though the required integration execution was never observed.
Code

.github/scripts/garnet-release-gate-verify.sh[R269-270]

+    run="$(jq -c --arg n "$INTEGRATION_WORKFLOW_NAME" '[.workflow_runs[] | select(.name == $n)] | sort_by(.run_attempt) | last // empty' <<<"$runs_json")"
+    [ -n "$run" ] || break
Evidence
Static inference: the state starts as untested, the no-match branch exits without recording a
failure, and the overall verdict only checks fail_reasons.

.github/scripts/garnet-release-gate-verify.sh[256-286]
.github/scripts/garnet-release-gate-verify.sh[382-384]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
L7 treats an absent integration workflow run as non-failing and exits the advertised polling loop immediately.

## Issue Context
Continue polling while no matching run is registered, then append an L7 failure if the deadline expires without one.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[256-286]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Large diffs bypass permission checks 🐞 Bug ⛨ Security
Description
L6 derives added workflow lines from the compare API's optional patch field and silently
substitutes an empty string when that field is absent or the request fails. A sufficiently large
comparison or unavailable API can therefore hide newly added token permissions and let the
integration-shape leg pass without examining them.
Code

.github/scripts/garnet-release-gate-verify.sh[R232-234]

+  added_lines="$(gh api "repos/$repo/compare/$PR_BASE_SHA...$PR_HEAD_SHA" --paginate 2>/dev/null \
+    | jq -r '.files[]? | select(.filename | startswith(".github/workflows/")) | "\(.filename)\t\(.patch // "")"' \
+    | awk -F'\t' '{f=$1; n=split($2, L, "\n"); for(i=1;i<=n;i++) if (L[i] ~ /^\+/) print f "\t" substr(L[i],2)}')"
Evidence
Static inference: .patch // "" converts missing patches into clean input, and the pipeline status
is never checked before the permission searches. GitHub documents that compare results have file
limits and that larger diffs may fail, so the API response is not a fail-closed source for this
policy check.

.github/scripts/garnet-release-gate-verify.sh[230-245]
🌐 GitHub documents limits on files returned by comparisons and notes that large diff or patch responses may time out with a 5xx response.

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The permission gate silently loses workflow additions when the compare API fails, truncates files, or omits a patch.

## Issue Context
The repository is already checked out, so use a complete local diff or explicitly fail L6 whenever the remote comparison cannot be proven complete.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[230-245]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View high (2)
4. Failed workloads still pass the gate ✓ Resolved 🐞 Bug ≡ Correctness
Description
L3 records every non-success workload outcome as untested without adding a failure reason.
Because the workload step is explicitly allowed to fail, a registry or command failure can leave
workload attribution entirely unverified while the overall gate reports PASS.
Code

.github/scripts/garnet-release-gate-verify.sh[R115-118]

+workload_state="untested"
+if [ "${WORKLOAD_OUTCOME:-}" != success ]; then
+  workload_state="untested: workload step outcome=${WORKLOAD_OUTCOME:-unknown}"
+elif [ -n "$profile_json" ]; then
Evidence
Static inference: the workflow deliberately continues after workload failure, the verifier's
non-success branch records no failure, and only entries in fail_reasons change the final verdict
from PASS.

.github/workflows/garnet-jibril-release-gate-job.yml[97-105]
.github/scripts/garnet-release-gate-verify.sh[112-132]
.github/scripts/garnet-release-gate-verify.sh[382-384]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A failed or skipped workload leaves L3 untested but does not prevent a PASS verdict.

## Issue Context
Preserve `continue-on-error` so the verifier runs, but have the verifier append an L3 failure or introduce a non-PASS inconclusive verdict whenever workload evidence is unavailable.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[112-132]
- .github/workflows/garnet-jibril-release-gate-job.yml[100-105]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


5. Anyone can suppress scheduled gates 🐞 Bug ⛨ Security
Description
detect treats any issue comment containing the predictable per-tag marker as proof that the
release was already gated, without validating the comment author. A public commenter can post that
marker before the scheduled run and force run_gate=false, while the updater can also select the
spoofed comment as its update target.
Code

.github/workflows/garnet-jibril-release-gate.yml[R74-77]

+            export MARKER="<!-- garnet:jibril-release-gate $tag -->"
+            existing="$(gh api "repos/$GH_REPO/issues/$report_pr/comments" --paginate \
+              | jq -r '.[] | select((.body // "") | contains($ENV.MARKER)) | .id' | head -n1)"
+            [ -z "$existing" ] || run_gate=false
Evidence
Static inference: the scheduled detector enumerates every issue comment and selects solely by a body
substring before disabling the run; the updater repeats the same unrestricted selection.

.github/workflows/garnet-jibril-release-gate.yml[70-81]
.github/workflows/garnet-jibril-release-gate-job.yml[224-239]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Scheduled gates trust a marker in any PR comment, allowing an untrusted commenter to suppress a release run or become the updater's selected comment.

## Issue Context
Both marker lookup sites must validate that the comment belongs to the trusted gate identity, not merely that its body contains the marker.

## Fix Focus Areas
- .github/workflows/garnet-jibril-release-gate.yml[74-77]
- .github/workflows/garnet-jibril-release-gate-job.yml[232-238]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

6. Failed runs are reported as green 🐞 Bug ◔ Observability
Description
L7 interpolates the integration run’s actual conclusion but then unconditionally appends “the job
was green” to every missing-profile message. When the selected run concluded with failure,
cancellation, or another non-success result and has no profile, the gate record contradicts itself
and misstates the underlying execution.
Code

.github/scripts/garnet-release-gate-verify.sh[288]

+      integration_state="$INTEGRATION_WORKFLOW_NAME run $integration_run_id finished $integration_conclusion with no profile ($why): the job was green, the sensor recorded nothing"
Evidence
Static inference: the workflow run’s .conclusion is stored in integration_conclusion, but the
newly changed diagnostic says “the job was green” without checking that value.

.github/scripts/garnet-release-gate-verify.sh[269-273]
.github/scripts/garnet-release-gate-verify.sh[277-289]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The missing-profile diagnostic always says the integration job was green, regardless of its recorded conclusion.

## Issue Context
Use the captured conclusion to emit the green-job diagnosis only for successful runs, and accurately describe unsuccessful runs.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[283-288]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


7. Credential failures are misdiagnosed 🐞 Bug ◔ Observability
Description
L7 unconditionally translates HTTP 403 from fetch_profile into “no profile is bound,” although
the failure ledger says the same response also represents a token without access. When credentials
are invalid or unauthorized, the gate still fails safely but sends operators toward sensor ingestion
rather than the broken authentication path.
Code

.github/scripts/garnet-release-gate-verify.sh[285]

+        403) why="no profile is bound to that run" ;;
Evidence
Static inference: fetch_profile authenticates with GARNET_API_TOKEN, while the newly added
ledger entry explicitly records that HTTP 403 is shared by missing profiles and tokens without
access; the new branch nevertheless assigns only the missing-profile explanation.

.github/scripts/garnet-release-gate-verify.sh[63-66]
.github/scripts/garnet-release-gate-verify.sh[283-288]
.github/garnet-gate/FAILURE_LEDGER.md[49-49]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
L7 reports every HTTP 403 as proof that no profile is bound, even though 403 can also mean the supplied token lacks access.

## Issue Context
The gate should preserve the ambiguity unless another authenticated check can distinguish an absent profile from invalid authorization.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[283-288]
- .github/garnet-gate/FAILURE_LEDGER.md[49-49]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


8. Missing timing data still passes 🐞 Bug ≡ Correctness
Description
L9 turns absent action, workload, or baseline clocks into an untested message without calling
leg_fail. If the baseline job fails before emitting its output, verify still runs through
always() and can declare PASS without checking the required overhead budget.
Code

.github/scripts/garnet-release-gate-verify.sh[R349-352]

+  [ "$overhead_seconds" -le "$OVERHEAD_BUDGET_SECONDS" ] || leg_fail "L9 timing: overhead ${overhead_seconds}s exceeds ${OVERHEAD_BUDGET_SECONDS}s"
+else
+  timing_state="untested: missing job clocks (action=${action_seconds:-?} workload=${REPRODUCE_WORKLOAD_SECONDS:-?} baseline=${BASELINE_WORKLOAD_SECONDS:-?})"
+fi
Evidence
Static inference: baseline network commands can terminate that job before seconds is emitted,
while the always-running verifier merely labels missing clocks untested and the final verdict
ignores that state.

.github/workflows/garnet-jibril-release-gate-job.yml[29-43]
.github/workflows/garnet-jibril-release-gate-job.yml[168-177]
.github/scripts/garnet-release-gate-verify.sh[343-354]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The timing leg does not affect the verdict when one of its required clocks is missing.

## Issue Context
Check dependency results and required timing outputs explicitly, then fail or return an inconclusive non-PASS verdict when the comparison cannot be calculated.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[343-352]
- .github/workflows/garnet-jibril-release-gate-job.yml[168-177]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View medium (3)
9. Fresh profiles are rejected too early 🐞 Bug ☼ Reliability
Description
L7 calls fetch_profile only once immediately after the selected integration workflow completes,
unlike L2, which polls for eventual control-plane visibility. When profile ingestion lags workflow
completion, a valid integration run is reported as having recorded nothing and fails the gate
spuriously.
Code

.github/scripts/garnet-release-gate-verify.sh[R277-280]

+  if [ -n "$integration_run_id" ] && [ -n "$integration_conclusion" ]; then
+    code="$(fetch_profile "$integration_run_id")"
+    if [ "$code" = 200 ]; then
+      integration_profile_id="$(jq -r '.id' "/tmp/profile-$integration_run_id.json")"
Evidence
Static inference: L2 explicitly polls 403 and 404 responses for up to 600 seconds, establishing that
profile visibility can lag, while L7 converts its first non-200 response directly into a failure.

.github/scripts/garnet-release-gate-verify.sh[60-86]
.github/scripts/garnet-release-gate-verify.sh[277-285]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
L7 performs a single profile lookup even though control-plane profile visibility is asynchronous.

## Issue Context
Reuse the bounded 403/404 polling behavior from L2 for the selected integration run, while still failing immediately on unexpected response codes.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[63-86]
- .github/scripts/garnet-release-gate-verify.sh[277-285]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


10. New releases can be missed 🐞 Bug ≡ Correctness
Description
detect chooses the latest publication only from the first 20 releases returned by the GitHub API.
A release created earlier as a draft and published after enough newer-created entries can remain
outside that page, causing the schedule to keep selecting an older tag instead of gating the new
release.
Code

.github/workflows/garnet-jibril-release-gate.yml[R53-56]

+          # Newest release by published_at, pre-releases included, drafts excluded.
+          json="$(curl -sSf -H 'Accept: application/vnd.github+json' https://api.github.com/repos/garnet-org/jibril-releases/releases?per_page=20)"
+          newest_tag="$(jq -r '[.[] | select(.draft==false)] | sort_by(.published_at) | last | .tag_name' <<<"$json")"
+          newest_at="$(jq -r '[.[] | select(.draft==false)] | sort_by(.published_at) | last | .published_at' <<<"$json")"
Evidence
Static inference: the request explicitly limits its response to 20 entries and performs the
publication-time sort only over that subset, despite claiming to select the newest release.

.github/workflows/garnet-jibril-release-gate.yml[53-68]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Scheduled release selection sorts only a truncated first page of releases.

## Issue Context
Fetch all relevant pages, or use an endpoint and ordering contract that reliably identifies the newest published non-draft release including prereleases.

## Fix Focus Areas
- .github/workflows/garnet-jibril-release-gate.yml[53-68]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


11. A redundant comment adds noise 📘 Rule violation ⚙ Maintainability
Description
The Poll up to 90s comment merely restates the following 18-iteration loop, five-second sleep, and
runtime-status-file check. If the retry count, delay, or file path changes without updating this
separate narration, a later maintainer receives an outdated description of the assertion step.
Code

.github/workflows/garnet-jibril-release-gate-job.yml[115]

+          # Poll up to 90s for the runtime status file.
Evidence
Compliance rule 2757161 prohibits comments that only duplicate straightforward code behavior. The
added comment states the timeout and target already conveyed by the immediately following loop.

Rule 2757161: Avoid redundant comments that duplicate code behavior
.github/workflows/garnet-jibril-release-gate-job.yml[115-120]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Remove the comment that narrates the polling loop's immediately visible behavior.

## Issue Context
The loop already expresses its retry count, delay, and checked status-file path, so the comment can drift independently without adding rationale.

## Fix Focus Areas
- .github/workflows/garnet-jibril-release-gate-job.yml[115-120]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
✅ Compliance rules (platform): 36 rules
✅ REVIEW.md
Review mode: ⚖️ Balanced: The small push changes CI gate behavior and shell-script failure reporting in a security-relevant workflow, so it warrants a careful single-pass review despite being localized.

Grey Divider

Tip of the day
💡 Did you know, you can commit Qodo's fix in one click with committable suggestions (GitHub & GitLab)

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Previous reviews

Review updated until commit a4abb18 ⚖️ Balanced

Results up to commit 3b9024a ⚖️ Balanced


🐞 Bugs (6) 📘 Rule violations (1) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)


Action required
1. Anyone can suppress scheduled gates 🐞 Bug ⛨ Security
Description
detect treats any issue comment containing the predictable per-tag marker as proof that the
release was already gated, without validating the comment author. A public commenter can post that
marker before the scheduled run and force run_gate=false, while the updater can also select the
spoofed comment as its update target.
Code

.github/workflows/garnet-jibril-release-gate.yml[R74-77]

+            export MARKER="<!-- garnet:jibril-release-gate $tag -->"
+            existing="$(gh api "repos/$GH_REPO/issues/$report_pr/comments" --paginate \
+              | jq -r '.[] | select((.body // "") | contains($ENV.MARKER)) | .id' | head -n1)"
+            [ -z "$existing" ] || run_gate=false
Evidence
Static inference: the scheduled detector enumerates every issue comment and selects solely by a body
substring before disabling the run; the updater repeats the same unrestricted selection.

.github/workflows/garnet-jibril-release-gate.yml[70-81]
.github/workflows/garnet-jibril-release-gate-job.yml[224-239]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Scheduled gates trust a marker in any PR comment, allowing an untrusted commenter to suppress a release run or become the updater's selected comment.

## Issue Context
Both marker lookup sites must validate that the comment belongs to the trusted gate identity, not merely that its body contains the marker.

## Fix Focus Areas
- .github/workflows/garnet-jibril-release-gate.yml[74-77]
- .github/workflows/garnet-jibril-release-gate-job.yml[232-238]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Large diffs bypass permission checks 🐞 Bug ⛨ Security
Description
L6 derives added workflow lines from the compare API's optional patch field and silently
substitutes an empty string when that field is absent or the request fails. A sufficiently large
comparison or unavailable API can therefore hide newly added token permissions and let the
integration-shape leg pass without examining them.
Code

.github/scripts/garnet-release-gate-verify.sh[R232-234]

+  added_lines="$(gh api "repos/$repo/compare/$PR_BASE_SHA...$PR_HEAD_SHA" --paginate 2>/dev/null \
+    | jq -r '.files[]? | select(.filename | startswith(".github/workflows/")) | "\(.filename)\t\(.patch // "")"' \
+    | awk -F'\t' '{f=$1; n=split($2, L, "\n"); for(i=1;i<=n;i++) if (L[i] ~ /^\+/) print f "\t" substr(L[i],2)}')"
Evidence
Static inference: .patch // "" converts missing patches into clean input, and the pipeline status
is never checked before the permission searches. GitHub documents that compare results have file
limits and that larger diffs may fail, so the API response is not a fail-closed source for this
policy check.

.github/scripts/garnet-release-gate-verify.sh[230-245]
🌐 GitHub documents limits on files returned by comparisons and notes that large diff or patch responses may time out with a 5xx response.

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The permission gate silently loses workflow additions when the compare API fails, truncates files, or omits a patch.

## Issue Context
The repository is already checked out, so use a complete local diff or explicitly fail L6 whenever the remote comparison cannot be proven complete.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[230-245]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Missing CI runs still pass the gate 🐞 Bug ≡ Correctness
Description
L7 immediately breaks its polling loop when no matching TS CI run exists and never calls
leg_fail. When the run is absent rather than merely still starting, the final verdict can remain
PASS even though the required integration execution was never observed.
Code

.github/scripts/garnet-release-gate-verify.sh[R269-270]

+    run="$(jq -c --arg n "$INTEGRATION_WORKFLOW_NAME" '[.workflow_runs[] | select(.name == $n)] | sort_by(.run_attempt) | last // empty' <<<"$runs_json")"
+    [ -n "$run" ] || break
Evidence
Static inference: the state starts as untested, the no-match branch exits without recording a
failure, and the overall verdict only checks fail_reasons.

.github/scripts/garnet-release-gate-verify.sh[256-286]
.github/scripts/garnet-release-gate-verify.sh[382-384]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
L7 treats an absent integration workflow run as non-failing and exits the advertised polling loop immediately.

## Issue Context
Continue polling while no matching run is registered, then append an L7 failure if the deadline expires without one.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[256-286]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View high (1)
4. Failed workloads still pass the gate ✓ Resolved 🐞 Bug ≡ Correctness
Description
L3 records every non-success workload outcome as untested without adding a failure reason.
Because the workload step is explicitly allowed to fail, a registry or command failure can leave
workload attribution entirely unverified while the overall gate reports PASS.
Code

.github/scripts/garnet-release-gate-verify.sh[R115-118]

+workload_state="untested"
+if [ "${WORKLOAD_OUTCOME:-}" != success ]; then
+  workload_state="untested: workload step outcome=${WORKLOAD_OUTCOME:-unknown}"
+elif [ -n "$profile_json" ]; then
Evidence
Static inference: the workflow deliberately continues after workload failure, the verifier's
non-success branch records no failure, and only entries in fail_reasons change the final verdict
from PASS.

.github/workflows/garnet-jibril-release-gate-job.yml[97-105]
.github/scripts/garnet-release-gate-verify.sh[112-132]
.github/scripts/garnet-release-gate-verify.sh[382-384]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A failed or skipped workload leaves L3 untested but does not prevent a PASS verdict.

## Issue Context
Preserve `continue-on-error` so the verifier runs, but have the verifier append an L3 failure or introduce a non-PASS inconclusive verdict whenever workload evidence is unavailable.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[112-132]
- .github/workflows/garnet-jibril-release-gate-job.yml[100-105]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended
5. A redundant comment adds noise 📘 Rule violation ⚙ Maintainability
Description
The Poll up to 90s comment merely restates the following 18-iteration loop, five-second sleep, and
runtime-status-file check. If the retry count, delay, or file path changes without updating this
separate narration, a later maintainer receives an outdated description of the assertion step.
Code

.github/workflows/garnet-jibril-release-gate-job.yml[115]

+          # Poll up to 90s for the runtime status file.
Evidence
Compliance rule 2757161 prohibits comments that only duplicate straightforward code behavior. The
added comment states the timeout and target already conveyed by the immediately following loop.

Rule 2757161: Avoid redundant comments that duplicate code behavior
.github/workflows/garnet-jibril-release-gate-job.yml[115-120]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Remove the comment that narrates the polling loop's immediately visible behavior.

## Issue Context
The loop already expresses its retry count, delay, and checked status-file path, so the comment can drift independently without adding rationale.

## Fix Focus Areas
- .github/workflows/garnet-jibril-release-gate-job.yml[115-120]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


6. Missing timing data still passes 🐞 Bug ≡ Correctness
Description
L9 turns absent action, workload, or baseline clocks into an untested message without calling
leg_fail. If the baseline job fails before emitting its output, verify still runs through
always() and can declare PASS without checking the required overhead budget.
Code

.github/scripts/garnet-release-gate-verify.sh[R349-352]

+  [ "$overhead_seconds" -le "$OVERHEAD_BUDGET_SECONDS" ] || leg_fail "L9 timing: overhead ${overhead_seconds}s exceeds ${OVERHEAD_BUDGET_SECONDS}s"
+else
+  timing_state="untested: missing job clocks (action=${action_seconds:-?} workload=${REPRODUCE_WORKLOAD_SECONDS:-?} baseline=${BASELINE_WORKLOAD_SECONDS:-?})"
+fi
Evidence
Static inference: baseline network commands can terminate that job before seconds is emitted,
while the always-running verifier merely labels missing clocks untested and the final verdict
ignores that state.

.github/workflows/garnet-jibril-release-gate-job.yml[29-43]
.github/workflows/garnet-jibril-release-gate-job.yml[168-177]
.github/scripts/garnet-release-gate-verify.sh[343-354]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The timing leg does not affect the verdict when one of its required clocks is missing.

## Issue Context
Check dependency results and required timing outputs explicitly, then fail or return an inconclusive non-PASS verdict when the comparison cannot be calculated.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[343-352]
- .github/workflows/garnet-jibril-release-gate-job.yml[168-177]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


7. Fresh profiles are rejected too early 🐞 Bug ☼ Reliability
Description
L7 calls fetch_profile only once immediately after the selected integration workflow completes,
unlike L2, which polls for eventual control-plane visibility. When profile ingestion lags workflow
completion, a valid integration run is reported as having recorded nothing and fails the gate
spuriously.
Code

.github/scripts/garnet-release-gate-verify.sh[R277-280]

+  if [ -n "$integration_run_id" ] && [ -n "$integration_conclusion" ]; then
+    code="$(fetch_profile "$integration_run_id")"
+    if [ "$code" = 200 ]; then
+      integration_profile_id="$(jq -r '.id' "/tmp/profile-$integration_run_id.json")"
Evidence
Static inference: L2 explicitly polls 403 and 404 responses for up to 600 seconds, establishing that
profile visibility can lag, while L7 converts its first non-200 response directly into a failure.

.github/scripts/garnet-release-gate-verify.sh[60-86]
.github/scripts/garnet-release-gate-verify.sh[277-285]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
L7 performs a single profile lookup even though control-plane profile visibility is asynchronous.

## Issue Context
Reuse the bounded 403/404 polling behavior from L2 for the selected integration run, while still failing immediately on unexpected response codes.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[63-86]
- .github/scripts/garnet-release-gate-verify.sh[277-285]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View medium (1)
8. New releases can be missed 🐞 Bug ≡ Correctness
Description
detect chooses the latest publication only from the first 20 releases returned by the GitHub API.
A release created earlier as a draft and published after enough newer-created entries can remain
outside that page, causing the schedule to keep selecting an older tag instead of gating the new
release.
Code

.github/workflows/garnet-jibril-release-gate.yml[R53-56]

+          # Newest release by published_at, pre-releases included, drafts excluded.
+          json="$(curl -sSf -H 'Accept: application/vnd.github+json' https://api.github.com/repos/garnet-org/jibril-releases/releases?per_page=20)"
+          newest_tag="$(jq -r '[.[] | select(.draft==false)] | sort_by(.published_at) | last | .tag_name' <<<"$json")"
+          newest_at="$(jq -r '[.[] | select(.draft==false)] | sort_by(.published_at) | last | .published_at' <<<"$json")"
Evidence
Static inference: the request explicitly limits its response to 20 entries and performs the
publication-time sort only over that subset, despite claiming to select the newest release.

.github/workflows/garnet-jibril-release-gate.yml[53-68]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Scheduled release selection sorts only a truncated first page of releases.

## Issue Context
Fetch all relevant pages, or use an endpoint and ordering contract that reliably identifies the newest published non-draft release including prereleases.

## Fix Focus Areas
- .github/workflows/garnet-jibril-release-gate.yml[53-68]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Results up to commit f76a862 ⚖️ Balanced


🐞 Bugs (2) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)


Remediation recommended
1. Credential failures are misdiagnosed 🐞 Bug ◔ Observability
Description
L7 unconditionally translates HTTP 403 from fetch_profile into “no profile is bound,” although
the failure ledger says the same response also represents a token without access. When credentials
are invalid or unauthorized, the gate still fails safely but sends operators toward sensor ingestion
rather than the broken authentication path.
Code

.github/scripts/garnet-release-gate-verify.sh[285]

+        403) why="no profile is bound to that run" ;;
Evidence
Static inference: fetch_profile authenticates with GARNET_API_TOKEN, while the newly added
ledger entry explicitly records that HTTP 403 is shared by missing profiles and tokens without
access; the new branch nevertheless assigns only the missing-profile explanation.

.github/scripts/garnet-release-gate-verify.sh[63-66]
.github/scripts/garnet-release-gate-verify.sh[283-288]
.github/garnet-gate/FAILURE_LEDGER.md[49-49]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
L7 reports every HTTP 403 as proof that no profile is bound, even though 403 can also mean the supplied token lacks access.

## Issue Context
The gate should preserve the ambiguity unless another authenticated check can distinguish an absent profile from invalid authorization.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[283-288]
- .github/garnet-gate/FAILURE_LEDGER.md[49-49]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Failed runs are reported as green 🐞 Bug ◔ Observability
Description
L7 interpolates the integration run’s actual conclusion but then unconditionally appends “the job
was green” to every missing-profile message. When the selected run concluded with failure,
cancellation, or another non-success result and has no profile, the gate record contradicts itself
and misstates the underlying execution.
Code

.github/scripts/garnet-release-gate-verify.sh[288]

+      integration_state="$INTEGRATION_WORKFLOW_NAME run $integration_run_id finished $integration_conclusion with no profile ($why): the job was green, the sensor recorded nothing"
Evidence
Static inference: the workflow run’s .conclusion is stored in integration_conclusion, but the
newly changed diagnostic says “the job was green” without checking that value.

.github/scripts/garnet-release-gate-verify.sh[269-273]
.github/scripts/garnet-release-gate-verify.sh[277-289]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The missing-profile diagnostic always says the integration job was green, regardless of its recorded conclusion.

## Issue Context
Use the captured conclusion to emit the green-job diagnosis only for successful runs, and accurately describe unsuccessful runs.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[283-288]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Results up to commit a4abb18 🚀 Fast


🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)


Action required
1. The gate rejects its own workload evidence 🐞 Bug ≡ Correctness
Description
The reproduction adds two id-bearing run steps before Workload with egress, while the verifier
requires the registry destination to be attributed to that workload name. Because Jibril numbers
id-less run steps differently from GitHub and the recorded failure shows this shape shifts
attribution to an earlier step, the gate can fail L3 on its deliberately reproduced workload even
when the sensor records the egress.
Code

.github/workflows/garnet-jibril-release-gate-job.yml[R104-106]

+      # step with an id records as `<unknown>` (ledger F23). The two
+      # id-bearing run steps above stay: pnpm's job has one before its test
+      # step, and Jibril counts them where GitHub does not (ledger F25).
Evidence
The reproduction's Start the clock and Stop the action clock steps both have ids and use run,
then the workload step is intentionally id-less; the verifier fails whenever the destination is
attributed to another step. The ledger documents that id-bearing steps before an id-less workload
shift Jibril attribution backward, so this reproduction does not actually preserve the claimed
one-step pnpm shape and can make the gate self-fail.

.github/workflows/garnet-jibril-release-gate-job.yml[71-95]
.github/workflows/garnet-jibril-release-gate-job.yml[101-115]
.github/scripts/garnet-release-gate-verify.sh[119-130]
.github/garnet-gate/FAILURE_LEDGER.md[46-50]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The reproduction places two id-bearing `run` steps before the id-less workload step, while the verifier requires attribution to `Workload with egress`. Jibril's step numbering can therefore attribute the registry egress to an earlier step and make L3 fail for a healthy reproduction.

## Issue Context
The focused change documents that the reproduced shape should match pnpm's test job, which has one id-bearing run step before the workload. The verifier treats any attribution other than the configured workload name as a failure.

## Fix Focus Areas
- .github/workflows/garnet-jibril-release-gate-job.yml[71-107]
- .github/scripts/garnet-release-gate-verify.sh[115-130]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

Comment on lines +74 to +77
export MARKER="<!-- garnet:jibril-release-gate $tag -->"
existing="$(gh api "repos/$GH_REPO/issues/$report_pr/comments" --paginate \
| jq -r '.[] | select((.body // "") | contains($ENV.MARKER)) | .id' | head -n1)"
[ -z "$existing" ] || run_gate=false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

2. Anyone can suppress scheduled gates 🐞 Bug ⛨ Security

detect treats any issue comment containing the predictable per-tag marker as proof that the
release was already gated, without validating the comment author. A public commenter can post that
marker before the scheduled run and force run_gate=false, while the updater can also select the
spoofed comment as its update target.
Agent Prompt
## Issue description
Scheduled gates trust a marker in any PR comment, allowing an untrusted commenter to suppress a release run or become the updater's selected comment.

## Issue Context
Both marker lookup sites must validate that the comment belongs to the trusted gate identity, not merely that its body contains the marker.

## Fix Focus Areas
- .github/workflows/garnet-jibril-release-gate.yml[74-77]
- .github/workflows/garnet-jibril-release-gate-job.yml[232-238]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment on lines +232 to +234
added_lines="$(gh api "repos/$repo/compare/$PR_BASE_SHA...$PR_HEAD_SHA" --paginate 2>/dev/null \
| jq -r '.files[]? | select(.filename | startswith(".github/workflows/")) | "\(.filename)\t\(.patch // "")"' \
| awk -F'\t' '{f=$1; n=split($2, L, "\n"); for(i=1;i<=n;i++) if (L[i] ~ /^\+/) print f "\t" substr(L[i],2)}')"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

3. Large diffs bypass permission checks 🐞 Bug ⛨ Security

L6 derives added workflow lines from the compare API's optional patch field and silently
substitutes an empty string when that field is absent or the request fails. A sufficiently large
comparison or unavailable API can therefore hide newly added token permissions and let the
integration-shape leg pass without examining them.
Agent Prompt
## Issue description
The permission gate silently loses workflow additions when the compare API fails, truncates files, or omits a patch.

## Issue Context
The repository is already checked out, so use a complete local diff or explicitly fail L6 whenever the remote comparison cannot be proven complete.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[230-245]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment on lines +269 to +270
run="$(jq -c --arg n "$INTEGRATION_WORKFLOW_NAME" '[.workflow_runs[] | select(.name == $n)] | sort_by(.run_attempt) | last // empty' <<<"$runs_json")"
[ -n "$run" ] || break

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

4. Missing ci runs still pass the gate 🐞 Bug ≡ Correctness

L7 immediately breaks its polling loop when no matching TS CI run exists and never calls
leg_fail. When the run is absent rather than merely still starting, the final verdict can remain
PASS even though the required integration execution was never observed.
Agent Prompt
## Issue description
L7 treats an absent integration workflow run as non-failing and exits the advertised polling loop immediately.

## Issue Context
Continue polling while no matching run is registered, then append an L7 failure if the deadline expires without one.

## Fix Focus Areas
- .github/scripts/garnet-release-gate-verify.sh[256-286]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

GitHub sets GITHUB_ACTION to a step's id when one is present. Jibril
derives the step value from the workflow file as __run_N, so a run step
with an id records on step <unknown> even when step discovery is ok.
pnpm's install and test steps carry no id; the gate now matches them and
logs GITHUB_ACTION for the record. The workload outcome reaches the
assert step through a file.

L7 names the control plane's 403 for what it is on this path: no profile
bound to the run.
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

/agentic_review

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

@coderabbitai review

@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit f76a862

…tion

With no id on the workload step the egress recorded on the job's first
run step. GitHub numbers only id-less run steps (GITHUB_ACTION=__run for
the third run step of the job); Jibril numbers every run step, so each
id-bearing run step ahead of the workload shifts attribution one step
back. pnpm's test job has one such step before its test step.

The gate keeps that shape and the L3 leg names the expected step next
to the recorded one.
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

/agentic_review

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

@coderabbitai review

Comment on lines +104 to +106
# step with an id records as `<unknown>` (ledger F23). The two
# id-bearing run steps above stay: pnpm's job has one before its test
# step, and Jibril counts them where GitHub does not (ledger F25).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. The gate rejects its own workload evidence 🐞 Bug ≡ Correctness

The reproduction adds two id-bearing run steps before Workload with egress, while the verifier
requires the registry destination to be attributed to that workload name. Because Jibril numbers
id-less run steps differently from GitHub and the recorded failure shows this shape shifts
attribution to an earlier step, the gate can fail L3 on its deliberately reproduced workload even
when the sensor records the egress.
Agent Prompt
## Issue description
The reproduction places two id-bearing `run` steps before the id-less workload step, while the verifier requires attribution to `Workload with egress`. Jibril's step numbering can therefore attribute the registry egress to an earlier step and make L3 fail for a healthy reproduction.

## Issue Context
The focused change documents that the reproduced shape should match pnpm's test job, which has one id-bearing run step before the workload. The verifier treats any attribution other than the configured workload name as a failure.

## Fix Focus Areas
- .github/workflows/garnet-jibril-release-gate-job.yml[71-107]
- .github/scripts/garnet-release-gate-verify.sh[115-130]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

@qodo-code-review

Copy link
Copy Markdown

PR Code Suggestions ✨

Warning

/improve is deprecated. Use /agentic_review instead (removal date not yet scheduled).

No code suggestions found for the PR.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 6 potential issues.

Devin Review

Comment on lines +271 to +275
run="$(jq -c --arg n "$INTEGRATION_WORKFLOW_NAME" '[.workflow_runs[] | select(.name == $n)] | sort_by(.run_attempt) | last // empty' <<<"$runs_json")"
[ -n "$run" ] || break
integration_run_id="$(jq -r '.id' <<<"$run")"
integration_conclusion="$(jq -r '.conclusion // ""' <<<"$run")"
[ "$(jq -r '.status' <<<"$run")" = completed ] && break

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Missing integration runs pass

When no matching run exists or it remains incomplete, integration_state stays untested without adding a failure. The gate can report PASS without exercising L7.

Prompt for agents
Update L7 in .github/scripts/garnet-release-gate-verify.sh so absence of a matching TS CI run, or failure to complete before INTEGRATION_POLL_SECONDS, adds an L7 failure. Continue polling when no run has appeared instead of breaking immediately. Preserve the separate error for inability to list runs, and only fetch a profile after a selected run completes.
Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +234 to +236
added_lines="$(gh api "repos/$repo/compare/$PR_BASE_SHA...$PR_HEAD_SHA" --paginate 2>/dev/null \
| jq -r '.files[]? | select(.filename | startswith(".github/workflows/")) | "\(.filename)\t\(.patch // "")"' \
| awk -F'\t' '{f=$1; n=split($2, L, "\n"); for(i=1;i<=n;i++) if (L[i] ~ /^\+/) print f "\t" substr(L[i],2)}')"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Permission additions evade validation

For multi-line patches, awk inspects only the first line because later lines lack the filename field. Forbidden permissions never reach added_lines, so L6 can accept them.

Prompt for agents
Rewrite the L6 compare parsing in .github/scripts/garnet-release-gate-verify.sh so every added patch line remains associated with its file. The GitHub compare API returns each patch as one string containing embedded newlines, so a filename plus tab prefix applies only to the first rendered line. Parse each file and its patch structurally, emit every '+' line with the filename, and distinguish API or parsing failures from an empty diff.
Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +280 to +283
code="$(fetch_profile "$integration_run_id")"
if [ "$code" = 200 ]; then
integration_profile_id="$(jq -r '.id' "/tmp/profile-$integration_run_id.json")"
integration_state="profile \`$integration_profile_id\` from $INTEGRATION_WORKFLOW_NAME run $integration_run_id ($integration_conclusion)"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Old profiles satisfy reruns

On a rerun, fetch_profile can return the previous attempt's profile, which L7 accepts without checking its attempt or job. A failed current attempt can satisfy the integration leg.

Prompt for agents
Validate the L7 profile against the selected TS CI run's current run_attempt and the expected instrumented job, as L2 already does for the gate profile. Treat a profile from another attempt or job as absent for the current integration run, and poll if current-attempt ingestion can lag.
Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +207 to +212
action_refs="$(grep -hoE 'garnet-org/action@[^ ]+' "${integration_files[@]}" 2>/dev/null | sort -u)"
unpinned="$(grep -vE '^garnet-org/action@[0-9a-f]{40}$' <<<"$action_refs" | sed '/^$/d')"
if [ -n "$unpinned" ]; then
shape_fail=true; shape_notes+=("action ref not a full SHA: $(tr '\n' ' ' <<<"$unpinned")")
else
shape_notes+=("action \`${action_refs#garnet-org/action@}\`")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Removed monitoring passes validation

When all monitoring actions are removed, action_refs and unpinned are empty. L6 treats that state as a valid full-SHA pin and can pass the broken integration.

Prompt for agents
Make L6 fail when no garnet-org/action reference is found in the required integration workflows. Keep that missing-action state distinct from the existing unpinned-reference failure, and ensure the expected workflow placement is validated rather than accepting one unrelated remaining reference.
Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +327 to +336
threads="$(gh api graphql -f query='
query($owner:String!,$name:String!,$number:Int!){
repository(owner:$owner,name:$name){ pullRequest(number:$number){
reviewThreads(first:100){ nodes{ isResolved path comments(first:1){ nodes{ author{ login } url } } } } } } }' \
-f owner="${repo%/*}" -f name="${repo#*/}" -F number="$PR_NUMBER" 2>/dev/null \
| jq -c '[.data.repository.pullRequest.reviewThreads.nodes[]?
| select(.isResolved == false)
| select(.path | startswith(".github/"))
| select((.comments.nodes[0].author.login // "") | test("coderabbit|qodo|greptile|devin|\\[bot\\]"; "i"))
| {path, author: .comments.nodes[0].author.login, url: .comments.nodes[0].url}]' || echo '[]')"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Unavailable reviews appear resolved

GraphQL failures become an empty thread list, so L8 passes when review status is unavailable. Threads after the first 100 are also ignored.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +101 to +107
# No `id:` here, as on pnpm's install and test steps. GitHub sets
# GITHUB_ACTION to the step id when one is present; Jibril derives
# `__run_N` from the workflow file and attributes by that value, so a
# step with an id records as `<unknown>` (ledger F23). The two
# id-bearing run steps above stay: pnpm's job has one before its test
# step, and Jibril counts them where GitHub does not (ledger F25).
- name: Workload with egress

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Workload attribution remains unresolved

Runtime evidence (Garnet, head a4abb18): Runner.Worker → bash → node reached registry.npmjs.org under “Start the clock” in Garnet Jibril Release Gate/reproduce. The recorded head still misses the intended workload step.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

break
fi
run="$(jq -c --arg n "$INTEGRATION_WORKFLOW_NAME" '[.workflow_runs[] | select(.name == $n)] | sort_by(.run_attempt) | last // empty' <<<"$runs_json")"
[ -n "$run" ] || break

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Integration check can pass untested

When the first workflow-run query finds no matching TS CI run, this line exits the loop immediately. L7 remains marked as “untested,” but no failure is recorded, so the gate can report PASS without verifying the required integration run. This is especially relevant when the gate starts before the parallel TS CI run has been registered, because the intended 25-minute polling window is bypassed after the first empty response.

Comment on lines +309 to +336
count_findings() { jq 'if type == "array" then map(select(.ignored | not)) | length else 0 end' "$1" 2>/dev/null || echo 0; }
# shellcheck disable=SC2086
zizmor --format json-v1 --no-exit-codes --offline $touched > /tmp/zizmor.json 2>/tmp/zizmor.err || true
zizmor_findings="$(count_findings /tmp/zizmor.json)"
# shellcheck disable=SC2086
zizmor --format json-v1 --no-exit-codes --offline --persona pedantic $touched > /tmp/zizmor-pedantic.json 2>/dev/null || true
zizmor_pedantic="$(count_findings /tmp/zizmor-pedantic.json)"
zizmor_state="$zizmor_findings finding(s) at the regular persona, $zizmor_pedantic pedantic, on $(wc -l <<<"$touched") touched workflow file(s)"
if [ "$zizmor_findings" -gt 0 ]; then
leg_fail "L8 reviewers: zizmor $zizmor_state"
jq -r '.[] | select(.ignored | not) | " - \(.ident) (\(.determinations.severity)): \(.locations[0].symbolic.key.Local.path // "")"' /tmp/zizmor.json 2>/dev/null
fi
else
zizmor_state="zizmor not installed"
fi
else
zizmor_state="no workflow files touched"
fi
threads="$(gh api graphql -f query='
query($owner:String!,$name:String!,$number:Int!){
repository(owner:$owner,name:$name){ pullRequest(number:$number){
reviewThreads(first:100){ nodes{ isResolved path comments(first:1){ nodes{ author{ login } url } } } } } } }' \
-f owner="${repo%/*}" -f name="${repo#*/}" -F number="$PR_NUMBER" 2>/dev/null \
| jq -c '[.data.repository.pullRequest.reviewThreads.nodes[]?
| select(.isResolved == false)
| select(.path | startswith(".github/"))
| select((.comments.nodes[0].author.login // "") | test("coderabbit|qodo|greptile|devin|\\[bot\\]"; "i"))
| {path, author: .comments.nodes[0].author.login, url: .comments.nodes[0].url}]' || echo '[]')"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Reviewer checks fail open

L8 treats failures of both required reviewer checks as clean results. Invalid or missing zizmor output becomes zero findings, while a failed GraphQL review-thread query becomes an empty thread list. A scanner crash, API outage, or query error can therefore let the release gate report PASS without checking for workflow findings or unresolved review threads.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

garnet-release-testing Garnet product/release test PR; the Jibril release gate runs and reports here

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants