Skip to content

docs(garnet): keep the release-testing label off demo pull requests - #47

Merged
jadoonf merged 1 commit into
mainfrom
devin/1788934410-garnet-label-rule
Sep 9, 2026
Merged

jadoonf merged 1 commit into
mainfrom
devin/1788934410-garnet-label-rule

Conversation

@jadoonf

@jadoonf jadoonf commented Sep 9, 2026 •

Copy link
Copy Markdown

Summary

The Jibril release gate runs on the garnet-release-testing label and posts a ten-leg verdict comment on the pull request. That is right for test pull requests (pin bumps, release candidates, gate work, acceptance runs) and wrong for a pull request written to be shared, presented, or offered upstream, where the reader should see the change, the Runtime Review comment, and the pull request's own evidence.

This records the distinction in two places agents read before labelling:

  • .github/garnet-gate/FAILURE_LEDGER.md, rule 2: test pull requests carry the label; demo pull requests never do; to gate the same commits, open a second pull request from the same branch and label that one.
  • AGENTS.md, new section "Garnet pull requests on this fork": the same two kinds, which workflow the label starts, and the note that removing the label does not remove a verdict the gate already posted.

No workflow change. The gate itself is unchanged.

Squash Commit Body

The gate posts its ten-leg verdict on every labelled pull request. A pull
request written to be shared or offered upstream needs its conversation
clear for the reader, so the label stays off it and the same commits get
gated through a second, labelled pull request from the same branch.
Recorded in the ledger rules and in the agent guide.

Checklist

  • Updated the documentation if needed.

Written by an agent (Devin).

Link to Devin session: https://app.devin.ai/sessions/0a9b03641469419c9060f560ca6b7e85
Open in Devin Desktop: https://app.devin.ai/desktop/session/0a9b03641469419c9060f560ca6b7e85?variant=devin
Requested by: @jadoonf


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Runtime evidence (Garnet)

Kernel-recorded execution record for head 5e817672e12ba8ad23f1ae6689bb97038d783479, mirrored verbatim from
the sticky Garnet Runtime Review comment on this PR so reviewers that read only
the description ground in the same bytes. Facts only. Judgment stays with the
reviewer. Cite grounded findings as:

Runtime evidence (Garnet, head 5e81767): <execution chain> → <destination> (<workflow>/<job>) —

Execution record (verbatim mirror)

Execution Profiles recorded for 1 job, triggered by 5e81767

3 destinations
recorded at the kernel by Garnet · 2026-09-09 06:21 UTC

Garnet Jibril Release Gate / reproduce ↗ · 3 destinations
Runner.Worker
└─ bash
   └─ curl (step: "Start the clock")
      ├─ ○ localhost (dns resolver)
      └─ ○ registry.npmjs[.]org

systemd
└─ systemd-resolved
   └─ ○ 8.8.8.8

View this job's Execution Profile in Garnet →


💡 How to read this
Runner.Worker          ← process on a path
└─ npm
   └─ node             ← process that acted
      └─ ○ npmjs[.]org ← observed action

follow a path downward to see what ran and what it did — each path to an observed action is an execution chain

names on the path = processes · ○ = observed action · (…) = context

Workload behaviour (Garnet)

Garnet Jibril Release Gate / reproduce · head eafe967 · 5 workload chains · 2 runner-platform chains

Workload behaviour vs ba67148: +0 −8 destinations
− github[.]com
− codeload.github[.]com
− nodejs[.]org
− node-registry.bit[.]cloud
− pnpm[.]io
− release-assets.githubusercontent[.]com
− unofficial-builds.nodejs[.]org
− npm.jsr[.]io
runner platform: +0 −19 destinations · not workload behaviour

5 workload chains · 2 destinations at head, 8 gone
destination chains Δ vs base reached by
localhost 3 19 → 3 curl, node, sudo
registry.npmjs[.]org 2 72 → 2 curl, node
codeload.github[.]com 0 gone node
github[.]com 0 gone git-remote-http, node
node-registry.bit[.]cloud 0 gone node
nodejs[.]org 0 gone node
npm.jsr[.]io 0 gone node
pnpm[.]io 0 gone node
release-assets.githubusercontent[.]com 0 gone node
unofficial-builds.nodejs[.]org 0 gone node
runner platform · 2 chains · 2 destinations at head, 19 gone · no recorded workflow step
destination chains Δ vs base reached by
8.8.8.8 1 2 → 1 systemd-resolved
localhost 1 7 → 1 sudo
140.82.112.22 0 gone nginx
140.82.113.21 0 gone nginx
140.82.114.21 0 gone nginx
169.254.169.123 0 gone chronyd
185.125.190.56 0 gone chronyd
185.125.190.57 0 gone chronyd
185.125.190.58 0 gone chronyd
192.168.127.1 0 gone blacksmithd, sudo
198.137.202.56 0 gone chronyd
23.143.196.204 0 gone chronyd
23.186.168.128 0 gone chronyd
5.161.94.12 0 gone chronyd
91.189.91.157 0 gone chronyd
api.snapcraft[.]io 0 gone snapd
cloudfront.cdn.snapcraftcontent[.]com 0 gone snapd
nodejs[.]org 0 gone pnpm
productionresultssa15.blob.core[.]windows 0 gone node
registry.npmjs[.]org 0 gone pnpm
unofficial-builds.nodejs[.]org 0 gone pnpm

recorded at the kernel by Garnet · profile view →

Greptile Summary

Summary

  • The documented release-gate workflow cannot open the required second pull request when a demo pull request already uses the same branch and base.
  • Update the instructions to create a distinct gate branch at the same commit before opening and labelling the gate pull request. The matching guidance in AGENTS.md must be updated as well.

Confidence Score: 3/5

Not safe to merge until the documented gate-pull-request procedure can be followed successfully.

A live request confirmed that GitHub rejects a second open pull request using the same head branch and base branch, so the required acceptance workflow is currently unusable.

Files Needing Attention: .github/garnet-gate/FAILURE_LEDGER.md and AGENTS.md

T-Rex T-Rex Logs

What T-Rex did

  • T-Rex produced a proof for a posted P1 finding and linked it to the review comment.
  • T-Rex validated contract behavior by running the GitHub duplicate PR proof script, captured the before/after states, and confirmed GitHub reported a duplicate PR with exit code 0, indicating the expected rejection.

View all artifacts

T-Rex Ran code and verified through T-Rex

Comments Outside Diff (1)

  1. General comment

    P1 The duplicate gate-PR procedure cannot create a second PR

    • Bug
      • FAILURE_LEDGER.md:21-22 and AGENTS.md:357 instruct users to open a second pull request from the same branch to gate the same commits. GitHub rejected a live Create Pull Request request using an already-open PR's exact head branch and base branch with HTTP 422.
    • Cause
      • GitHub allows only one open pull request for a given head repository/branch and base repository/branch combination.
    • Fix
      • Replace the procedure with one that creates a separate gate head branch pointing at the same commit, then opens and labels the gate PR from that distinct branch. Update both mirrored instructions together.

    T-Rex Ran code and verified through T-Rex

Reviews (1): Last reviewed commit: "docs(garnet): keep the release-testing l..." | Re-trigger Greptile

Greptile also left 1 inline comment on this PR.

The gate posts its ten-leg verdict on every labelled pull request. A pull
request written to be shared or offered upstream needs its conversation
clear for the reader, so the label stays off it and the same commits get
gated through a second, labelled pull request from the same branch.
Recorded in the ledger rules and in the agent guide.
@devin-ai-integration

Copy link
Copy Markdown

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration devin-ai-integration Bot added the garnet-release-testing Garnet product/release test PR; the Jibril release gate runs and reports here label Sep 9, 2026
@coderabbitai

coderabbitai Bot commented Sep 9, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@garnet-runtime-review

garnet-runtime-review Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

Execution Profiles recorded for 1 job, triggered by 5e81767

3 destinations
recorded at the kernel by Garnet · 2026-09-09 06:21 UTC

Garnet Jibril Release Gate / reproduce ↗ · 3 destinations
Runner.Worker
└─ bash
   └─ curl (step: "Start the clock")
      ├─ ○ localhost (dns resolver)
      └─ ○ registry.npmjs[.]org

systemd
└─ systemd-resolved
   └─ ○ 8.8.8.8

View this job's Execution Profile in Garnet →


💡 How to read this
Runner.Worker          ← process on a path
└─ npm
   └─ node             ← process that acted
      └─ ○ npmjs[.]org ← observed action

follow a path downward to see what ran and what it did — each path to an observed action is an execution chain

names on the path = processes · ○ = observed action · (…) = context

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

/agentic_review

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

@coderabbitai review

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Jibril release gate · v2.17.0-rc.10 · FAIL

leg result
L1 startup PASS — sensor active, status ok, github steps resolution ok
L2 profile uploaded after 0s · 01a084d4-5433-7115-a2b5-34efcb30aa86 · 4 egress peers, 3 destinations
L3 workload destination present, but attributed to steps ["0. Start the clock"] instead of "Workload with egress"
L4 App comment final after 1s · #47 (comment)
L5 permalink html 200 · api 200 · https://app.garnet.ai/public/runs/34318659037?profile=01a084d4-5433-7115-a2b5-34efcb30aa86
L6 integration shape action 3d47f4a9; sensor version left to the action; explicit token
L7 integration run TS CI run 34318651427 finished success with no profile (no profile is bound to that run): the job was green, the sensor recorded nothing
L8 reviewers zizmor: no workflow files touched · unresolved bot threads on .github/: 0
L9 timing sensor start 14s · workload 1s with sensor vs 1s without · overhead 14s (budget 120s) · profile visible 0s after the job · comment final 1s after the profile
L10 coverage 1 of 6 CI cells instrumented: ubuntu/node 24 · not instrumented: ubuntu/node 22, ubuntu/node 26, windows 1/3/node 22, windows 2/3/node 22, windows 3/3/node 22 · Linux x86_64 only; macOS, Windows, fork PRs and Dependabot runs record nothing
record value
jibril requested v2.17.0-rc.10 · resolved v2.17.0-rc.10
action garnet-org/action@e546567a72e4fede11ec39d6e9f75b539adef22c
run 34318659037 attempt 1 · blacksmith-01m22cmhqemvsx8m34y8qeqbag-8vcpu · pull_request
workflow ref seen by sensor garnet-labs/pnpm/.github/workflows/garnet-jibril-release-gate.yml@refs/pull/47/merge
base → head 16aa2a8 → 5e81767
merge sha at run eafe967 · now eafe967 · profile recorded eafe967 (refs/pull/47/merge)
comparison in comment no previous (first record on this PR)
synthetic merge yes · sensor recorded merge commit eafe967, PR head is 5e81767

Failing legs:

  • L3 workload: destination present, but attributed to steps ["0. Start the clock"] instead of "Workload with egress"
  • L7 integration run: TS CI run 34318651427 finished success with no profile (no profile is bound to that run): the job was green, the sensor recorded nothing

Disclosed, not failing:

  • L6 shape: release.yml keeps 2 Garnet step(s) on macOS jobs; the sensor never records there (CodeRabbit and Qodo flagged this on fork PRs)
  • L6 shape: ci.yml passes the token explicitly, so Dependabot-actor runs get an empty value and record nothing (upstream pull 13270)

Legs map to past failures in .github/garnet-gate/FAILURE_LEDGER.md. Full record in the run's garnet-release-gate artifact.

@qodo-code-review

qodo-code-review Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Grey Divider

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Grey Divider

Tip of the day
💡 Did you know, you can commit Qodo's fix in one click with committable suggestions (GitHub & GitLab)

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Previous reviews

Review updated until commit 5e81767 ⏭️ Skipped

Results up to commit 5e81767 ⏭️ Skipped


No changes from previous review

Grey Divider

Qodo Logo

@jadoonf
jadoonf marked this pull request as ready for review September 9, 2026 06:23
@jadoonf
jadoonf merged commit 704df8a into main Sep 9, 2026
31 of 34 checks passed
@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 5e81767

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Clarify Garnet release-testing labels for demo pull requests

📝 Documentation 🕐 Less than 10 minutes

Grey Divider

AI Description

• Distinguishes release-test pull requests from reader-facing Garnet demos.
• Reserves garnet-release-testing for pull requests requiring Jibril gate verdicts.
• Directs demos toward separate labeled gate pull requests from the same branch.
Diagram

graph TD
  A["Agent guide"] --> B{"PR purpose?"} -->|Test| C["Test PR"] --> D["Release label"] --> E["Jibril gate"]
  H["Failure ledger"] --> B
  B -->|Demo| F["Demo PR"] --> G["Runtime evidence"]
Loading
High-Level Assessment

The documentation-first approach is appropriate because test-versus-demo intent requires human judgment. Automatic workflow enforcement was considered but would need unreliable heuristics or additional metadata; keeping the workflow unchanged avoids blocking legitimate pull requests while clearly documenting the separate-PR gating pattern.

Files changed (2) +17 / -1

Documentation (2) +17 / -1
FAILURE_LEDGER.mdDefine separate labeling rules for Garnet test and demo PRs +8/-1

Define separate labeling rules for Garnet test and demo PRs

• Expands the release-gate rules to require the label for test pull requests and prohibit it on reader-facing demos. It directs maintainers to gate identical commits through a second labeled pull request.

.github/garnet-gate/FAILURE_LEDGER.md

AGENTS.mdAdd agent guidance for Garnet pull request classification +9/-0

Add agent guidance for Garnet pull request classification

• Adds instructions for distinguishing test and demo pull requests before applying 'garnet-release-testing'. It explains the triggered workflow, preferred demo evidence, separate gate PR process, and manual cleanup of existing verdict comments.

AGENTS.md

@qodo-code-review

Copy link
Copy Markdown

PR Code Suggestions ✨

Warning

/improve is deprecated. Use /agentic_review instead (removal date not yet scheduled).

No code suggestions found for the PR.

Comment on lines +21 to +22
To gate the same commits, open a second pull request from the same
branch and label that one.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Use a distinct gate branch

When the demo pull request is already open from this branch to its base, GitHub rejects another pull request with that identical head/base pair. The documented procedure therefore cannot create the required labelled gate pull request, leaving the acceptance path unusable. Create a separate gate branch at the same commit, then open and label that pull request; update the matching instruction in AGENTS.md as well.

Artifacts

Evidence from the check

  • The authored Bash script performs a GET for an open pull request and then submits an identical head/base POST request, showing the duplicate is rejected.

Command output from the check

  • The live GitHub GET response identifies open PR 46 and records its same-repository head branch and `main` base before the duplicate request.

Command output from the check

  • The live GitHub Create Pull Request response returned HTTP 422 with the message that a pull request already exists for the selected head branch, proving the written procedure is not viable.

Command output from the check

  • The command execution log records the proof script, repository working directory, successful expected-rejection assertion, and exit code 0.

View artifacts

T-Rex Ran code and verified through T-Rex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

garnet-release-testing Garnet product/release test PR; the Jibril release gate runs and reports here

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant