Skip to content

Commit d6c578c

Browse files
fix: update AI review score thresholds and improve documentation
- Change AI approval threshold from 90 to 85 - Update corresponding messages for approval and rejection - Clarify instructions in the repair documentation
1 parent 67e4fae commit d6c578c

File tree

3 files changed

+12
-13
lines changed

3 files changed

+12
-13
lines changed

.github/workflows/impl-review.yml

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -129,7 +129,7 @@ jobs:
129129
uses: anthropics/claude-code-action@v1
130130
with:
131131
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
132-
claude_args: "--model opus"
132+
claude_args: "--model sonnet"
133133
allowed_bots: '*'
134134
prompt: |
135135
Read `prompts/workflow-prompts/ai-quality-review.md` and follow those instructions.
@@ -217,8 +217,8 @@ jobs:
217217
# Add verdict label early to ensure it's set even if later steps fail
218218
# This is idempotent - re-running won't cause issues
219219
220-
if [ "$SCORE" -ge 90 ]; then
221-
echo "::notice::Adding ai-approved label (score >= 90)"
220+
if [ "$SCORE" -ge 85 ]; then
221+
echo "::notice::Adding ai-approved label (score >= 85)"
222222
gh pr edit "$PR_NUM" --add-label "ai-approved" 2>/dev/null || {
223223
echo "::warning::Failed to add ai-approved label, retrying..."
224224
sleep 2
@@ -232,7 +232,7 @@ jobs:
232232
gh pr edit "$PR_NUM" --add-label "ai-approved"
233233
}
234234
else
235-
echo "::notice::Adding ai-rejected label (score < 90)"
235+
echo "::notice::Adding ai-rejected label (score < 85)"
236236
gh pr edit "$PR_NUM" --add-label "ai-rejected" 2>/dev/null || {
237237
echo "::warning::Failed to add ai-rejected label, retrying..."
238238
sleep 2
@@ -467,8 +467,8 @@ jobs:
467467
ISSUE_NUMBER: ${{ steps.pr.outputs.issue_number }}
468468
run: |
469469
# Score thresholds:
470-
# >= 90: Excellent (approved, merge immediately)
471-
# < 90: Rejected (repair loop, up to 3 attempts)
470+
# >= 85: Approved (merge immediately)
471+
# < 85: Rejected (repair loop, up to 3 attempts)
472472
# After 3 attempts: >= 50 = merge, < 50 = close PR
473473
474474
# Check if verdict label was already added by earlier step
@@ -492,10 +492,10 @@ jobs:
492492
fi
493493
494494
# Fallback: Add verdict labels if not already set (shouldn't happen but ensures robustness)
495-
if [ "$SCORE" -ge 90 ]; then
495+
if [ "$SCORE" -ge 85 ]; then
496496
# Approved - merge immediately
497497
gh pr edit "$PR_NUM" --add-label "ai-approved" 2>/dev/null || true
498-
echo "::notice::Added ai-approved label (score $SCORE >= 90)"
498+
echo "::notice::Added ai-approved label (score $SCORE >= 85)"
499499
echo "Triggering impl-merge.yml for approved PR"
500500
gh workflow run impl-merge.yml -f pr_number="$PR_NUM"
501501
@@ -569,7 +569,7 @@ jobs:
569569
fi
570570
571571
else
572-
# Score < 90, still have repair attempts left
572+
# Score < 85, still have repair attempts left
573573
gh pr edit "$PR_NUM" --add-label "ai-rejected"
574574
if [ "$SCORE" -lt 50 ]; then
575575
gh pr edit "$PR_NUM" --add-label "quality-poor"

prompts/workflow-prompts/impl-repair-claude.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,8 @@ Read both sources to understand what needs to be fixed:
2626

2727
1. `prompts/library/{LIBRARY}.md` - Library-specific rules
2828
2. `plots/{SPEC_ID}/specification.md` - The specification
29-
3. `prompts/quality-criteria.md` - Quality requirements
29+
30+
**Do NOT re-read `prompts/quality-criteria.md`** — the review already distilled all criteria into `review.criteria_checklist` in the metadata YAML (Step 1). Use that checklist directly: items with `passed: false` are the ones to fix.
3031

3132
## Step 3: Read current implementation
3233

tests/unit/api/test_debug.py

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -177,9 +177,7 @@ def test_debug_status_missing_tags(self, db_client) -> None:
177177
spec_id="no-tags", title="No Tags", impls=[impl], tags={"plot_type": [], "domain": []}
178178
)
179179
# Spec with proper tags
180-
spec_with_tags = _make_spec(
181-
spec_id="has-tags", title="Has Tags", impls=[impl], tags={"plot_type": ["scatter"]}
182-
)
180+
spec_with_tags = _make_spec(spec_id="has-tags", title="Has Tags", impls=[impl], tags={"plot_type": ["scatter"]})
183181

184182
mock_repo = MagicMock()
185183
mock_repo.get_all = AsyncMock(return_value=[spec_empty_tags, spec_with_tags])

0 commit comments

Comments
 (0)