should the reviewer's verdict rubric be re-weighted toward APPROVE? #456
Labels
No labels
Agent
Compat/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No milestone
No assignees
1 participant
joshtronic
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
joshtronic/igor#456
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Split out of #455 at the reviewer's request -- it was ~60 undisclosed lines in a PR about the rework agent, and it changes the merge gate, so it wants your call rather than a quiet ride-along.
The text is on that branch:
git show e388efc -- bin/lib/review-directive.md.What it does: replaces "fail closed, the cost of a wrong APPROVE is far higher than the cost of a wrong REQUEST_CHANGES" with a rule that a defect you can POINT AT blocks and a risk you can only IMAGINE does not, and that "I can only see the diff" is a standing condition rather than a reservation about any particular PR. It also spells out what each verdict actually triggers now (an APPROVE on a shadow-gated repo auto-merges without you; a COMMENT spends your attention).
Why I wrote it: 63% of verdicts land on you, and the common shape is the reviewer correctly noting it cannot verify something from a diff. But it loosens the gate, and I do not think a bot should loosen its own gate on its own say-so.
Second, smaller question while you are here: #455 posts the rework agent's dismissals to the PR as a comment, and I found while reworking it that the reviewer never sees them -- do_review_tick's prompt is title + body + linked issue + CI + diff, no comments. So a dismissed finding can be re-raised next round and the argument only reaches you. Worth feeding bot dismissal comments into the review prompt? Cheap to do, and it is bot-authored text so no new injection surface. Or is "the reviewer re-raises, the agent dismisses again" the correct amount of friction?
Both questions answered by the operator.
1. Re-weight the verdict rubric toward APPROVE — NO. Keep it paranoid. The rubric change made sense when the reviewer was the last word; with #455 merged the rework agent adjudicates, so loosening the reviewer's bar now would just make it surface less. You were right not to grant this yourself.
2. Feed bot dismissal comments into the review prompt — YES. Built in the PR linked below: scoped to bot-authored comments carrying the adjudication marker, fenced as untrusted, and explicitly non-binding so the reviewer can still rebut a bad argument.
Both questions are resolved; closing.
1. Re-weight the verdict rubric toward APPROVE — NO. Keep it paranoid. The rubric change made sense while the reviewer was the last word. With #455 merged the rework agent adjudicates, so loosening the reviewer's bar now would make it surface less, which is the wrong lever. No code change. You were right not to grant this yourself — a bot widening its own merge gate is exactly the thing to route to a human.
2. Feed bot dismissal comments into the review prompt — YES. Shipped in #460 (merged 2026-07-30). Scoped to bot-authored comments carrying the adjudication marker, fenced as untrusted, sentinel-scrubbed, and explicitly non-binding:
bin/lib/review-directive.mdnow states a dismissal never on its own turns a REQUEST_CHANGES into an APPROVE.#460 said
Part of #456rather thanCloseson purpose — at the time only the second question had code behind it, and auto-closing would have taken the first with it unanswered.