joshtronic/igor

Watch

CEO digest steering is unreachable: the footer says "comment", the fast path listens for "unassign" #433

New issue

Closed

opened 2026-07-27 20:10:25 +00:00 by igor · 1 comment

igor commented

2026-07-27 20:10:25 +00:00

Collaborator

The problem

A board comment on a CEO digest is not picked up until the next weekly digest composition -- up to 7 days. Live example: joshtronic/idolidle.com#27, commented 2026-07-27 16:06 with an explicit unblock ("I am not a blocker on this, the agent can write the content if you give the direction to do so"). It will not be read until 2026-W32, because idolidle.com is already stamped 2026-W31 in .ceo state.

The CEO already HAS a same-day response path -- it used it today at 12:18 (ceo: acted on 1 answered board question(s) on joshtronic/idolidle.com). Digests just cannot reach it.

Root cause: two conventions that disagree

ceo_file_digest() (lib/ceo.sh) stamps the digest with this footer:

📋 CEO DIGEST -- The weekly board digest. Comment to steer next week's read; close to acknowledge or drop.

But the fast path keys on something else entirely. ceo_answered_question_numbers() selects on the reviewer having UNASSIGNED themselves -- its own comment says "their go-signal: answered, hand it back". ceo_responded_proposal_numbers() likewise requires select(([.assignees[]?.login] | index($r)) | not).

A digest is filed assigned to the reviewer and the footer never mentions unassigning. So the board does exactly what the footer says, and the machinery never sees it. The only consumer of a digest comment is ceo_prior_digest_steering(), whose sole caller is the next weekly composition.

What "fixed" looks like

A board comment on an open digest is acted on at the next CEO pass, the same as an answered question -- not the next week.

Requirements, not an implementation:

The digest's stated convention and the machinery must agree. Either the fast path learns to accept a comment as the go-signal for digests, or the footer starts telling the board to unassign. Whichever way it goes, the footer text and the selector must be changed together, in this ticket. They are currently a trap.
Must NOT be gated by the weekly ISO-week stamp. .ceo[<repo>] (e.g. "joshtronic/idolidle.com":"2026-W31") gates composing a new digest. Responding to steering is a different action and must run regardless -- otherwise this fix lands and idolidle#27 still waits until W32, which is the entire bug.
The digest stays OPEN and stays assigned (if unassignment is not chosen as the signal). ceo_prior_digest_steering() still needs it open for the weekly fold-in, and closing is already defined as "acknowledge or drop".
Idempotence. Whatever the signal, the same comment must not be re-acted on every tick. A CEO reply on the thread is the cheapest watermark -- "a board comment newer than the CEO's last reply is unhandled" -- and needs no new state.
The CEO replies on the thread when it acts, naming what it queued or did. This is a requirement, not a nicety: today the board comments into a channel that never acknowledges, so "acted on" and "ignored" look identical from the outside. That is half of what makes the current behaviour feel broken.
The CEO acts on its own judgment. Once the board has responded, it should file the Agent-labeled work or do the thing -- NOT file a fresh proposal asking the board to approve what it just approved. The human merge gate remains the backstop; a second approval round-trip is the loop this ticket exists to remove.

Acceptance criteria

With an open digest carrying a board comment, a CEO pass acts on it without waiting for the ISO week to roll.
The CEO leaves a reply on that digest naming what it did.
A second CEO pass over the same digest with no new board comment does nothing (no duplicate work, no reply spam).
The digest remains open and is still folded into the next weekly composition.
bin/test-ceo.sh (92 assertions today) covers: the selector matches a commented digest; it does NOT match one already answered by a CEO reply; the weekly stamp does not suppress it; and the footer text matches whatever signal the selector implements.

Live verification

joshtronic/idolidle.com#27 is open right now with an unanswered board comment. Once this lands, that ticket is the end-to-end test -- it should be picked up on the next CEO pass and receive a reply. Do not close it as part of this work.

Explicitly NOT in scope

Cascade starvation. As of 2026-07-27 there were zero cascade-stage lines in a 2.5-hour window -- no tick got deep enough to reach the CEO stage at all, because automerge, deploy-watching, PR review and issue work consumed them higher up. That limits how fast this fix delivers, but it is a separate concern with a separate fix and must not be bundled here.

REQUIRED: authorization gate (added after the first attempt was blocked)

The first implementation was correctly rejected by the harness security review. Read this before rebuilding — it is in the body because the issue-work prompt receives only .body; issue comments are never passed to the model, so the review comment on this ticket is invisible to you.

What went wrong: the pending-steering selector tested .user.login != $bot — i.e. any comment author who is not the bot counted as "the board". Combined with requirement 6 above (file Agent-labeled work directly, no greenlight round), that turned a comment into a way to inject work into the autonomous execution queue. Previously that required adding the Agent label, which is a write action on the tracker. The diff silently converted an authorization gate into a triage stamp.

That is my spec's fault as much as the implementation's: requirement 6 asks for the greenlight round to be skipped, and I did not notice the label was doing double duty as the authorization step.

The requirement:

Gate on the AUTHOR, not on "not the bot". Only the digest's assigned reviewer (or the configured FORGEJO_REVIEWER) counts as board steering — select(.user.login == $reviewer), in BOTH the pending selector and wherever comments are marked NEW --.
A comment from anyone else must never mark a digest as having pending steering, and must never reach the steering prompt. Untrusted text must not land in a prompt whose system message says the board pre-approved it.
Requirement 6 still stands for the reviewer's OWN comments — do not add a second approval round-trip for the person who just approved.
Add a test asserting a non-reviewer comment does NOT trigger steering, alongside the existing coverage.

Severity note for context, not an excuse to skip it: this instance is private with two accounts, so today "any commenter" is effectively the reviewer or the bot. The fix is one predicate and closes the gap before the trust boundary ever widens. Take it.

## The problem A board comment on a CEO **digest** is not picked up until the next weekly digest composition -- up to 7 days. Live example: `joshtronic/idolidle.com#27`, commented 2026-07-27 16:06 with an explicit unblock ("I am not a blocker on this, the agent can write the content if you give the direction to do so"). It will not be read until 2026-W32, because `idolidle.com` is already stamped `2026-W31` in `.ceo` state. The CEO already HAS a same-day response path -- it used it today at 12:18 (`ceo: acted on 1 answered board question(s) on joshtronic/idolidle.com`). Digests just cannot reach it. ## Root cause: two conventions that disagree `ceo_file_digest()` (`lib/ceo.sh`) stamps the digest with this footer: > _📋 **CEO DIGEST** -- The weekly board digest. **Comment to steer** next week's read; **close** to acknowledge or drop._ But the fast path keys on something else entirely. `ceo_answered_question_numbers()` selects on the reviewer having **UNASSIGNED themselves** -- its own comment says "their go-signal: answered, hand it back". `ceo_responded_proposal_numbers()` likewise requires `select(([.assignees[]?.login] | index($r)) | not)`. A digest is filed **assigned to the reviewer** and the footer never mentions unassigning. So the board does exactly what the footer says, and the machinery never sees it. The only consumer of a digest comment is `ceo_prior_digest_steering()`, whose sole caller is the next weekly composition. ## What "fixed" looks like A board comment on an open digest is acted on at the **next CEO pass**, the same as an answered question -- not the next week. Requirements, not an implementation: 1. **The digest's stated convention and the machinery must agree.** Either the fast path learns to accept a comment as the go-signal for digests, or the footer starts telling the board to unassign. Whichever way it goes, the footer text and the selector must be changed together, in this ticket. They are currently a trap. 2. **Must NOT be gated by the weekly ISO-week stamp.** `.ceo[<repo>]` (e.g. `"joshtronic/idolidle.com":"2026-W31"`) gates *composing a new digest*. Responding to steering is a different action and must run regardless -- otherwise this fix lands and idolidle#27 still waits until W32, which is the entire bug. 3. **The digest stays OPEN and stays assigned** (if unassignment is not chosen as the signal). `ceo_prior_digest_steering()` still needs it open for the weekly fold-in, and closing is already defined as "acknowledge or drop". 4. **Idempotence.** Whatever the signal, the same comment must not be re-acted on every tick. A CEO reply on the thread is the cheapest watermark -- "a board comment newer than the CEO's last reply is unhandled" -- and needs no new state. 5. **The CEO replies on the thread when it acts**, naming what it queued or did. This is a requirement, not a nicety: today the board comments into a channel that never acknowledges, so "acted on" and "ignored" look identical from the outside. That is half of what makes the current behaviour feel broken. 6. **The CEO acts on its own judgment.** Once the board has responded, it should file the Agent-labeled work or do the thing -- NOT file a fresh proposal asking the board to approve what it just approved. The human merge gate remains the backstop; a second approval round-trip is the loop this ticket exists to remove. ## Acceptance criteria - With an open digest carrying a board comment, a CEO pass acts on it without waiting for the ISO week to roll. - The CEO leaves a reply on that digest naming what it did. - A second CEO pass over the same digest with no new board comment does nothing (no duplicate work, no reply spam). - The digest remains open and is still folded into the next weekly composition. - `bin/test-ceo.sh` (92 assertions today) covers: the selector matches a commented digest; it does NOT match one already answered by a CEO reply; the weekly stamp does not suppress it; and the footer text matches whatever signal the selector implements. ## Live verification `joshtronic/idolidle.com#27` is open right now with an unanswered board comment. Once this lands, that ticket is the end-to-end test -- it should be picked up on the next CEO pass and receive a reply. Do not close it as part of this work. ## Explicitly NOT in scope Cascade starvation. As of 2026-07-27 there were **zero** cascade-stage lines in a 2.5-hour window -- no tick got deep enough to reach the CEO stage at all, because automerge, deploy-watching, PR review and issue work consumed them higher up. That limits how fast this fix delivers, but it is a separate concern with a separate fix and must not be bundled here. --- ## REQUIRED: authorization gate (added after the first attempt was blocked) The first implementation was correctly rejected by the harness security review. Read this before rebuilding — **it is in the body because the issue-work prompt receives only `.body`; issue comments are never passed to the model, so the review comment on this ticket is invisible to you.** **What went wrong:** the pending-steering selector tested `.user.login != $bot` — i.e. *any* comment author who is not the bot counted as "the board". Combined with requirement 6 above (file Agent-labeled work directly, no greenlight round), that turned a **comment** into a way to inject work into the autonomous execution queue. Previously that required *adding the `Agent` label*, which is a write action on the tracker. The diff silently converted an authorization gate into a triage stamp. That is my spec's fault as much as the implementation's: requirement 6 asks for the greenlight round to be skipped, and I did not notice the label was doing double duty as the authorization step. **The requirement:** 1. Gate on the AUTHOR, not on "not the bot". Only the digest's assigned reviewer (or the configured `FORGEJO_REVIEWER`) counts as board steering — `select(.user.login == $reviewer)`, in BOTH the pending selector and wherever comments are marked `NEW --`. 2. A comment from anyone else must never mark a digest as having pending steering, and must never reach the steering prompt. Untrusted text must not land in a prompt whose system message says the board pre-approved it. 3. Requirement 6 still stands for the reviewer's OWN comments — do not add a second approval round-trip for the person who just approved. 4. Add a test asserting a non-reviewer comment does NOT trigger steering, alongside the existing coverage. Severity note for context, not an excuse to skip it: this instance is private with two accounts, so today "any commenter" is effectively the reviewer or the bot. The fix is one predicate and closes the gap before the trust boundary ever widens. Take it.

igor added the

Kind/Bug

label

2026-07-27 20:10:25 +00:00

igor self-assigned this

2026-07-27 20:11:13 +00:00

igor commented

2026-07-27 20:24:46 +00:00

Author

Collaborator

The harness security review flagged a material issue in this change, so it was NOT pushed:

Findings

Authorization gap: any commenter (not just the board/reviewer) can now trigger auto-filed, Agent-labeled work — lib/ceo.sh:ceo_digest_steering_pending() / bin/tick.sh Path 1c.
- The "pending steering" test is any(.[]?; .user.login != $b and .created_at > $w) — i.e. any comment author other than the bot itself counts as "the board." There is no allowlist against the digest's assigned reviewer (CEO_REVIEWER / .assignees[0]).
- The resulting work is filed via the new ceo_file_agent_work(), which deliberately posts an issue already labeled Agent and skips the proposal/greenlight round ("no separate greenlight needed").
- This crosses a privilege boundary the diff itself removes: previously, turning CEO output into claimable agent work required the reviewer to add the Agent label — a write-permission action on the tracker. Now a comment — a read/comment-permission action, available to any account that can see the tracker — is sufficient to inject work into the autonomous execution queue. On a public or multi-user Forgejo instance this is a direct escalation from "can comment" to "can make the agent do work."
- Compounding it, the comment body is interpolated into the model prompt (ceo_digest_steering_thread → ceo_build_digest_steering_prompt) with a system message that explicitly tells the model "The board already gave the direction — act on it, do not ask them to re-approve it," so untrusted comment text is treated as pre-approved instruction. ceo_codecheck_proposal is a redundancy check, not an authorization check.
- Suggested fix: gate on author, e.g. select(.user.login == $reviewer) (digest assignee or configured CEO_REVIEWER) instead of != $bot, both in the pending selector and in the NEW -- marking.

Non-findings (checked, clean): all printf calls use literal format strings with untrusted data passed as %s args (no format-string injection); all API payloads and prompt data go through jq --arg/--argjson (no JSON/shell injection); no eval, no shell interpolation of comment/model text into commands; no credentials, tokens, or TLS/verification changes; mktemp used correctly in tests.

Address it, then remove Status/Blocked to re-queue. (If the note above says the gate could not complete, that's a transient error -- just re-queue.)

The harness security review flagged a material issue in this change, so it was NOT pushed: **Findings** 1. **Authorization gap: any commenter (not just the board/reviewer) can now trigger auto-filed, Agent-labeled work** — `lib/ceo.sh:ceo_digest_steering_pending()` / `bin/tick.sh` Path 1c. - The "pending steering" test is `any(.[]?; .user.login != $b and .created_at > $w)` — i.e. *any* comment author other than the bot itself counts as "the board." There is no allowlist against the digest's assigned reviewer (`CEO_REVIEWER` / `.assignees[0]`). - The resulting work is filed via the new `ceo_file_agent_work()`, which deliberately posts an issue **already labeled `Agent`** and skips the proposal/greenlight round ("no separate greenlight needed"). - This crosses a privilege boundary the diff itself removes: previously, turning CEO output into claimable agent work required the reviewer to *add the `Agent` label* — a write-permission action on the tracker. Now a **comment** — a read/comment-permission action, available to any account that can see the tracker — is sufficient to inject work into the autonomous execution queue. On a public or multi-user Forgejo instance this is a direct escalation from "can comment" to "can make the agent do work." - Compounding it, the comment body is interpolated into the model prompt (`ceo_digest_steering_thread` → `ceo_build_digest_steering_prompt`) with a system message that explicitly tells the model *"The board already gave the direction — act on it, do not ask them to re-approve it,"* so untrusted comment text is treated as pre-approved instruction. `ceo_codecheck_proposal` is a redundancy check, not an authorization check. - Suggested fix: gate on author, e.g. `select(.user.login == $reviewer)` (digest assignee or configured `CEO_REVIEWER`) instead of `!= $bot`, both in the pending selector and in the `NEW --` marking. Non-findings (checked, clean): all `printf` calls use literal format strings with untrusted data passed as `%s` args (no format-string injection); all API payloads and prompt data go through `jq --arg`/`--argjson` (no JSON/shell injection); no `eval`, no shell interpolation of comment/model text into commands; no credentials, tokens, or TLS/verification changes; `mktemp` used correctly in tests. Address it, then remove `Status/Blocked` to re-queue. (If the note above says the gate could not complete, that's a transient error -- just re-queue.)

igor added the

Status

Blocked

label

2026-07-27 20:24:46 +00:00

igor removed their assignment

2026-07-27 20:24:47 +00:00