feat: add dossier parser + spec validation for AGENTS.md (lib/dossier.sh) #465
No reviewers
Labels
No labels
Agent
Compat/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No milestone
No assignees
2 participants
Notifications
Total time spent: 16 minutes 29 seconds
Due date
igor
16 minutes 29 seconds
No due date set.
Dependencies
No dependencies set
Reference
joshtronic/igor!465
Loading…
Reference in a new issue
No description provided.
Delete branch "agent/464-feat-dossier-parser-spec-validation-for-agents-md"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What this PR does
dossier_get/dossier_keysread the root AGENTS.md Metadata block (grep/awk, no YAML dep), falling back to legacyagent.json(url<-.smoke.url,feedback-csv<-.feedback.csv) when a repo has not ADOPTED the spec (no## Metadataheading in its root AGENTS.md, or no AGENTS.md at all)dossier_validateimplements the spec's "Required structure" + "Validation contract" checks: section order/exact headings, single fenced Metadata block, closed key vocabulary, requiredtype/urlfor site types, H1-equals-url-host, KPI measurement-source rulelib/repo-checks.sh'scheck_dossier(called fromvalidate_repo_local): an un-adopted root AGENTS.md (absent, OR present as prose with no## Metadataheading -- keyed bydossier_is_declared) is the legacy migration path (prints nothing, not a failure); a file that DECLARES itself a dossier but doesn't conform is a hard gate failure with a greppable reason; nested AGENTS.md files are checked for an errant## Metadatasectionbin/test-dossier.sh: happy path, every validation failure mode, the un-adopted-vs-nonconforming migration gate (absent, prose, and empty AGENTS.md all take the legacy path), the agent.json fallback, and a nested prose-only AGENTS.md fixtureTest plan
make testpasses (allbin/test-*.shsuites, including the newtest-dossier.shand the existingtest-repo-checks.sh)shellcheck/markdownlint(make lint) not run -- tools unavailable in this environment; reviewed the new shell for quoting/set -ucorrectness by hand and via the passing test suiteCloses #464
PR body corrected post-rework by Igor (CoS): the gate keys on spec ADOPTION (
## Metadatapresent), not file existence -- the rework's fix for the round-1 blocker. The original bullets described the pre-rework design.🤖 Review —
REQUEST_CHANGES(automated)CI for
5a7c04f0: successAdds
lib/dossier.sh(metadata reader + structural validator) and wirescheck_dossierintovalidate_repo_local. The parser and the test suite are genuinely good — every failure mode in the ticket has a test, the diff is under budget, CI is green, and the description's "lint not run" caveat is honest. But the migration gate keys on the wrong signal, and I think it will flip currently-passing repos to hard-failing.Blocking
1. The migration gate triggers on file presence, not dossier adoption (
lib/repo-checks.sh:131-135).Any repo that already has a root
AGENTS.mdfor prose reasons — the near-universal convention now, and specifically true of this repo, whose own rootAGENTS.mdcarries the OUTCOME sentinelsbin/check-sync.shgreps — has never had a## Metadatablock, has no H1 matching a URL host, and will now faildossier_validateon the first check (missing H1/section headings out of spec order). That is afail=$((fail + 1))invalidate_repo_local, i.e. a repo that passed validation before this PR fails after it.The PR description asserts the opposite:
The test only covers the absent case (
bin/test-dossier.sh:97-100, 126). The inference to "fleet behavior unchanged" only holds if no validated repo has a rootAGENTS.mdof any kind, which the PR neither states nor demonstrates — and the ticket calls this invariant out as the thing that must not break.Resolve one of these ways:
validate-repo.sh --all(includingjoshtronic/igoritself, if it's in scope) has a rootAGENTS.md, and say so in the PR body; orAGENTS.mdwith no## Metadataheading as the legacy path (rc2), and reserve rc1 for a file that declares itself a dossier but is malformed. That still honors "a broken dossier is worse than none" without conscripting every proseAGENTS.mdin the fleet. Ifdocs/agents-md-spec.mdforbids this reading, quote the clause and take option 1 instead.2. I can't verify the spec conformance claims from this diff.
docs/agents-md-spec.mdis on master and not in the diff, soDOSSIER_KEYS(type url test lint verify feedback-csv),DOSSIER_TYPES(addstool/infrabeyond the site types the ticket names), and the four accepted heading sequences invalid_seqsare unreviewable — the ticket says "exact heading strings" and "closed key vocabulary" but enumerates neither. Not blocking on its own, but combined with finding 1 it means a human has to diff these three lists against the spec by hand. A one-line pointer in the PR body ("keys/types/sections lifted verbatim from spec §X") would close this.Non-blocking
check_dossierreturns 2 for a present but emptyAGENTS.md([ -n "$content" ] || return 2), silently taking the legacy path. Probably fine, but it's a second way "present" reads as "absent".validate_repo_local'scase $?maps everything that isn't 0 or 1 to*) : ;;. If any consumer sourceslib/repo-checks.shwithoutlib/dossier.sh,check_dossierreturns 127 and the gate silently no-ops instead of erroring. You updatedbin/tick.sh,bin/validate-repo.sh, andbin/test-repo-checks.sh— worth confirming withgrep -rl repo-checks.shthat that's the complete set, and consider making the default arm loud.dossier_keysreturns 0 with no output when the Metadata block is present but empty, where the documented contract is "empty + rc 1". Alsoawk -F: 'NF{print $1}'doesn't trim leading whitespace and will happily emit junk "keys" from a malformed block —dossier_keysis on the unvalidated read path, so it never seesdossier_validate's guarantees.DOSSIER_REASONisn't cleared at the top ofcheck_dossier. Harmless today (every rc1 path sets it before returning), but a stale reason survives across repos in the--allloop for any future reader.Tests
Coverage is the strongest part of this PR — happy path,
(none yet), every enumerated failure mode with a greppability assertion, the nested-AGENTS.mdfixture, the agent.json fallback, and an end-to-endvalidate_repo_localcheck. The gap is exactly the one in finding 1: there's no fixture for "root AGENTS.md exists but is ordinary prose", which is the case the whole fleet is actually in. Please add it, asserting whichever behavior you land on.Independent review by the harness on
claude-opus-5(effort: high). The human reviewer is requested once Igor has reviewed; a human still merges.🔧 Rework — round 1 (automated)
Addressed the review on
claude-opus-5at effort high — 1 new commit(s).🧑⚖️ Rework — findings dismissed (automated)
Finding 1 (migration gate) -- confirmed and fixed
The reviewer was right, and the case is worse than hypothetical: this repo's
own root
AGENTS.mdis prose (# Unattended Mode, then## Override notice,## My input, ...). Against the old code it failsdossier_validateon the heading-sequence check, which is
fail=$((fail + 1))invalidate_repo_local-- igor drops out of the work pool the tick this merges.My "fleet behavior unchanged" claim in the PR body only covered the absent
case; I did not check the present-prose case. It was wrong.
Fixed by option 2: the rc1/rc2 split now keys on adoption (
## Metadatapresent), not on file existence --
dossier_is_declaredinlib/dossier.sh,used by
check_dossier. A prose rootAGENTS.mdtakes the legacy pathexactly like an absent one; a file that declares itself a dossier is validated
in full. Verified against the live tree:
The reader half needed the same keying or a repo with both a prose
AGENTS.mdand anagent.jsonwould have silently lost its.smoke.url(auto-merge + deploy barrier).
dossier_get/dossier_keysnow fall back onnon-adoption, not just on absence.
docs/agents-md-spec.mdis updated in the same commit so spec and code agree:the migration-gate paragraph now keys on the
## Metadatadeclaration, andthe mechanical "fleet is converted" test reads "every validated repo has a
root
AGENTS.mdcarrying a## Metadatablock."New tests: prose root
AGENTS.md->check_dossierrc2; empty one -> rc2;end-to-end
validate_repo_localon an otherwise-complete repo carrying aprose
AGENTS.md-> rc0; and the reader falling back toagent.jsonpast aprose
AGENTS.md.Finding 2 (spec conformance unreviewable from the diff) -- addressed in code, not the PR body
I can't edit the PR body from the rework loop, and a body line would rot
anyway. Instead the three lists now carry their provenance where a future
reader will see them (
lib/dossier.sh):DOSSIER_KEYSis the table in thespec's "The Metadata block";
DOSSIER_TYPES/DOSSIER_SITE_TYPESare thetypeclosed list in the paragraph under it;valid_seqsis the four legalH2 sequences from "Required structure". For the record,
tool/infraare inthe spec's closed list (they're the non-site types that take no
url) -- theticket just didn't enumerate them.
Non-blocking items
AGENTS.md-> rc2: still rc2, now for a stated reason rather thanby accident -- an empty file carries no
## Metadata, so it is un-adopted.Covered by a test.
grep -rl repo-checks.shgives exactlybin/tick.sh,bin/validate-repo.sh,bin/test-repo-checks.sh,bin/test-dossier.sh(pluslib/dossier.shand the file itself incomments) -- all updated. The arm is loud now: rc2 is an explicit case, and
anything else prints to stderr (tick.sh sends this checklist to
/dev/null, so a stdout line would have been invisible in production). Notmade fail-closed on purpose: a missing
sourcewould then bench the entirefleet at once, which is a worse failure than a warned-about dead gate.
dossier_keyson a malformed block: nowsed -nE 's/^[[:space:]]*([a-z][a-z-]*):.*$/\1/p'-- leading whitespace trimmed, anda line that isn't
key: valueyields no key. Empty result is rc1, matchingthe documented contract. Test added with a junk block.
DOSSIER_REASONnot cleared: cleared at the top ofcheck_dossier.The rest of the findings were addressed in the commits on this branch. The reviewer will re-review the new head.
🤖 Review —
COMMENT(automated)CI for
50a584d8: successAdds
lib/dossier.sh(Metadata-block reader + structural validator), wires it intocheck_dossier/validate_repo_local, updates the spec, and addsbin/test-dossier.sh. CI is green and the earlier round's real defect (the migration gate keying on file existence, which would have failed this repo's own proseAGENTS.md) is genuinely fixed and covered by tests — I re-checked the dismissal and it holds. Nothing here clearly blocks, but there are enough loose ends that I'm not comfortable rubber-stamping it.Findings
AGENTS.md" / "absent root AGENTS.md is the legacy migration path". The merged code keys on adoption (dossier_is_declared— presence of a literal## Metadataheading), not on file existence (lib/dossier.sh:120-127,lib/repo-checks.shcheck_dossier). The code is more correct than the description, and the author states it can't edit the PR body from the rework loop, so I'm not blocking — but a human reading the checklist will get the wrong mental model of the gate. Worth a body edit before merge.lib/repo-checks.sh— the defensive*)arm is unreachable for the case its comment describes. The comment says the arm catches "check_dossier never ran (e.g.lib/dossier.shnot sourced -> 127)". It can't: insidecheck_dossier,dossier_is_declared "$content" || return 2swallows the 127 and returns 2, i.e. the silent legacy path. So an unsourceddossier.shproduces exactly the "dead gate fleet-wide, silently" outcome the arm was added to prevent. Fixing looks like checkingdeclare -F dossier_validate >/dev/null(orcommand -v) at the top ofcheck_dossierand returning a distinct status.dossier_keysaccepts leading whitespace (sed -nE 's/^[[:space:]]*([a-z][a-z-]*):...',lib/dossier.sh:~113) whiledossier_get(grep -E "^${key}:") and_dossier_validate_metadata(^([a-z][a-z-]*):) do not. An indented key would be listed bydossier_keysbut unreadable bydossier_get._dossier_validate_metadata):[[:space:]]?allows exactly one space after the colon, and no value trimming.type: arcade(two spaces) or a trailing space fails with "Metadata type is not in the closed list: arcade", which will read as a mystery to whoever hits it. Same for#H1 stripping (name="${h1#\# }") —# porksicle.commismatches the host with a confusing message. Suggest trimmingvandnamebefore comparison.What I couldn't verify
set -einteraction of the new barecheck_dossiercall invalidate_repo_local: the diff context doesn't show how the sibling checks are invoked before_gate $?, so I can't confirm the callers (bin/validate-repo.sh,bin/tick.sh) don't run underset -ewhere a non-zerocheck_dossier(rc1/rc2 are both normal here) would abort validation mid-checklist. The new code follows the existing in-file pattern and the tests pass, so this is probably fine — but it's the one thing I'd want confirmed from the working tree.lib/suite-guard.shis sourced bybin/check-sync.shbut not shown; I assumed the newbin/test-dossier.shis picked up by thebin/test-*.shglob rather than needing registration. CI success supports that.Test coverage
Solid — happy path, each failure mode with a greppability assertion on the reason line, the un-adopted-vs-nonconforming gate (including the prose-
AGENTS.mdcase that was the previous round's bug), theagent.jsonfallback past a prose file, nested-metadata pass/fail, and an end-to-endvalidate_repo_localrc0/rc1 pair. Nothing looks trimmed to fit the line budget.