Validation: accept a static site's build+deploy-sha smoke as a test signal (not just unit tests) #385

New issue

Closed

opened 2026-07-15 01:46:17 +00:00 by igor · 1 comment

igor commented

2026-07-15 01:46:17 +00:00

Collaborator

Auto-merge eligibility requires passing validation, and check_test_signal demands a package.json test script (unit tests). That excludes static sites / games with no unit suite — e.g. snail.io (no test script) — even though their real verification is different: the site builds (tsc && vite build, run by CI on the PR) and deploys + passes the deploy-sha "bundle hash" smoke check (the deploy barrier asserting the live <meta name="deploy-sha"> == the merged commit).

Per Josh: "games are still websites — the smoke test is the site deploying and passing that bundle-hash validation." So a deploy-verifiable static site (agent.json .smoke.url + a deploy-sha marker + a build CI runs on pull_request) should satisfy validation for auto-merge without a unit-test script.

Proposed: in validate_repo_local, treat ".smoke.url present + a build/verify step runs on pull_request (existing check_ci_workflow) + emits a deploy-sha marker" as a valid signal, alongside the existing unit-test path.

Nuance to weigh: the deploy-sha smoke verifies propagation (right bundle went live), not full correctness — but the CI build passing on the PR is the pre-merge correctness bar for a static site (a broken build never deploys). So the signal is "build-green on PR + deploy-verifiable," not liveness alone.

Companion repo status (corrected 2026-07-15): snail.io already has the deploy-sha marker — its vite.config.ts injects <meta name="deploy-sha"> at build and the live site serves it (verified: content="1e7299b3…"). So snail.io is immediately smoke-verifiable and will satisfy the new signal the moment this lands — no companion PR needed.

Guardrail note: this expands who can auto-merge, so it's Josh's greenlight (add Agent) + the human merge-gate stays.

Correction (2026-07-15, CoS): the "Companion repo change" note above is stale — snail.io already has the deploy-sha marker. Its vite.config.ts injects <meta name="deploy-sha"> at build and the live site serves it (content="1e7299b3…", verified). No companion PR is needed; snail.io will satisfy the new signal the moment this change lands.

Auto-merge eligibility requires passing validation, and `check_test_signal` demands a package.json `test` script (unit tests). That excludes static sites / games with no unit suite — e.g. **snail.io** (no test script) — even though their real verification is different: the site **builds** (`tsc && vite build`, run by CI on the PR) and **deploys + passes the deploy-sha "bundle hash" smoke check** (the deploy barrier asserting the live `<meta name="deploy-sha">` == the merged commit). Per Josh: *"games are still websites — the smoke test is the site deploying and passing that bundle-hash validation."* So a **deploy-verifiable static site** (`agent.json .smoke.url` + a deploy-sha marker + a build CI runs on `pull_request`) should satisfy validation for auto-merge **without** a unit-test script. **Proposed:** in `validate_repo_local`, treat "`.smoke.url` present + a build/verify step runs on `pull_request` (existing `check_ci_workflow`) + emits a deploy-sha marker" as a valid signal, alongside the existing unit-test path. **Nuance to weigh:** the deploy-sha smoke verifies *propagation* (right bundle went live), not full correctness — but the **CI build passing on the PR** is the pre-merge correctness bar for a static site (a broken build never deploys). So the signal is "build-green on PR + deploy-verifiable," not liveness alone. **Companion repo status (corrected 2026-07-15):** snail.io **already has** the deploy-sha marker — its `vite.config.ts` injects `<meta name="deploy-sha">` at build and the live site serves it (verified: `content="1e7299b3…"`). So snail.io is immediately smoke-verifiable and will satisfy the new signal the moment this lands — **no companion PR needed**. **Guardrail note:** this expands who can auto-merge, so it's Josh's greenlight (add `Agent`) + the human merge-gate stays. --- > **Correction (2026-07-15, CoS):** the "Companion repo change" note above is stale — **snail.io already has** the deploy-sha marker. Its `vite.config.ts` injects `<meta name="deploy-sha">` at build and the live site serves it (`content="1e7299b3…"`, verified). **No companion PR is needed**; snail.io will satisfy the new signal the moment this change lands.

igor commented

2026-07-15 01:50:50 +00:00

Author

Collaborator

Correction: snail.io ALREADY emits a deploy-sha marker (injected in vite.config.ts), so it is already smoke-verifiable — no marker PR needed. Its only gap for auto-merge is the unit-test validation gate, which is exactly what this ticket addresses. (Fleet sweep: the .smoke.url sites still MISSING a marker are certifiedtradejobs.com, parsleymakesitbetter.com, scenekids.com, sharktankdb.com — separate per-repo hygiene tickets filed.)

Correction: snail.io ALREADY emits a deploy-sha marker (injected in `vite.config.ts`), so it is already smoke-verifiable — no marker PR needed. Its only gap for auto-merge is the unit-test validation gate, which is exactly what this ticket addresses. (Fleet sweep: the `.smoke.url` sites still MISSING a marker are certifiedtradejobs.com, parsleymakesitbetter.com, scenekids.com, sharktankdb.com — separate per-repo hygiene tickets filed.)

igor added the

Kind/Enhancement

Priority

Medium

labels

2026-07-15 14:57:15 +00:00

igor self-assigned this

2026-07-15 15:10:39 +00:00

igor referenced this issue from a pull request that will close it,

2026-07-15 15:16:27 +00:00

chore: feat(validation): accept a deploy-verifiable static site as a test signal #390

igor added spent time