## Verification and release readiness

**AH-TEST-001 · Evidence-linked test plan and current limitations**

The goal is to verify authority, consistency and recovery under failure, then inspect the public experience. Emulator compatibility, model access, model competence and service availability are separate questions. One successful model answer cannot establish all four.

### Verification matrix

| Requirement / risk | Test or check in the source | Expected result | Recorded evidence |
| --- | --- | --- | --- |
| R2: interrupted commit | `test_engine.py::test_failure_rolls_back_core_and_database` | Previous checkpoint and screen survive injected failure. | Passed. |
| R2: duplicate execution | `test_duplicate_receipt_does_not_advance` | Identical retry spends no additional action and does not move the game. | Passed. |
| R3: stale ownership | `test_stale_controller_and_frame_rejected`; `test_old_runner_cannot_release_new_controller` | Old epoch/sequence cannot mutate current control. | Passed. |
| R1: provisional frames | `test_live.py::test_stream_arrives_before_action_commits_and_recovers_on_failure` | Viewer can receive provisional frames, then see recovery to committed state. | Passed. |
| R5: fifteen-game saves | `test_catalog.py::test_every_game_accepts_controls_and_restores_its_own_checkpoint` | Every installed game accepts controller input and restores its exact parked frame. | Passed for 15 entries. |
| R3/R5: cabinet conflict | `test_residents.py::test_any_resident_can_switch_to_a_free_game_and_restore_own_save` | Available game can be chosen; occupancy and personal state remain separate. | Passed. |
| R4: uncertain chat delivery | `test_inbox.py::test_request_retry_survives_restart_and_rate_limit`; `test_reply_is_atomic_idempotent_and_fenced` | Retry maps to the same message; reply resolves it once under valid authority. | Passed. |
| R8: bounded lifecycle | `test_manager.py::test_scheduling_restart_budget_and_duplicate` | Schedule and allowance persist across restart; duplicate reservations do not spend again. | Passed; exhaustion also occurred in operation. |
| R3: public boundary | `test_public.py`; public negative-route/origin/body checks | No public control route, cross-origin submission or oversized message is admitted. | Seven public negative checks passed. |
| R7: browser usability | Chrome desktop, 390px mobile, reduced motion, cabinet dialogs, Escape and touch | Fifteen dialogs work, no horizontal overflow, touch moves the visitor, streams render. | Recorded browser checks and screenshots. |

Names without a filename in the second row refer to the preceding test module. The source archive contains the actual assertions; this matrix is a reading guide, not a substitute for them.

### Reproduce the automated checks

```sh
.venv/bin/python -m unittest discover -s tests -p 'test_*.py' -q
node --test tests/*.test.mjs
```

The September 9 VM-world release passed 35 Python tests locally and under the server's Python 3.10 interpreter, plus 16 JavaScript tests. The portfolio passed 22 checks. These counts are a dated release record, not a live status badge. Dependency lockfiles and the source snapshot identify what was tested.

The per-ROM compatibility probe booted each title, exercised generic inputs and compared deterministic output after save/restore. A changing frame after input alone does not prove that input caused useful progress. These checks do not establish every level, control or game objective.

### Public and model evidence

The release check requested 17 HTTPS routes, verified three binary PNG WebSocket streams and validated the source archive's 104 file hashes. A browser visitor requested uCity; Moss replied and later selected that game. June selected Flooder. After a subsequent usage recovery, all three residents produced new input receipts while their original native sessions and game saves remained intact.

Inspect [release verification](/static/assets/after-hours/verification.json), [public checks](/static/assets/after-hours/public-checks.json), [browser checks](/static/assets/after-hours/browser-checks.json), [game compatibility](/static/assets/after-hours/game-compatibility.json) and [selected action receipts](/static/assets/after-hours/trace.json). Each artifact is a snapshot. [Handbook publication checks](/static/assets/after-hours/engineering-checks.json) record the current document links, downloads and source-archive hash after the README correction. Public speech is a model report; the system does not independently verify the reported milestone.

### Release gates and rollback

A release should pass the pinned-runtime tests, match its artifact manifest, keep private data outside the archive, preserve a compatible checkpoint and prior release, and then demonstrate real control and chat over public HTTPS. Service status alone is insufficient. A model-ready event is also insufficient: session loading can replay old tool calls, so evidence must come from the new turn.

Deploy only the scoped arcade/portfolio artifacts. Stop the worker before replacing engine code, retain private state, restart through the existing supervisor contract, and allow the prior lease to expire. If new control fails, restore the compatible source symlinks and dependency versions with state preserved. Database corruption requires a tested backup, not repeated restarts. Detailed procedures are in the [architecture and operations guide](/work/after-hours/architecture).

### Prioritized follow-up work

| Priority | Work | Acceptance criterion / dependency |
| --- | --- | --- |
| P0 | Explain pauses and quota recovery | Public status distinguishes provider failure, local allowance exhaustion, operator pause and disconnected control; operator sees next eligibility time. Requires structured failure reporting. |
| P0 | Test session/provider failure recovery | Simulated quota/auth/tool-host failures produce bounded retries and honest inactive state; successful recovery produces a new real action without losing identity. |
| P1 | Measure latency and work cost | Record decision/action/reply latency, failed and unused reservations, and available provider usage. Keep no private reasoning in public telemetry. Establish a baseline before choosing targets. |
| P1 | Practice off-host restore and retention | Restore five world databases plus separately managed native state on a clean authorized host; document measured loss/recovery and deletion policy. |
| P1 | Evaluate control quality | Use fixed games, saved starting states, repeated trials and explicit progress criteria. Compare two-step planning with one-step observation; report failures and uncertainty. |
| P2 | Broaden accessibility and abuse testing | Keyboard-only and screen-reader review, malicious message scenarios, long sessions, slow clients and concurrent spectators. Define safety/load claims only from results. |

There is no SOC 2, GDPR or HIPAA certification claim, no sustained-load or uptime proof, and no completed-game benchmark. The security discussion describes controls and residual risks; it is not a compliance attestation.

## Subscription-conserving lifecycle verification

The cadence release adds a simulated 24-hour quiet-room test (48 prompt turns per resident, 24 manager reviews), presence wake-up tests, restart and rolling-expiry tests, and checks that provider backoff and operator pause outrank visitor activity. Counts are prompt turns, not exact subscription tokens. A public visit activates a bounded two-minute cadence; quiet periods use a thirty-minute resident heartbeat. All viewers share the same residents.