UIAO Test Plans — The Catalog, Not the Plans

testing
governance
canon
ci
UIAO has one canonical catalog of test plans. Every plan has a canon entry; every plan’s pass signal is unambiguous. Here is how the catalog works.
Author

Michal Doroszewski

Published

April 17, 2026

TipFirst delivery shipped (2026-04-26)

ScubaGear conformance test plan — UIAO_121 instantiation against the scubagear adapter, the first concrete plan in the catalog. See deliveries.qmd for future plans.

UIAO Test Plans — The Catalog, Not the Plans

A substrate this ambitious needs more than a pile of tests. It needs a catalog — one place you can look to answer the question “what tests govern what?” UIAO has exactly one such place: UIAO_126 — Test Plans Program.

The catalog does not restate the individual plans. It names them, scopes them, and points at their canonical home. Doctrinal templates (UIAO_121, UIAO_123) live as canon specs; executable suites live under tests/ and scripts/. Either way, everything appears in one table.

The entry rules

Three rules keep the catalog honest.

  1. Every plan has a canonical entry. No plans live only on wikis, tickets, or in someone’s head. If a plan governs behavior, it is in the catalog with a document ID or a path.
  2. Every plan declares its pass signal. Executable plans emit an exit code, a JSON report, or a JUnit stream. Doctrinal reviews name the reviewer and the date. A plan whose pass signal is “seems fine” is not a plan; it is an essay.
  3. Retired plans stay in the catalog. They move to Deprecated and carry a pointer to their successor. This is how we keep audit trails intact as the substrate evolves.

What runs in CI

Four of the catalog entries run on every relevant PR as blocking gates:

  • pytest.yml — substrate walker smoke + full impl/ suite
  • schema-validation.yml — every registry against its schema
  • metadata-validator.yml — frontmatter on every canon doc
  • substrate-drift.yml — walker-emitted drift findings

The doctrinal templates apply per-adapter: a new adapter lands with filled copies of UIAO_121 and UIAO_123 in the same PR, and the metadata validator checks their frontmatter. Test results stay inside the provenance chain — no side-channel spreadsheets.

Companion pages

Back to top