Skip to content
Spectral Codex
GitHub
Test Failures

Fabricated fixture values keep unit suites green on wire-impossible assumptions — derive fixtures from the real enum/wire values

Fabricated fixture values mask wire-impossible assumptions

Problem

The redesigned cockpit rail (SPEC-607/608) filtered worlds with world.status === "active". The real worlds.world_models.status machine is draft | published | retired (WorldModelStatus) — "active" never appears on the wire — so the per-ruleset rail list rendered empty for every real world. The operations unit suite stayed green because its fixtures also fabricated status: "active", and a stale doc comment in api/worlds.ts (status → "active" | "retired") corroborated the wrong value for both the implementer and the independent reviewer. The rewritten qa-replay scenario (navigation-ia 4, asserting the created world appears in the rail) caught it at the merge gate.

Root Cause

Fixture values were invented to look plausible instead of being derived from the producing enum, and a wrong client doc comment laundered the invented value into “documented fact”. Self-consistent fiction: component, fixture, and comment all agreed with each other and disagreed with the wire.

Solution

Fix 8141210e: rail filters status !== "retired"; the stale "active" doc comments in worlds.ts / worlds-list.tsx / world-detail.tsx, the never-matching green status pill, and every fabricated fixture row were corrected to real values.

Prevention

  • When frontend code branches on a wire enum value, verify the literal against the producing StrEnum / response model — not against a client doc comment (those drift; two were wrong here).
  • Fixture values for enums must be copied from the domain enum, never invented. If a fixture literal can’t be grepped in the producing code, it’s fiction.
  • A filter/branch on a wire enum deserves one test on the excluded value using real values — the rail had no test for “a draft world appears, a retired one doesn’t”.
  • This is exactly the failure class the per-merge local qa-replay gate exists for: a live-driving scenario asserts the data actually round-trips, which no amount of self-consistent unit fixtures can.

The qa-seed corollary: fixtures must match the producer shape AND seed every table a newly-consumed read composes

The same class recurs in the qa seed fixtures, not just unit fixtures — and a new UI surface is usually the first thing to expose it, because it’s the first consumer of a read. Three instances in one wave (dashboard Wave B, 2026-06-10):

  1. Wrong producer key. tools/dev/qa_customer_seed.py hand-rolled the rule_set_snapshot rule entry with key rule_id, but the producer (worlds … world_model_state_snapshotter.RULE_SNAPSHOT_RULE_KEYS) and the customer reader key on id. The reader silently dropped the rule, so “The rules” (SPEC-623) rendered empty against a domain reporting total_rules=1. Fix: derive the seed entry from the producer key tuple, not by hand.
  2. Incomplete seed for a newly-consumed read. The same seed populated platform.world_model_card_projection but not worlds.world_model_card — the body the customer World Model Card read (SPEC-646) composes — so the read 404’d and the SPEC-647 surface showed its empty state. The projection seed had been “enough” only because nothing read the worlds table yet. Fix: seed every table the read composes (mirror the read’s own SQL / the read’s integration-test seed).
  3. Empty-state-only fixtures. The customer dashboard had no seeded decision, so the log / deep-dive / attention surfaces were only ever exercised against the empty state (and the deep-dive qa scenarios skipped-with-reason). Firing a couple of real /decide calls in the seed made the surfaces render real data and the deep-dive verify live.

Generalize: a qa fixture is a contract with the producers a surface reads from. When a wave adds the first consumer of a read, audit the fixture against that read’s actual SQL — wrong key, missing table, or empty-only data all present as “renders the empty state” and are invisible to unit suites. Mirror the read’s integration-test seed where one exists.

References

  • Merge gate catch + fix: 8141210e (Wave 0 C-A, SPEC-607/608)
  • Status machine: src/spectral/worlds/domain/authoring/types.py::WorldModelStatus
  • qa-seed corollary fixes (Wave 1 dashboard): rule_id→id, worlds.world_model_card body seed, and the /decide seed in tools/dev/qa_customer_seed.py
  • Producer key tuple: RULE_SNAPSHOT_RULE_KEYS in world_model_state_snapshotter.py