Skip to content
GitHub
Decisions

ADR-069: Trigger-driven `spectral.core` re-audit

Context

The spectral.core admission discipline (ADR-065 + ADR-068) has three layers: per-addition articulation (a one-line module docstring stating each module’s functional-area-membership rationale, enforced by ruff D100/D104); structural enforcement (the validator’s slug-identified rule set per ADR-097); and per-PR judgment (the founder reviews each addition for “no owner context” plausibility against the docstring rationale).

What none of these catch is cumulative drift — modules that pass per-PR judgment individually but, viewed together over months, fail the killer test: “if spectral.core did not exist, where would this go?” The PR-statement governance this discipline replaced (ADR-024) failed precisely here — every type in spectral.core failed the killer test under inspection, and every one of those had passed individual PR review. A periodic re-audit is the structural backstop against that failure mode reasserting itself.

The re-audit is trigger-driven, not calendar-based. A calendar cadence drifts: a quarter’s audit lands late, lighter, or skipped because the calendar didn’t reflect actual kernel activity, and it fires against a stable kernel in quiet quarters. A trigger-driven model fires when there is something to look at — which eliminates the drift risk and removes the silent-quarter case where a calendar audits a kernel that hasn’t changed. The validator’s structural coverage (ADR-097 + ADR-068’s class-internal method-signature checks) also narrows the audit’s residual judgment scope: each layer the lint enforces is one fewer thing the audit must catch by hand.

Decision

D1 — Audit cadence is fully trigger-driven; no calendar floor

The kernel re-audit fires when one or more audit-candidate signals are present, not on a calendar. Candidates are surfaced by tools/quality/check_kernel_audit_candidates.py, which runs in CI and prints a candidate list to the workflow log on every push to main.

Audit-candidate signals (any one fires):

  • Module-count growth. Any core/<subdir>/ has grown by ≥25% in module file count since the last audit recorded in docs/audits/.
  • Module added or removed. Any core/<subdir>/ has had a module file added or removed since the last audit, regardless of percentage.
  • Substantial body change. Any core/<subdir>/ module has had ≥30 lines added or removed (cumulative git log --numstat since the last audit) on files other than __init__.py.
  • Stale admission rationale. Any core/<subdir>/ module’s body has changed since the last audit but its module docstring (admission rationale) has not. This catches the “code drifts but the rationale freezes” failure mode that ADR-024 governance also missed.
  • No prior audit on record. If docs/audits/ is empty, every core/<subdir>/ is a candidate. The first audit seeds the baseline.

When candidates surface, the responsible owner (D5) runs the audit per the procedure (D4) and lands a new record.

D2 — Audit records live in docs/audits/ (tracked)

Audit records live at:

docs/audits/<YYYY-MM-DD>-core-reaudit.md

The directory is tracked (an earlier planning/audits/ location was gitignored, which would defeat trigger-driven scheduling — the trigger tool reads the records to determine what changed since the last audit). A tracked docs/audits/index.md (created at first audit, not pre-created by this ADR) optionally lists past audits with one-line descriptions, for human browsability; the trigger tool does not depend on it — it discovers records by globbing docs/audits/*-core-reaudit.md.

D3 — The trigger tool is a reporter, not a CI gate

check_kernel_audit_candidates.py prints candidates to stdout and exits 0 regardless of how many candidates surface. It does not fail CI. The audit is a judgment-required activity; the tool surfaces the work, the owner does it.

Failing CI on candidate-presence would force every PR that touches core/<subdir>/ to wait on an audit — re-creating exactly the calendar-discipline-becomes-paperwork failure mode the trigger-driven model is supposed to eliminate. The tool runs in the existing architecture CI job and captures its output as a workflow-log annotation; owners read it when reviewing CI status, and downstream tooling can scrape for notification purposes (out of scope here).

D4 — Re-audit procedure (the killer-test walk)

  1. Walk each core/<subdir>/ in turn (events, auth, db, retention, llm, embeddings, tools).
  2. For each Python module file (including __init__.py), re-apply the killer test — “if spectral.core did not exist, where would this go?” — using the module’s admission-rationale docstring as the starting input.
  3. Categorize each entry:
    • Retain. Killer test still has no clean single-context answer; remains in core/<subdir>/.
    • Relocate. Killer test has a clean single-context home now (the consumer pattern shifted, or the type’s usage narrowed); mark for relocation to <context>.contracts.* or to a context’s domain/application/infrastructure layer.
    • Refine. Admission rationale needs sharper articulation (docstring update); no relocation.
  4. For each Relocate, mint or queue the relocation as a refinement-track item per the epic-template doctrine. Relocations are real code moves, not paperwork.
  5. Land the audit-record document (docs/audits/<YYYY-MM-DD>-core-reaudit.md) noting items audited, retentions, relocations, and refinements — roughly one paragraph per core/<subdir>/. Future audits start from the previous record.

D5 — Responsible owner

Founder under current operational reality. Re-evaluate ownership when team scaling materially shifts the landscape.

Alternatives considered

  • A quarterly calendar cadence. Drifts when a quarter’s audit lands late or is skipped because the kernel is stable; audits a stable kernel anyway in quiet quarters. Rejected — the audit’s value is in re-examining when there is something to re-examine, not in hitting a date. (If the re-examination-on-a-floor posture is ever wanted back, this ADR is the supersession point.)
  • Halve the cadence to biannual + keep triggers as overrides. Preserves a cadence floor; rejected on the same grounds.
  • Per-PR governance only (no periodic audit). This was ADR-024’s mode; failed on cumulative drift (the orchestration that produced ADR-065 is the evidence). Rejected.
  • Continuous via lint only. Lint enforces structural and import constraints (ADR-097’s rule set); killer-test judgment is irreducibly human (intent + actual usage patterns, not type shape). Lint cannot replace the audit, nor the audit the lint; both are layered.
  • Trigger on spectral.core line-count growth instead of module count. Module count is a cleaner proxy for “new admission decisions made” than LOC, which can grow within a single module without a new admission. Rejected for the primary growth signal (a body-change signal covers in-module growth separately, D1).
  • Move the trigger tool into validate_architecture.py --show-audit-candidates. Couples the validator (a CI gate that fails closed) with the audit reporter (informational, never gates); the two have different purposes. Standalone tool keeps the separation clean. Rejected for now; revisitable if two-tool maintenance surfaces a real cost.
  • Make the trigger tool a hard CI gate. Recreates the calendar-becomes-paperwork failure mode (every core/-touching PR blocks on audit completion). Rejected.
  • Track audit records via git tags rather than a tracked directory. Cleaner for “when did audit X happen,” but adds tag-management overhead and still needs a separate home for the audit content. Rejected; the tracked directory carries both the date and the content.

Consequences

  • No calendar reminder is needed. Owners check CI workflow logs for audit-candidate notices when the kernel has visibly changed (which is when the candidates fire).
  • The cumulative-drift defense remains intact. Stale-admission-rationale detection is the structural backstop against the failure mode ADR-024 governance was designed to catch: when a module’s body changes but its admission docstring does not, the trigger fires and the audit re-examines whether the module still belongs in core.
  • Silent zero-audit periods are an explicit feature. If the kernel sits stable for six months, no candidates fire and no audit is needed — the model trusts that “nothing changed” means “nothing to re-examine.”
  • Relocations identified during audits feed normal refinement-track work; they are not a separate “audit follow-up” track.
  • DX cost is bounded. The trigger tool runs in CI; pre-push is unchanged. The audit itself is the only judgment cost.