Living Specs

The idea in one paragraph

You describe a test in plain language — as precise or as vague as you like. The first time Talaia sees it, an LLM exercises the real app (drives a browser, discovers selectors, confirms the described outcome actually happens) and writes a deterministic Playwright test. From then on that generated test runs on the normal schedule with no LLM in the hot path — fast, cheap, reliable, like any hand-written suite. When it later breaks, Talaia re-reads the natural-language intent and decides: was this cosmetic drift (a button moved, a selector renamed) or did the feature itself stop working? Drift gets healed — the LLM re-derives the mechanics and proposes an updated test for your review. A genuine feature regression is never healed — it alerts you, loudly, because that’s the whole point of monitoring.

The natural language is the durable source of truth. The .py file is a regenerable artifact.

Why this is safe: sacred intents

The obvious failure mode of “self-healing tests” is a machine that, when a test goes red, quietly rewrites it until it’s green again — erasing the exact signal the monitor exists to catch. Talaia avoids this with one structural rule:

A spec’s “expect” clauses compile to sacred intent() assertions. Only the mechanics between them are ever healed. An intent assertion that fails is a real regression, alerted at full priority, and is never auto-rewritten. A mechanic failure is initially unknown, not assumed to be harmless drift.

Mechanics are the steps that get you into position: navigate, log in, click, fill a form. A broken mechanic could be selector drift — or a backend 500, broken JavaScript, or an outage. So it is classified unknown and alerts at full priority.
Intent is the described outcome: “a Client notified toast appears”, “an invalid password shows an error”. If the flow reached an intent assertion and it is false, the feature genuinely misbehaves: regression, alert, never heal.

A mechanic failure is reclassified as drift only after an offline re-drive demonstrates alternative mechanics that reach every unchanged intent.

Three layers

A living spec is a pipeline of three artifacts:

<name>.spec.md — you write this. Human intent, vague or precise. The source of truth.
<name>.plan.md — the LLM’s elaboration into explicit numbered mechanics and explicit “expect” clauses. Committed and reviewed — the stable test contract. Without this layer, a vague spec would be re-interpreted on every regeneration and the intents themselves would drift, making “never heal an intent” meaningless.
generated/test_<name>.py — deterministic Playwright generated from the plan. Runs on the suite’s normal schedule with no LLM at run time. Each intent compiles to an intent() assertion that raises a distinct IntentFailure exception, so failure classification in the hot path is pure exception-type inspection — cheap, deterministic, LLM-free.

Healing re-derives only layer 3 from the unchanged plan. Editing the spec (layer 1) re-runs elaboration — and always goes through review, because the meaning of the test changed.

Everything is review-gated

All generated code — first generation and every heal — lands as a candidate diff stored outside pytest’s collected path and marked pending-review. Until approved:

the previous trusted test keeps running and the original alert stays open;
the candidate never executes on schedule and can never silently clear an alert.

Generation and healing run offline (an explicit CLI or a low-frequency maintenance schedule), never in the scheduled monitoring path — so LLM spend is bounded by how often specs change or drift, not by monitoring frequency.

The planned CLI:

talaia specs list                 # status per spec, fingerprint freshness
talaia specs sync [--all|<spec>]  # elaborate + generate new/changed specs
talaia specs heal [<spec>]        # drain the heal queue → review-gated diffs
talaia specs review               # show pending diffs
talaia specs approve <spec>       # commit + trust a pending change
talaia specs reject <spec>        # discard it, keep the spec degraded

Safety on production

Generated write-flows obey the same golden rules as every Talaia suite, enforced by mechanically restricted capabilities rather than prompt instructions alone: the suite’s dedicated test account only, a mandatory sandbox preflight gate before any write capability, SMOKE- prefixes with mandatory cleanup steps in the plan, notification recipients limited to configured test sinks, and least-privilege browser driving (only the suite’s allowed hosts and credentials — no shell, no raw environment, no unrestricted HTTP client). Irreversible operations stay forbidden outside explicitly reviewed low-frequency suites.