Turns a written spec into failing tests
You wrote the spec in English and the agent said it was done
Spec Lock turns the checkable sentences in your spec into properties that run on every change, and is honest about which sentences were never checkable in the first place.
The problem
The spec became the primary artifact and nothing verifies against it
Work now starts with a written requirement and ends with generated code that claims to satisfy it. Between those two things sits a human reading a diff and deciding whether it matches — the least reliable step in the pipeline, repeated hundreds of times a week. Meanwhile the spec itself drifts, and nobody notices that clause four stopped being true in March.
The insight
Most of a spec is unfalsifiable, and pretending otherwise is the failure
Four modalities cover nearly everything a spec genuinely asserts: must, must never, must happen within a bound, and at most N. Each maps to a property with a counterexample. Everything else — "should feel responsive", "on brand" — is prose, and a tool that quietly generates a test for it produces a green check that means nothing. Spec Lock extracts what it can, runs it, and lists the rest as unmapped so the coverage number is the truth.
Modality extraction over spec clauses into typed properties, checked against observed run records to produce counterexamples with the failing run attached, and an explicit unmapped set so coverage is never overstated.
How it works
Four steps, no data science team
Markdown, a ticket, a policy document. Whatever the team actually writes.
Extracted properties on one side, unfalsifiable prose on the other. The split is the useful part.
Properties check observed runs, so a violation comes with the run that broke it.
When the code stops matching the sentence, the build says which sentence.
Who it is for
Whoever owns the requirement, not the implementation
Teams shipping agent-built features against written requirements, especially where the requirement is a policy someone external cares about.
Pricing
- –Property extraction
- –Coverage report
- –Unmapped listing
- –Continuous checking
- –Counterexamples
- –CI gate
- –Spec drift alerts
- –Self-hosted
- –Custom modalities
- –Policy library
- –SSO
Competition
What exists, and what it does not do
| Who | What they do | The gap |
|---|---|---|
| Cucumber and BDD frameworks | Executable specifications in structured English. | They require the spec to be written in their syntax from the start. Spec Lock reads the document the team already wrote. |
| Property-based testing libraries | Hypothesis, fast-check and friends generate inputs against invariants. | Excellent, and they need a developer to state the invariant in code. The gap between the sentence and the invariant is the whole problem. |
| Guardrails and output validators | Constrain agent output at runtime. | They enforce a schema at request time. Neither checks that behaviour still matches an English commitment made in a document. |
| A reviewer reading both | Current practice. | Works once, does not repeat, and is the step that generated volume has already overwhelmed. |
Extraction from natural language is fragile, and the failure mode is bad in a specific way: a property extracted with a subtly wrong subject produces a green check on a clause it is not actually testing, which is worse than no coverage at all. Reporting the unmapped set honestly mitigates the overstatement but not the misparse. This needs to start in one narrow domain with conventional phrasing — refund and eligibility policies are the obvious first one — and a general-purpose version is probably not reachable.
Market
Sits between the testing budget and the compliance one, which is a real position and an awkward one to price
Strongest where a written policy has an external consequence. Narrower than general testing, with a buyer who feels the pain sharply.