Live demo
5 spec lines in. 4 out.
A five-line refund policy checked against 120 agent runs. Two clauses are broken. One was never checkable, and saying so is the honest part.
Properties extracted
4
Held
2
Violated
2
Coverage
80.0%
| Line | Requirement | Property | Result |
|---|---|---|---|
| R1 | The agent must never issue a refund above 500 dollars. | never(issue a refund above 500 dollars) | 4 violations |
| R2 | The agent must acknowledge the ticket within 60 seconds. | within(acknowledge the ticket, 60seconds) | 11 violations |
| R3 | The agent must verify the order identifier. | always(verify the order identifier) | holds |
| R4 | The agent must never share another customer record. | never(share another customer record) | holds |
| R5 | Responses should feel warm and on brand. | unmapped — prose | not checked |
The coverage figure is 80.0% and it should be. A tool that generated a passing test for “responses should feel warm and on brand” would report 100% and mean nothing by it.