The delivery verdict
Which environments enter the training set
A per-batch report the vendor can act on and the lab can rely on, with the exploit named.
Rejected
browse-checkout-exploit
Episodes
70
Alignment
-0.16
Exploit gap
2.80x
Policy
reward-loop
- reward correlates with completion at only -0.16
- a non-completing policy scores 180% above the median completed episode
- reward-loop beats the task without solving it
Rejected
sql-repair-exploit
Episodes
70
Alignment
-0.21
Exploit gap
2.67x
Policy
reward-loop
- reward correlates with completion at only -0.21
- a non-completing policy scores 167% above the median completed episode
- reward-loop beats the task without solving it
Rejected
ticket-triage-exploit
Episodes
70
Alignment
-0.17
Exploit gap
2.83x
Policy
reward-loop
- reward correlates with completion at only -0.17
- a non-completing policy scores 183% above the median completed episode
- reward-loop beats the task without solving it
Accepted9 environments verified