Verify
Is quality holding? An agent that runs cleanly can still produce degraded output. Verify layers structured signal on top of every run.
Three layers
| Layer | What | Tier |
|---|---|---|
| Checks | Server-side rules: JSON schema, regex, length, required fields, tool calls. No LLM cost | Starter (basic), Team and up (full) |
| Evaluations | AI review of a run against fixed questions: did it achieve its goal, how good was the output, what went wrong. Comes out of your monthly allowance | Starter and up |
| Rubrics | Your own plain-English scoring guide, applied by an LLM judge to a sample of one agent's runs. Separate from the allowance | Starter and up |
Checks and rubrics are opt-in per agent. Evaluations cover every agent on the team once enabled (evaluations_enabled on the team, via the API or MCP). Self-reported scores from your code (scores={...} on run.finish) feed the same distribution.
Drift detection
The signal is not any single score; it is the distribution shifting. The Verify page compares today's average judge score against the trailing two weeks and flags a drop that is well outside the normal day-to-day spread (at least five days of history are needed before it says anything). Team and up.
Alerting on quality is explicit rather than statistical: set a pass threshold on a rubric or a check and route the failing event to a channel. See Verify alerts.
Cost
Checks are free. Evaluations are metered by your plan's monthly allowance, with smart sampling and packs when you need more; the details are on How evaluations work. Rubrics are not metered today: the sample rate on each rubric is the cost dial.