---
title: How evaluations work
description: AI review of whether each run achieved its goal, out of the box, spread sensibly across your month.
section: billing
order: 2
---

# How evaluations work

An evaluation is an AI review of whether a run achieved its goal. There
is nothing to configure: no scorers to write, no prompts to tune. Give
your runs a goal (the SDK's `goal` field, or `agentping.goal` on
OpenTelemetry traces) and AgentPing reviews the outcome.

Two layers run on every account:

- **Health checks**: automatic, on every run, on every plan, no limits.
  Did it complete, did it error, was it unusually slow or expensive, did
  a tool spin in a loop, did it produce output.
- **Evaluations**: the AI review. Each plan includes a monthly
  allowance: 1,000 on Starter, 6,000 on Team, 20,000 on Business.

## How your allowance is spent

- **Spread evenly across the month.** Evaluations pace themselves so
  coverage is steady rather than spent in the first week.
- **Smart sampling.** Routine healthy runs don't use up your allowance:
  when an agent keeps doing the same thing and keeps passing, AgentPing
  samples it lightly and concentrates evaluations on new and unusual
  runs. These runs show as covered by routine baseline.
- **Multi-agent fairness.** When you have multiple agents, no single
  agent can use more than half your allowance.
- **Returned when we fail.** If an evaluation fails on our side, it is
  returned to your allowance automatically.

## Tell us when we get it wrong

See a result you disagree with? Tell us with one tap on the run's
evaluation panel. It directly improves how we evaluate your runs.

## When the allowance runs out

Health checks continue, unlimited. Evaluations pause until next month,
a [top-up](/docs/billing/top-ups), or an upgrade. Nothing else changes.

## Per-team control

Each team has an "Evaluate runs" switch. Teams you create later (a
staging team, a client team) start with it off, so experiments never
consume your allowance unless you choose.

## FAQ

**Why wasn't this run evaluated?** Either its team has evaluations
switched off, it was covered by the routine baseline, or the month's
allowance was already spent at the time.

**Can I evaluate every run?** If your volume is near your allowance,
top-ups cover a busy month; if it is consistently above, the next tier
is cheaper.

**Do top-ups roll over?** No; they apply to the current month only.

**How long do results take?** Most evaluations appear within the hour;
the run shows "Evaluation in progress" until then.