Most teams start with offline evals: a fixed dataset, a few scorers, a number that goes up when the new prompt is better. That is the right way to choose what to ship. The trap is believing it is the whole job. A fixed set, by definition, contains the inputs you thought of. Production contains the ones you did not, and that is exactly where agents quietly get worse. So production evaluation is not a nicer version of offline evaluation; it is a different layer with a different purpose.
Here is how the layers stack.
Layer 1: deterministic checks, on every run, free
Before any model is involved, a lot of "is this output acceptable" is just assertions. Did the run complete or error. Is the JSON valid and do the required fields exist. Is the length within bounds. Did it cite a source at all. Did it avoid a banned phrase.
These cost nothing, add no latency, and run on every single run, not a sample. They catch the structural breakage (a schema change, an empty output, a truncation) that is both common and unambiguous. Start here, because it is the cheapest signal with the highest precision.
Layer 2: the judged sample
Deterministic checks cannot tell you whether the answer was good, only whether it was well-formed. For the judgement call you need a model in the loop: an LLM-as-judge that scores the output against a rubric. The key decisions:
- Goal-based vs rubric-based. The lowest-friction version scores whether the run achieved its stated goal. A richer version scores against a rubric you write in plain English (cites a real source, answers the question, stays on policy). Both work; the goal-based version needs nothing but a goal on the run.
- Sample, do not score everything. Judging every run is expensive and unnecessary. Score a sample, weighted toward new and unusual runs, because the thousandth identical success tells you nothing the first one did not.
- Show the reasoning. A score with no explanation is not trustworthy. The judge should show why, so a human can disagree, and that disagreement should sharpen future scoring.
Layer 3: drift, on the trend
A single judged run is noisy. The signal is the distribution over time. Quality drift is a slide in the score across many runs after a prompt change, a model upgrade or a temperature tweak, and you only see it by trending the score per agent and tying the trend to the changes you shipped. A drop on the day a change lands, with the runs to confirm it, is the alert worth having.
What to evaluate: black-box, trajectory, step
For agents specifically, there are three depths:
- Black-box: score the final output. Simplest, and often enough.
- Trajectory: judge the path the agent took, not just the answer, useful when the how matters (did it call the right tool, in the right order).
- Step-level: score individual steps, for debugging a specific failing node.
Most production monitoring lives at black-box and trajectory. Step-level is a debugging tool, not a standing signal.
The honest minimum
You do not need an eval pipeline to start. The honest minimum for a production agent is: deterministic checks on every output, a goal-based judged score on a sample, and drift detection on the trend. That catches structural breakage, declining answers, and regressions after a change, which are the three quality failures that reach customers. Offline evals and golden datasets make you better at choosing what to ship; production evaluation tells you it is still true once real traffic hits it.
AgentPing runs all three layers out of the box: unlimited deterministic checks on every run, goal-based LLM-as-judge on a sample with the reasoning shown, and drift detection on the score distribution, no scorers to write. See Verify, or get started free.