The short version. OpenTelemetry will carry your agent telemetry, and the GenAI semantic conventions mean a span from LangChain and a span from Semantic Kernel describe the same things in the same words. What it will not do is tell you what a run cost, whether a scheduled agent went missing, or whether the output was any good.
That is not a criticism. OTel is a transport and a schema, deliberately, and it is the closest thing the industry has to a neutral way of getting telemetry out of an application. For agents it matters more than usual, because they are built on a dozen different frameworks and nobody wants a separate proprietary SDK for each one.
But "we emit OpenTelemetry" and "we can operate our agents" are different sentences. The gap between them is where the work lives, and it is worth knowing exactly how wide it is before you plan around it.
What OTel gives you
A standard schema. The GenAI semantic conventions define attributes for model requests and responses: the model name, input and output tokens, the operation, and spans for agent steps and tool calls. So a span from LangChain and a span from Semantic Kernel describe the same things in the same words. That is the whole point of a convention.
A standard transport. OTLP is the wire format. You export spans to a collector, or straight to a backend that accepts OTLP, over HTTP or gRPC. No bespoke ingest per tool.
Framework reach without bespoke code. Many agent frameworks already emit OpenTelemetry, or have an instrumentation package that does. When they do, you get traces of agent runs without writing capture code yourself.
This is genuinely useful. A trace of a run, with each model call and tool call as a span and token counts on each, is most of the raw material you need to understand what happened.
What an agent span actually contains
It is worth being concrete, because "agent telemetry" gets used loosely and the shape of the data decides what you can do with it downstream.
A single agent run usually produces a tree, not one span. At the top sits a span for the invocation itself. Underneath it are spans for each model call, each tool call, and each nested agent if one hands off to another. The GenAI semantic conventions name the attributes on those spans, so across frameworks you can rely on finding roughly the same things:
- On a model call span: the system and model name, the operation, and the input and output token counts. This is the raw material for cost, though not cost itself.
- On a tool call span: the tool name and the call's status, so a failing tool is attributable rather than buried in a stack trace.
- On the parent span: the overall status and duration of the run, and the trace id that ties the whole tree together.
Two consequences follow from the tree shape, and both catch people out.
First, spans are not runs. Counting spans tells you how chatty your agents are, not how much work they did. A single run might be four spans or forty depending on how many tools it reaches for, which is also why per-span pricing gets uncomfortable for agent workloads specifically.
Second, the interesting numbers live at the top of the tree but are computed from the bottom. The cost of a run is the sum of its model calls. Whether the run succeeded is not simply whether the root span errored, because an agent can complete cleanly having achieved nothing. Both are roll-ups, and OTel does not do roll-ups.
Where OTel stops
OTel is a pipe and a schema. It deliberately does not decide what the data means operationally. Three gaps show up immediately for agents:
- The run as a unit. A trace is a tree of spans. "This run cost $0.04, succeeded, and scored 3.8" is a roll-up the backend has to compute. OTel carries the spans; it does not define the run-level record you actually operate on.
- Cost. Token counts are in the spans, but a price is not. Turning tokens into money, cache aware, from a maintained rate card per provider and model, is backend work. A raw trace tells you 8,000 tokens, not 4.3 cents on the right cost centre.
- Absence and quality. OTel records what happened. It cannot tell you that a scheduled agent should have run at 14:00 and did not, or that a successful run produced a worse answer than last week. Schedule freshness and output scoring are judgements layered on top of the data, not properties of the data.
None of this is a criticism of OTel. It is doing its job. But "we emit OpenTelemetry" and "we can operate our agents" are different sentences.
Turning traces into answers
The pattern that works: let your framework emit OTel, and send it to a backend that reconstructs the run and adds the operational layer. Concretely, the backend should:
- Assemble the run from the spans, so cost, latency, status and tools roll up per run and per agent.
- Price it server-side from a rate card, cache aware, so the number matches the invoice rather than a client guess.
- Watch for absence by comparing actual runs against the schedule each agent is supposed to keep, and page when one goes missing.
- Score the output with deterministic checks and a judged sample, so a confident wrong answer is caught.
Do that and the trace stops being a debugging artifact you open after an incident, and becomes a live operational signal.
AgentPing accepts OTLP directly: point your existing OpenTelemetry exporter at it and the agent run is reconstructed, priced, schedule-checked and quality-scored, no second instrumentation. See the OpenTelemetry docs, or get started free.