OpenAI

Auto-instrument the OpenAI SDK. Every chat.completions.create and responses.create call inside an active run emits an llm_call event with model, token usage, cached and reasoning tokens, latency, finish reason, and tool calls. Cost is computed server-side from the rate card.

Install / enable

Python:

pip install "agentping-io[openai]"

import agentping
from openai import OpenAI

agentping.init()
agentping.instrument_openai()

client = OpenAI()

with agentping.run("support-triage", customer_id="acme-corp") as r:
    reply = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "classify this ticket"}],
    )
    r.finish(scores={"confidence": 0.88})

instrument_openai() patches the Completions and Responses resource classes for both openai.OpenAI and openai.AsyncOpenAI. Idempotent and global; call it once at module load. Azure OpenAI (AzureOpenAI) goes through the same classes; the event's provider is openai and model carries whatever the response reports. Supported openai versions: 1.30 up to, but not including, 3.0.

TypeScript:

npm install @agentping/sdk openai

import OpenAI from "openai";
import * as agentping from "@agentping/sdk";

agentping.init({ apiKey: process.env.AGENTPING_API_KEY });

const run = agentping.run("answer-bot");
const openai = agentping.instrumentOpenAI(new OpenAI(), { run });

const reply = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "hi" }],
});

await run.finish({ status: "success" });

instrumentOpenAI returns a wrapped client whose return types match the original; reply.choices[0].message.content works as before. Pass { run, mode: "batch" } to price calls made through the Batch API at the batch rate.

What's captured

Field Chat Completions Responses API
provider openai openai
model response.model response.model
input_tokens usage.prompt_tokens (already gross) usage.input_tokens
output_tokens usage.completion_tokens usage.output_tokens
cached_input_tokens prompt_tokens_details.cached_tokens input_tokens_details.cached_tokens
reasoning_tokens completion_tokens_details.reasoning_tokens output_tokens_details.reasoning_tokens
finish_reason choices[0].finish_reason status, or incomplete_details.reason when the response is incomplete
tool_calls Number of message.tool_calls Number of function_call items in output
latency_ms Wall-clock time of the call Wall-clock time of the call
stream true on streamed calls true on streamed calls

OpenAI has no cache-creation concept, so cache_creation_input_tokens is never set. Zero-valued fields are omitted.

A call that raises is recorded as an llm_call with status: "error", error and exception, then re-raised. Function-call exchanges stay one llm_call per round trip; the tool executions themselves come from the agent framework or your own tool_call events.

Streaming

Both SDKs capture streamed calls on both APIs. The llm_call fires once the stream is fully consumed or closed, with stream: true.

Chat Completions only sends usage on the final chunk when you ask for it. Without stream_options, the streamed event has the model, latency, finish reason and tool-call count but no token counts:

with agentping.run("answer-bot"):
    stream = client.chat.completions.create(
        model="gpt-4o-mini",
        stream=True,
        stream_options={"include_usage": True},
        messages=[{"role": "user", "content": "explain briefly"}],
    )
    for chunk in stream:
        ...
    # llm_call is emitted here, once the stream is exhausted

Responses API streams carry usage on the response.completed event, so no extra option is needed.

Prompt capture and time to first token

Streamed calls on both APIs also report ttft_ms, the time from the request to the first delta carrying text or a tool call. Pulse charts its p95 per agent.

Prompt and completion text are off by default. Turn them on once at init, with an optional cap per piece (default 4000 characters):

agentping.init(capture_llm_content=True, llm_content_max_chars=4000)

agentping.init({ captureLlmContent: true, llmContentMaxChars: 4000 });

Chat Completions calls then carry input (the messages list, system message included, as the API takes it) and output (the first choice's content). Responses calls carry input (the input parameter, a string or items), system_instructions (the instructions parameter) and output (output_text). Images and tool parts appear as markers such as [tool_call lookup], and each piece is clipped at the cap. The run page renders them as messages; see events for the wire shape. In Go, pass instrumentation.WithContentCapture(4000) to the transport instead.

Source / notes

Using the OpenAI Agents SDK? Model calls are captured by the same wrapper; see the OpenAI Agents SDK page for handoffs and tool calls. Unknown models still land with their token counts and appear in the unpriced-models list until you add a rate under Spend, Rate cards. See Spend.