LangChain

LangChain reports what a chain does through callbacks. AgentPing ships a callback handler for Python (langchain-core) and TypeScript (@langchain/core); pass it in callbacks or config and every model call, tool call and error inside the chain lands on the active run. LangGraph graphs use the same callback system, so the same handler works there.

Install / enable

Python:

pip install "agentping-io[langchain]"

import agentping
from agentping import AgentPingCallbackHandler
from langchain_openai import ChatOpenAI

agentping.init()

llm = ChatOpenAI(model="gpt-4o-mini")

with agentping.run("support-triage", customer_id="acme-corp"):
    reply = llm.invoke(
        "classify this ticket",
        config={"callbacks": [AgentPingCallbackHandler()]},
    )

AgentPingCallbackHandler(run=None, capture_tool_payloads=True, tool_payload_max_chars=4000). Supported langchain-core versions: 0.3 up to, but not including, 2.0. The handler is plain BaseCallbackHandler, so it goes anywhere LangChain accepts callbacks: invoke(config=...), with_config, an agent executor, or a LangGraph graph.invoke(state, config={"callbacks": [...]}).

TypeScript:

npm install @agentping/sdk @langchain/core

import { ChatOpenAI } from "@langchain/openai";
import * as agentping from "@agentping/sdk";

agentping.init({ apiKey: process.env.AGENTPING_API_KEY });

const run = agentping.run("support-triage", { customerId: "acme-corp" });
const handler = new agentping.AgentPingLangChainCallbackHandler(run);

const llm = new ChatOpenAI({ model: "gpt-4o-mini" });
const reply = await llm.invoke("classify this ticket", { callbacks: [handler] });

await run.finish({ status: "success" });

new AgentPingLangChainCallbackHandler(run?, { captureToolPayloads?, toolPayloadMaxChars? }).

Events

LangChain callback AgentPing event What is recorded
on_llm_end / handleLLMEnd llm_call Provider, model, input and output tokens from usage_metadata (cached and reasoning tokens when the provider reports them), latency.
on_llm_error / handleLLMError llm_call with status: "error" Provider, model, the exception message and class, latency.
on_tool_end / handleToolEnd tool_call Tool name, tool_invocation_id (LangChain's run_id), input and output (capped at tool_payload_max_chars), latency.
on_tool_error / handleToolError tool_call with status: "error" Tool name, input, the exception message and class, latency.
on_chain_error / handleChainError error The exception message and class. The run stays open; finish it with the status you want.

provider is inferred from the model class name (ChatAnthropic gives anthropic), the ls_provider metadata LangChain attaches, or the model name prefix, and falls back to langchain when none of those identify it. Token counts come from LangChain's normalised usage_metadata, so what you see is what the provider reported to LangChain; Anthropic's cache split is passed through as the provider gives it.

Streamed model calls are recorded the same way, from the final on_llm_end, but without a stream flag. Callbacks that are not listed (on_retriever_end, on_chain_end, on_agent_action) emit nothing.

Naming and runs

One handler instance per AgentPing run is the simplest pattern: create the handler after agentping.run(...) and pass it to the top-level invoke. LangChain propagates callbacks down to every nested chain, tool and model call, so one handler at the top covers the whole graph.

In Python, tool_call and error events honour an explicit run=; llm_call events go to the run that is active in the current context. Inside a with agentping.run(...) block those are the same run. If you build handlers for runs that are not the active one (a worker fanning out several runs on one thread), keep the model calls inside the run's with block rather than relying on run= alone.

Source / notes

For span-level traces of a LangGraph graph (node timing, state at each step), export LangChain's OpenTelemetry traces to the OpenTelemetry endpoint; the callback handler and the OTLP route can run side by side.