---
title: LlamaIndex
description: Register the AgentPing callback handler with LlamaIndex so model calls, embeddings, retrievals and exceptions land on one run.
section: frameworks
order: 5
---

# LlamaIndex

LlamaIndex reports what a query engine, chat engine or agent does through its callback manager. AgentPing ships a handler for Python (`llama-index-core`); register it once on `Settings` and every model call, embedding batch, retrieval and exception inside LlamaIndex lands on the active run. Python only.

## Install / enable

```bash
pip install "agentping-io[llamaindex]"
```

```python
import agentping
from agentping import AgentPingLlamaIndexHandler
from llama_index.core import Settings, VectorStoreIndex
from llama_index.core.callbacks import CallbackManager

agentping.init()
Settings.callback_manager = CallbackManager([AgentPingLlamaIndexHandler()])

index = VectorStoreIndex.from_documents(documents)
engine = index.as_query_engine()

with agentping.run("kb-answer", customer_id="acme-corp"):
    answer = engine.query("What is our refund window?")
```

`AgentPingLlamaIndexHandler(run=None)`. Supported `llama-index-core` versions: 0.10 up to, but not including, 2.0. Set the handler on `Settings.callback_manager` for global coverage, or pass a `CallbackManager` to a specific index, engine or LLM to scope it.

## Events

| LlamaIndex event | AgentPing event | What is recorded |
|---|---|---|
| `CBEventType.LLM` | `llm_call` | Provider, model, input and output tokens as the underlying LLM reports them in `response.raw`, cached input tokens when OpenAI reports them, latency. |
| `CBEventType.EMBEDDING` | `llm_call` with `kind: "embedding"` | Provider, model, latency, and an estimated `input_tokens` (chunk characters divided by four, since LlamaIndex does not surface embedding usage). `output_tokens` is 0. |
| `CBEventType.RETRIEVE` | `retrieve` | `node_count`: how many nodes the retriever returned. |
| `CBEventType.EXCEPTION` | `error` | The exception message and class. The run stays open; finish it with the status you want. |

Model name comes from the event payload's `model_name` (or the serialised LLM's `model` / `deployment_name`), then from the response. `provider` is inferred from the model name: `claude-*` is `anthropic`, `gpt-*`, `o1`, `o3` and `text-embedding-*` are `openai`, `gemini-*` is `gemini`, `mistral-*` and `mixtral` are `mistral`, `command-*` and `embed-*` are `cohere`, and anything else is `llamaindex`. Pass a model name the rate card recognises and cost follows.

Honest limits. Token counts are taken as the provider returned them, so an Anthropic call lands with the net input count and its cache split is not applied; use the [Anthropic](/docs/providers/anthropic) wrapper alongside the handler if that matters for your spend numbers. Embedding token counts are an estimate. Tool calls inside a LlamaIndex agent and the agent's own steps emit nothing; record them yourself with `run.event("tool_call", {...})` from inside the tool function.

## Naming and runs

The handler reads the active run from the current context, so create it once at startup and wrap each unit of work in `with agentping.run(...)`. Events that fire outside a run are dropped. An explicit `run=` is honoured for `retrieve` and `error` events; `llm_call` events go to the active run, so keep LlamaIndex calls inside the run's `with` block.

## Source / notes

- Python: `agentping.AgentPingLlamaIndexHandler` in [agent-ping-python](https://github.com/agent-ping/agent-ping-python)

For span-level traces of a whole query (retriever, synthesiser, each LLM call with timing), OpenInference's `openinference-instrumentation-llama-index` exports LlamaIndex traces over OTLP; point it at the [OpenTelemetry](/docs/integrations/opentelemetry) endpoint.