AgentPing now ships a Model Context Protocol server. Point ChatGPT, Claude, Cursor, VS Code, Codex or Gemini CLI at https://agentping.io/mcp, approve access in your browser, and your assistant can search runs, read what happened inside them, read evaluation verdicts, and on paid plans act on what it finds. Reads are on every plan, including Free, from today.
Setup for each client is in the MCP guide. This post is about why we built it and what it is actually useful for.
Why an MCP server
Agent failures do not look like failures. The run completes, the JSON parses, the status code is 200, and the customer got an invoice with a blank name on it. We wrote up four of these last week, and the common thread was that the answer was in the run data the whole time; someone just had to go and look.
Looking has a cost. You get the alert on your phone, open the dashboard, find the run, open the events, read the tool call that returned an empty body, work out what to change, switch to your editor, and by then you have lost the thread twice. Everything AgentPing knows about a run is one browser tab away from where you are actually going to fix it.
MCP is an open standard that lets an assistant discover and call a service's tools directly. With it, the assistant that is already in your editor pulls the run, reads the events, reads the verdict and tells you what went wrong in the same conversation where you are about to edit the code. That is the whole pitch: detection and diagnosis in one place.
There is nothing to install. It is a hosted server; you paste one URL, sign in, and the assistant discovers the tools on connection.
What the server exposes
Twenty-one tools in three permission groups.
- Read (every plan).
list_teams,list_agents,search_runs,get_run_events,get_run_evaluation. Agents come back with pass rate, quality, last-seen and month-to-date cost. Runs come back newest first with status, duration, cost and customer, filterable by status, agent and date range. Events are the ordered step-by-step trace of one run; the evaluation is the verdict, quality score, issues and reasoning. - Ops (paid plans).
acknowledge_incident,resolve_incident,snooze_rule,submit_evaluation_feedback. - Manage (paid plans, admin role). Create, update and delete alert rules, budgets, channels and status pages, regenerate a status page slug, and update team settings.
Every tool is annotated so the assistant knows which are read-only and which change state, and the deletes refuse to run without a confirmation that names the exact thing being deleted. The full list is in the guide under what you can ask.
Use cases
These are the workflows we designed the tool set around, in roughly the order we expect people to try them.
Why did it fail last night?
"Why did invoicer fail last night?"
The assistant calls search_runs with the agent, status: failed and last night's window, picks a run, then get_run_evaluation and get_run_events. It comes back with something like: three failures between 02:00 and 02:40, the lookup_customer tool returned 200 with an empty body, the agent produced an invoice with a blank customer, the evaluation caught it. That is the worked example in the guide, and it is the question people ask most.
Fix it without leaving the editor
In Cursor, VS Code or Claude Code the assistant has your agent's code open. "Look at yesterday's failed runs for enricher and tell me which tool call went wrong" gets you the events, and because it can see the code that made that call, it can propose the fix in the same turn. When you have deployed, "check the last ten runs for enricher" confirms the verdicts have gone green, and on a paid plan "resolve the incident" closes the loop.
Who is this costing us?
"Which customers cost the most on the support agent this week?"
Run summaries carry cost_usd and customer_id, so the assistant pages through search_runs for the week and totals by customer. Same shape for "what did we spend per agent this month", which list_agents answers in one call. If you have ever pasted a CSV export into a spreadsheet to answer a question like this, this replaces the spreadsheet.
Triage from the chat you are already in
An alert fires while you are in a Claude or ChatGPT conversation. "Acknowledge the incident on scheduler and tell me what the last run did" stops the reminders and gives you the events in one exchange. If it turns out to be a known flap, "snooze the silent-agent rule on scheduler for 24 hours" does exactly that, and the snooze shows up in your Activity log with the app that made it.
Set up monitoring by describing it
"Add a $200 monthly budget on the
researcheragent, alert at 50, 80 and 100 percent, to the #ai-spend Slack channel."
create_budget with the scope, amount, thresholds and channel. Budgets in AgentPing only warn, they never stop an agent, and the assistant is told that in the tool description so it will not promise a hard cap. Alert rules work the same way: "page me if invoicer has three consecutive failures" becomes create_alert_rule with the consecutive_failures condition.
Tune the evaluator from the run
Evaluation verdicts are only as good as the rubric behind them. When one is wrong, "I disagree with the verdict on that run, the empty customer is expected for internal accounts" calls submit_evaluation_feedback with the note, and the disagreement feeds rubric tuning. Doing this from the conversation where you were already looking at the run makes it far more likely to actually happen.
Status page housekeeping
"Create a status page for the customer-facing agents" and "regenerate the slug for the status page we shared with the old vendor" are both one tool call. Small jobs, but the kind that otherwise sit on a list for a fortnight.
What it will not do
Some boundaries were deliberate.
- No money, no members, no keys. The server cannot change billing, invite or remove members, create or revoke API keys, or delete your account or your data. Money through an assistant is a fraud surface, and deleting data needs a human in a browser.
- Team scoped, permission scoped. Reads work for every member on every plan. Ops tools need a paid plan. Manage tools need a paid plan and the admin role, the same as the dashboard.
- Confirmed deletes. Deleting a rule, budget, channel or status page requires
confirm: trueand the exact name of the thing, so a wrong guess cannot remove the wrong rule. - Everything is logged. Every change an assistant makes lands in your Activity log with who, when, via which connected app, and from what to what.
Authentication is OAuth 2.1 with PKCE. You approve once in the browser, no secrets are pasted anywhere, and you can revoke any connected app from Settings, Connected apps. Revoking takes effect immediately.
Getting connected
Every client is covered in the guide: ChatGPT, Claude, Claude Code, Cursor, VS Code, Codex CLI, Gemini CLI and Windsurf, plus notes for other clients. Cursor and VS Code have one-click install links in their sections. Everyone else needs the URL and a sign-in.
If you are not on AgentPing yet, the Free plan includes MCP reads from the first minute, and the five-minute try page will get a real run into your account to ask about.