cd /news/ai-agents/how-to-investigate-a-production-inci… · home topics ai-agents article
[ARTICLE · art-95216] src=blog.appsignal.com ↗ pub= topic=ai-agents verified=true sentiment=· neutral

How to Investigate a Production Incident Using an AI Agent (AppSignal MCP)

AppSignal's MCP server gives AI agents direct access to production monitoring data, enabling incident investigations in plain English. In a real scenario, an agent triaged a payments service alert, identified a Stripe::CardError spike with a 5x error rate increase at 2:42 a.m., and surfaced a backtrace pointing to PaymentProcessor#charge, cutting investigation time from 10-15 minutes to seconds.

read9 min views1 publishedAug 13, 2026
How to Investigate a Production Incident Using an AI Agent (AppSignal MCP)
Image: Blog (auto-discovered)

An incident has hit your product. I've been there: you're context-switching between hosting, CI/CD, codebase, AppSignal for monitoring, and whatever else your product depends on to minimize downtime and potential losses. You're trying to piece everything together, but it takes a lot of time, and that's something you don't have.

AI agents connected to your tooling and your monitoring data via MCP free up that time for you. Instead of juggling dashboards, you can just ask an agent something like, "Mate, what's happening?" The agent will then perform queries to AppSignal, correlate the data, and reason about a potential cause.

TL;DR AppSignal's MCP server gives your AI agents direct access to production monitoring data: errors, performance metrics, anomaly alerts, deploy markers, and much more. This article will guide you through a real incident investigation, from the first alert to a confirmed root cause, using plain English and your favorite AI agent.

MCP and Its Role in Incident Response

I've found MCP impossible to avoid for the past couple of months. You've probably heard of it even if you haven't touched it yet.

MCP is an open standard that makes it possible for AI agents to interact with external tools and data sources, like your monitoring tool.

AppSignal provides an MCP server that exposes a bunch of tools across different areas:

Your agent can pull all of that data and combine it with other context, thus creating a more complete image of the situation.

And that's where the real difference lies: instead of checking a dashboard and trying to figure everything out on your own, the agent just tells you what's happening.

Setup in Two Minutes

The complete two-minute instructions are available in the AppSignal MCP docs. But here's the short version using the public endpoint:

Prerequisites

[AppSignal account](https://appsignal.com/users/sign_up)with at least one app firing data- MCP token from
[Accounting Settings > MCP Tokens](https://appsignal.com/users/mcp_tokens) - Claude Code, Cursor, VS Code, Zed or Windsurf installed

Claude Code

Cursor

Edit ~/cursor/mcp.json

:

One-click install

If you'd rather skip hand-editing config files, use these deep links: Running Windsurf, Zed, Gemini CLI, GitHub Copilot CLI, the Claude app, or OpenAI Codex instead? Full setup steps for each are on the AppSignal MCP docs.

Verify the connection by asking your agent, "What apps do I have in AppSignal?" If it returns your app list, you're good to go.

Real Incident Investigation

Here is how things work in practice.

The scenario: It's 2 past midnight. An alert fires. The error rate in the payments service has spiked 5 times over the notifying limit. Normally, you would open AppSignal, check error grouping, cross-reference deploy history, pull host metrics, and search logs. About 10-15 mins later you might have a solid hypothesis.

With AppSignal MCP, the investigation can look like this:

Step 1: Triage

What's going on with the payments service in the last 30 minutes?

The agent calls your tools scoped to the last 30 minutes and returns:

  • Error rate increased 5x starting at 2:42 a.m.
- Dominant error:
`Stripe::CardError`
  • Possibly happening in a different time zone
  • No deployments in the last 6 hours
  • P99 latency increased from 180 ms to 2.1 s

Your understanding of the situation has been upgraded from "something is wrong" to a structured incident summary in seconds, without you touching the dashboard.

Step 2: Dig in

Show me a sample backtrace for the Stripe::CardError.

The agent now queries the incident tool for the most relevant recent matches and surfaces:

  • Full stack trace pointing to PaymentProcessor#charge

  • Specific error messages

  • Amount of users affected over the last 30 mins

Now, you're able to pinpoint the exact piece of code involved and the scale of impact.

Step 3: Check for correlations

Did anything change around 2:40 a.m.? Deploys, traffic spikes?

Your agent pulls deploy markers via the get_app_resources

tool and cross-references them with the incident timeline:

  • No deploys since 9 a.m.
  • Normal traffic volume, no spike
  • External API response times: Stripe latency increased from 50 ms to 1.8 s starting at 2:20 a.m.

You've got a clear timeline, thanks to the agent.

Step 4: Confirm the situation

Show me Stripe API response times over the last two hours.

The agent calls get_metrics_timeseries

for the external service metric and returns a chart of Stripe response times. The spike lines up exactly with the error increase. Conclusion: This is an external service degradation. Time to check Stripe's status page and notify the responsible people.

Total investigation time: < 3 mins.

Step 5: Document and hand off

Add a note to the Stripe::CardError incident summarizing what we've found. Mark it as acknowledged and assign it to me.

The agent runs manage_incident_note

and update_incidents

. The incident now has context attached, and whoever picks it up next doesn't have to reconstruct the investigation from scratch.

Prompt Cheat Sheet

If you've been woken up at 2 a.m. and your brain is still booting up, feel free to take a peek at the prompt cheat sheet, no strings attached.

Phase Prompts
Triage "What errors have increased in the last hour?" / "Is anything broken in production right now?"
Scoping "Which endpoints are affected?" / "How many users are impacted?"
Timeline "When did this start?" / "What changed around [time]?"
Correlation "Were there any deploys before the spike?" / "Is this correlated with CPU or memory?"
Deep dive "Show me a sample backtrace for Stripe::CardError " (or MyApp.Payments.ChargeError in Elixir, stripe.error.CardError in Python) / "What's the slowest ActiveRecord, Ecto, or Django ORM query right now?"
Comparison "Compare error rates today vs. yesterday." / "Is this higher than our baseline?"
Resolution "Has the error rate recovered?" / "Show me the incident status across all open issues"
Handoff "Summarize what we've found and add it as a note to incident #[number]"

What AppSignal MCP Can and Cannot Do

MCP is not a one-click solution to all incidents. Therefore, it's worth looking at what it can do well and where its limits are.

MCP can:

  • Query errors, stack traces, performance data, host metrics, and deploy markers
  • Correlate events with natural language filters
  • Summarize patterns and surface anomalies faster than manual dashboard navigation
  • Update incidents, assign handles, add notes, and manage anomaly triggers

MCP cannot:

  • Access data AppSignal doesn't have; if you're not catching stuff with AppSignal, the agent simply can't find it
  • Trigger rollbacks, restart services, or take action outside AppSignal (unless you connect a different tool and chain them)
  • Replace human judgement
  • Guarantee perfect accuracy on ambiguous queries

MCP allows your AI agent to be a very fast analyst by giving it near-instant, context-aware data. Still, decisions are made solely by you.

Tips for Getting the Most Out of MCP

The following tips will help you interact more effectively with the MCP agent and get faster, more accurate insights.

Be specific about time ranges:"Last 30 mins" produces tighter and more useful results than "recently". The agent passes these as filters straight toAppSignal's API.Name services and endpoints directly: The agent has no knowledge of your internal jargon, so giving it precise service and endpoint names is a good practice.Iterate: Start broadly with a question like, "What's happening?", then narrow it down with, "Tell me more about the X error". You should treat incident investigation with agents as a conversation.Verify before escalation: I always confirm in the AppSignal dashboard before posting in Slack that it's definitely an external service's (or anyone else's) fault. The agent may be fast, but it's not infallible.Combine signals: If you're sending logs to AppSignal, ask the agent to search them alongside error data. Cross-signal correlation is a huge time saver.

Next Steps

AI will not replace monitoring, and it shouldn't. However, with an MCP, you can chain tools together to get a much clearer picture of an ongoing incident.

I think AppSignal's MCP server makes incident resolution faster, especially under pressure. For more on this distinction, see MCP Won't Replace Your Monitoring Tool.

To get started, you should check out AppSignal MCP docs for setup and MCP Tool Reference for the full query surface.

No AppSignal account yet? Start a [free 30-day trial](https://appsignal.com/users/sign_up). No credit card required.

Frequently Asked Questions (FAQ)

Does this work with any AI agent or only specific ones?

Claude Code, the Claude app, Cursor, Windsurf, Zed, VS Code, GitHub Copilot CLI, Gemini CLI, and OpenAI Codex all have setup instructions. Any MCP-compatible client will work. The ones mentioned here are just the ones with configurations tested in the docs.

Can the agent only read data, or can it update incidents too?

Both. The agent can update incident state and severity, assign handlers, add investigation notes, manage anomaly triggers, and create dashboards. Permissions are configurable per toolset when you generate your MCP token.

What if my app is not sending all signals to AppSignal yet?

The agent can only query the data that AppSignal already has. If you are not sending logs, log queries will return nothing. If you have no anomaly triggers configured, anomaly queries will return nothing, as well. The more signals you share with AppSignal, the more useful the investigation workflow becomes, which is a good reason to instrument fully before you need it.

Is this safe to use during an active incident with real production data?

Absolutely. MCP token permissions are configurable per area (app, exceptions, performance, metrics, anomalies, dashboards, logging), each set to read, write, or disabled, so read-only tokens are available if you want to limit the agent to querying only. As for write operations, those stay inside AppSignal's own data: incidents, anomaly triggers, log actions, and dashboards. The agent cannot touch your infrastructure via this integration.

How does AppSignal's MCP server compare to Scout's or Honeybadger's?

Scout and Honeybadger both ship official MCP servers too, but neither goes as far as AppSignal's:

Scout— read-only across the board. It can query metrics, traces, jobs, errors, and anomalies, but there's no way for an agent to update an incident, manage a trigger, or take any action at all.Honeybadger— scoped to error/fault data only. No APM, logs, metrics, or dashboards are exposed, and its write access is limited to project administration rather than managing faults or incidents themselves.AppSignal— the only one of the three that lets an agent close the loop: querying errors, performance, metrics, logs, and anomalies, then acting on what it finds by updating incidents, adding notes, managing triggers, and building dashboards, all from the same conversation.

Published

Wondering what you can do next? #

Try out AppSignal with a 30-day free trial.Reach out to our support team with any feedback or questions.

  • Share this article on social media

Dejan Lukić #

Our guest author Dejan is an electronics and backend engineer, who is pursuing entrepreneurship with SaaS and service-based agencies and is passionate about content creation.

All articles by Dejan Lukić Become our next author!

Find out more

AppSignal monitors your apps #

AppSignal provides insights for Ruby, Rails, Elixir, Phoenix, Node.js, Express and many other frameworks and libraries. We are located in beautiful Amsterdam. We love stroopwafels. If you do too, let us know. We might send you some!

Discover AppSignal

── more in #ai-agents 4 stories · sorted by recency
── more on @appsignal 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/how-to-investigate-a…] indexed:0 read:9min 2026-08-13 ·