Triage Production Incidents with a Single Prompt Using the AppSignal CLI

AppSignal released a CLI tool that lets developers and AI agents triage production incidents with a single prompt, enabling searches of production logs, incident closures, and pull request checks from the terminal. Customer Oliver Kriška built an agent skill around the CLI, reducing what previously took half a day of data gathering to seconds. The CLI supports machine-readable JSON output and composable query syntax for reliable automation.

AppSignal Triage Production Incidents with a Single Prompt Using the AppSignal CLI At AppSignal, we love talking to our customers to learn how they're using the product. Recently, one of them showed us something worth sharing: with a single chat prompt, his AI agent searches production logs, closes incidents, and checks whether a pull request has shipped. Automating that kind of work turned out to be a matter of building the right agent skill. That customer is Oliver Kriška https://www.linkedin.com/in/quatermain/?locale=en . He helps companies cut costs, remove bottlenecks, and keep their systems reliable as they scale. Much of his work is with high-availability platforms for fintech, SaaS, and q-commerce, including payment systems for regulated markets across several countries. He also builds practical AI integrations into products and team workflows, which is exactly where the agent skill in this article fits. We learned from Oliver how he uses the AppSignal CLI with his agents to check shipped pull requests, close incidents quickly, and pull the context he needs from production logs. As Oliver tells it, the agent skill finally made that data easy to reach: The monitoring data was always there. What was missing was a way to reach it without leaving the terminal. Now I ask in plain words and the agent runs the same commands I would run myself. It just remembers the flags for me. What used to cost me half a day of gathering data, I have in seconds. In this article, we'll walk through what the AppSignal CLI can do, and how wrapping it in an agent skill lets you ask for those commands in plain language. Meet the AppSignal CLI The AppSignal CLI https://github.com/appsignal/appsignal-cli is our official command-line tool, and it works well for both people and AI agents. It has stable subcommands, real --output json for machine-readable results and LLMs, and a composable query syntax. Those are the qualities an agent needs to drive a tool reliably. It also meets developers in the workflow they already use. You can search logs, check deploys, and update incidents straight from your editor and terminal, without breaking focus to switch to a browser tab. Getting started takes two commands. Install it with Homebrew: Then authenticate once through your browser: From there, the CLI is ready. Oliver wrote his own agent skill to drive it, but you don't have to start from scratch: the CLI ships with a bundled one you can install in one command. So what exactly is an agent skill, and why wrap the CLI in one? Let's break it down. What Is an Agent Skill? An agent skill is a short, plain-text file that teaches an AI coding agent like Claude how to do one specific task, so you can ask for it in plain words instead of explaining the steps each time. It's more than a list of commands. A good agent skill tells the agent three things: when a task applies, which tools or commands to reach for, and how to read the results. It also points out the common mistakes, so the agent gets things right the first time. Once the agent skill is in place, you don't have to remember the exact command, flag, or query syntax. You describe what you want, such as "search production for errors in the last hour," and the agent reads the skill, runs the right command, and gives you the answer. A minimal agent skill is a file with a short header and a few instructions, like the one Oliver uses to drive the CLI: That's the whole idea. Now let's look at what Oliver's agent skill lets him do with the AppSignal CLI. What the Agent Skill Lets You Do In each case below, Oliver types a plain-language request, and the agent skill turns it into the right AppSignal CLI command. Search and Tail Production Logs Not every incident leaves behind an error for AppSignal to catch. A request that runs out of memory and gets killed, or a user action that quietly fails, might never surface as one. But the logs still record what happened at that exact moment, and that single log line is often the whole answer. Search production for errors in the last hour. Behind the scenes, the agent skill runs: Need to watch events as they happen? The agent skill can stream them live: Triage and Update Incidents The agent skill doesn't only read. It can act. Oliver uses it to list open incidents, then close or re-prioritize them without leaving the chat. Close incident 1423. Assigning an incident and setting its severity is just as quick: Answer "Is My PR Deployed Yet?" Deploy markers map a revision to the moment it went live, so the agent skill can answer one of the most common questions on any team. Is my latest change live in production yet? The agent correlates the revision with the incident you're investigating, and tells you whether the deploy you're worried about has actually landed. Define Anomaly Triggers as Code The agent skill can also set up alerts for you. You tell it which metric to watch and the limit that matters, and it creates a rule that notifies you when the metric crosses that line. Notify me when the P95 queue time goes over 100ms. Manage Dashboards from the Terminal The agent skill can also list your dashboards or create a new one from the terminal: The create command sets the dashboard's title and description. You still add the graphs themselves in the AppSignal UI. Why Drive the CLI with an Agent Skill? You could run every one of these commands by hand. Wrapping them in an agent skill adds a few things: Plain language over syntax. You describe the goal; the agent skill remembers the exact flags, query syntax, and app IDs. The right tool, every time. A good agent skill knows its own boundaries and hands off to the correct command instead of guessing. Built-in guardrails. Oliver's agent skill documents gotchas, such as running from the project root and the fact that authentication is interactive, so the agent avoids the mistakes that cost real time. Reusable and shareable. An agent skill is a plain file. Write it once, drop it into your team's repo, and everyone's agent gains the same capabilities. The CLI, the MCP Server, and the API The CLI is one of several ways to reach your AppSignal data, and each suits a different audience: The is ideal for the terminal: log tailing, deploy markers, and incident updates from the shell, whether you're at the keyboard or an agent is driving. AppSignal CLI https://github.com/appsignal/appsignal-cli The exposes the same monitoring data as tools an AI assistant can call conversationally, right inside your editor. AppSignal MCP Server https://docs.appsignal.com/mcp.html is for code: custom integrations that pipe AppSignal data into your own systems. The REST API https://docs.appsignal.com/api/v2/overview The short version: the API is for code, and the MCP Server and CLI are for AI agents and people , all reading from the same monitoring data underneath. Real-World Use Cases Once the agent skill is in place, these everyday tasks become a plain-language request. A few scenarios Oliver runs regularly: Post-incident retrospectives: "Pull the error logs between 4:00 and 4:15 this morning," and read the exact lines from that window. Deploy verification: "Did the checkout fix ship, and is that incident still open?" The agent checks deploy markers and incidents together. Clearing the incident backlog: "List everything still open in the web namespace, and close the ones I've already fixed." Setting up monitoring: "Watch our database queue time and notify us when it goes over our limit." Each of these used to mean recalling a command and its flags. Now they're a single sentence. Bring AppSignal into Your AI Workflow As AI agents take on more of the daily work of shipping software, the tools that work well with them stand out. The AppSignal CLI fits this well: it's stable, scriptable, and works the same whether you run it yourself or let an agent run it. An agent skill is the thin layer that lets you ask for it in plain language. Got questions or an agent skill of your own to share? We're always listening Reach out in our Discord community https://discord.gg/sBWWGwYzfC or email us at support@appsignal.com mailto:support@appsignal.com . Published Wondering what you can do next? Karen Patteri de Souza AI advocate and Senior Technical Writer at AppSignal, shaping developer-first documentation at the intersection of LLMs, SDKs, APIs, and user experience. Always up for chatting about LLMs in docs, LLM output quality evaluation, scientific research, music, and great films or series. All articles by Karen Patteri de Souza /authors/karen-patteri-de-souza.html Become our next author Find out more /write-for-us.html AppSignal monitors your apps AppSignal provides insights for Ruby, Rails, Elixir, Phoenix, Node.js, Express and many other frameworks and libraries. We are located in beautiful Amsterdam. We love stroopwafels https://www.appsignal.com/waffles . If you do too, let us know mailto:contact@appsignal.com . We might send you some Discover AppSignal https://www.appsignal.com