Triage Production Incidents with a Single Prompt Using the AppSignal CLI

At AppSignal, we love talking to our customers to learn how they're using the product. Recently, one of them showed us something worth sharing: with a single chat prompt, his AI agent searches production logs, closes incidents, and checks whether a pull request has shipped. Automating that kind of work turned out to be a matter of building the right agent skill.

That customer is Oliver Kriška. He helps companies cut costs, remove bottlenecks, and keep their systems reliable as they scale. Much of his work is with high-availability platforms for fintech, SaaS, and q-commerce, including payment systems for regulated markets across several countries. He also builds practical AI integrations into products and team workflows, which is exactly where the agent skill in this article fits.

We learned from Oliver how he uses the AppSignal CLI with his agents to check shipped pull requests, close incidents quickly, and pull the context he needs from production logs. As Oliver tells it, the agent skill finally made that data easy to reach:

The monitoring data was always there. What was missing was a way to reach it without leaving the terminal. Now I ask in plain words and the agent runs the same commands I would run myself. It just remembers the flags for me. What used to cost me half a day of gathering data, I have in seconds.

In this article, we'll walk through what the AppSignal CLI can do, and how wrapping it in an agent skill lets you ask for those commands in plain language.

Meet the AppSignal CLI

The AppSignal CLI is our official command-line tool, and it works well for both people and AI agents. It has stable subcommands, real --output json for machine-readable results and LLMs, and a composable query syntax. Those are the qualities an agent needs to drive a tool reliably.

It also meets developers in the workflow they already use. You can search logs, check deploys, and update incidents straight from your editor and terminal, without breaking focus to switch to a browser tab.

Getting started takes two commands. Install it with Homebrew:

brew install appsignal/appsignal-cli/appsignal-cli

Then authenticate once through your browser:

appsignal-cli auth login

From there, the CLI is ready. Oliver wrote his own agent skill to drive it, but you don't have to start from scratch: the CLI ships with a bundled one you can install in one command.

# Install the bundled AppSignal agent skill
appsignal-cli skill install

So what exactly is an agent skill, and why wrap the CLI in one? Let's break it down.

What Is an Agent Skill?

An agent skill is a short, plain-text file that teaches an AI coding agent like Claude how to do one specific task, so you can ask for it in plain words instead of explaining the steps each time. It's more than a list of commands. A good agent skill tells the agent three things: when a task applies, which tools or commands to reach for, and how to read the results. It also points out the common mistakes, so the agent gets things right the first time.

Once the agent skill is in place, you don't have to remember the exact command, flag, or query syntax. You describe what you want, such as "search production for errors in the last hour," and the agent reads the skill, runs the right command, and gives you the answer.

A minimal agent skill is a file with a short header and a few instructions, like the one Oliver uses to drive the CLI:

---
name: appsignal-cli
description: Use when the user wants to search or tail production logs, or triage AppSignal incidents (exception, performance, anomaly). Handles requests like "search production for errors in the last hour", "tail error logs", or "close incident 1423".
allowed-tools: Bash, Read
---
 
# AppSignal CLI
 
Use `appsignal-cli` to search and tail production logs, search and triage exception, performance, and anomaly incidents (list, close, re-prioritize, add notes), check deploy markers, and manage anomaly triggers and dashboards.
 
- Search logs for a term: `appsignal-cli --output json logs search --app "MyApp" --environment "production" --query "timeout"`
- Show one incident's details: `appsignal-cli incidents show --number 42 --app "MyApp" --environment "production"`
- Tail logs in real time: `appsignal-cli logs tail --app "MyApp" --environment "production"`

That's the whole idea. Now let's look at what Oliver's agent skill lets him do with the AppSignal CLI.

What the Agent Skill Lets You Do

In each case below, Oliver types a plain-language request, and the agent skill turns it into the right AppSignal CLI command.

Search and Tail Production Logs

Not every incident leaves behind an error for AppSignal to catch. A request that runs out of memory and gets killed, or a user action that quietly fails, might never surface as one. But the logs still record what happened at that exact moment, and that single log line is often the whole answer.

Search production for errors in the last hour.

Behind the scenes, the agent skill runs:

appsignal-cli logs search --app-id YOUR_APP_ID \
  --severities "ERROR,CRITICAL" --limit 50 --output json

Need to watch events as they happen? The agent skill can stream them live:

appsignal-cli logs tail --app-id YOUR_APP_ID --severities ERROR

Triage and Update Incidents

The agent skill doesn't only read. It can act. Oliver uses it to list open incidents, then close or re-prioritize them without leaving the chat.

Close incident 1423.

appsignal-cli incidents update --number 1423 --app-id YOUR_APP_ID --state CLOSED

Assigning an incident and setting its severity is just as quick:

appsignal-cli incidents update --number 1423 --app-id YOUR_APP_ID --severity HIGH --assign-me

Answer "Is My PR Deployed Yet?"

Deploy markers map a revision to the moment it went live, so the agent skill can answer one of the most common questions on any team.

Is my latest change live in production yet?

appsignal-cli apps resources deploy-markers --app-id YOUR_APP_ID --output json

The agent correlates the revision with the incident you're investigating, and tells you whether the deploy you're worried about has actually landed.

Define Anomaly Triggers as Code

The agent skill can also set up alerts for you. You tell it which metric to watch and the limit that matters, and it creates a rule that notifies you when the metric crosses that line.

Notify me when the P95 queue time goes over 100ms.

appsignal-cli triggers create --app-id YOUR_APP_ID --name "queue p95" \
  --metric-name queue_time --kind Advanced --field p95 \
  --comparison-operator '>' --condition-value 100 \
  --warmup-duration 5 --cooldown-duration 5

Manage Dashboards from the Terminal

The agent skill can also list your dashboards or create a new one from the terminal:

appsignal-cli dashboards list --app-id YOUR_APP_ID --output json
appsignal-cli dashboards create --app-id YOUR_APP_ID --title "Checkout Health"

The create command sets the dashboard's title and description. You still add the graphs themselves in the AppSignal UI.

Why Drive the CLI with an Agent Skill?

You could run every one of these commands by hand. Wrapping them in an agent skill adds a few things:

The CLI, the MCP Server, and the API

The CLI is one of several ways to reach your AppSignal data, and each suits a different audience:

• The AppSignal CLI is ideal for the terminal: log tailing, deploy markers, and incident updates from the shell, whether you're at the keyboard or an agent is driving.
• The AppSignal MCP Server exposes the same monitoring data as tools an AI assistant can call conversationally, right inside your editor.
• The REST API is for code: custom integrations that pipe AppSignal data into your own systems.

The short version: the API is for code, and the MCP Server and CLI are for AI agents and people, all reading from the same monitoring data underneath.

Real-World Use Cases

Once the agent skill is in place, these everyday tasks become a plain-language request. A few scenarios Oliver runs regularly:

• Post-incident retrospectives: "Pull the error logs between 4:00 and 4:15 this morning," and read the exact lines from that window.
• Deploy verification: "Did the checkout fix ship, and is that incident still open?" The agent checks deploy markers and incidents together.
• Clearing the incident backlog: "List everything still open in the web namespace, and close the ones I've already fixed."
• Setting up monitoring: "Watch our database queue time and notify us when it goes over our limit."

Each of these used to mean recalling a command and its flags. Now they're a single sentence.

Bring AppSignal into Your AI Workflow

As AI agents take on more of the daily work of shipping software, the tools that work well with them stand out. The AppSignal CLI fits this well: it's stable, scriptable, and works the same whether you run it yourself or let an agent run it. An agent skill is the thin layer that lets you ask for it in plain language.

Got questions or an agent skill of your own to share? We're always listening! Reach out in our Discord community or email us at support@appsignal.com.