Grafana Coding Agent Observability

Monitor the coding agents you already use — Cursor, Claude Code, Codex, Copilot CLI, OpenCode, Pi, Vibe, and others. Observe usage, cost, tokens, and tools across all of them in one place. Keep sessions on your machine with the local Agent Observability app, or send them to Grafana Agent Observability.

1. Installagento11y .
2. Configure withagento11y login (use local Agent Observability app or Grafana Cloud Agent Observability).
3. Launch a coding agent withagento11y <agent> (for exampleagento11y claude ).
4. If something looks wrong, run agento11y doctor .

Or hand setup to a coding agent already in your terminal — see Skills.

Quick install (Linux/macOS):

curl -fsSL https://raw.githubusercontent.com/grafana/agento11y/main/plugins/agento11y/scripts/install.sh | sh

Installs to ~/.local/bin. Put that directory on your PATH if it is not already.

Homebrew (macOS):

brew install grafana/grafana/agento11y

mise:

mise use -g aqua:grafana/agento11y@latest

Make sure mise is activated in your shell so agento11y is on your PATH.

Go install (Windows, or any platform with Go 1.25+):

go install github.com/grafana/agento11y/plugins/agento11y/cmd/agento11y@latest

Installs to $(go env GOPATH)/bin (or GOBIN). Put that directory on your PATH.

Windows (prebuilt binary): download the windows_amd64 or windows_arm64 zip from the releases page, extract agento11y.exe, and put it on your PATH.

Verify with agento11y --version.

Note: The command was renamed from sigil; the old name still works but will be removed.

Run agento11y login to configure capture. On first run it asks where sessions go: Local only, or Grafana Cloud. Local only needs no Cloud credentials and does not forward sessions. Grafana Cloud prints your stack's coding-agent setup page and asks you to paste the connection block from that page.

Interactive Cloud setup on macOS and Linux also asks for Local web UI in the preferences. The question starts on Yes by default and remembers an earlier answer. Yes keeps a full local copy and forwards a copy to Grafana Cloud. No sends directly to Grafana Cloud without a local copy. Windows has no local receiver, so it does not show either local choice. Complete credential flags also skip the preferences and keep direct Cloud behavior.

Run agento11y login again to change the Cloud connection, local web UI, content capture, tags, or guard settings. With a complete saved Cloud connection, a rerun shows its Grafana stack first. Keep this connection jumps to preferences without changing the connection. Change connection opens the stack and credential questions. A rerun asks where sessions go only when neither a destination nor credentials are saved.

For scripts and unattended rollout (register an agent without prompts), see Noninteractive agent setup and Fleet reconciliation.

Run agento11y <agent> with your coding agent's command name:

agento11y claude

Agent	How to run
Claude Code	agento11y claude
Codex	agento11y codex
Copilot CLI	agento11y copilot
Cursor	agento11y cursor install , then start Cursor
OpenCode	agento11y opencode
Pi	agento11y pi
Vibe	agento11y vibe

Cursor has no launcher. Run agento11y cursor install once, then start Cursor normally. Remove its hooks with agento11y cursor uninstall. See also cursor/README.md. Per-agent notes and glue live under plugins/.

Export claude plugin eval results, conversations, traces, and usage without another runner or judge. Run Claude with --keep-temp, then:

agento11y claude eval import results.json --trace-root /tmp --include-content

Use --dry-run to preview and --expect-tenant <stack-id> to guard the destination. --trace-root authorizes retained trace files under that directory; omit it for results-only import. Content requires --include-content; without it, trajectory capture preserves identities, model/tool spans and usage but not message bodies. The command imports both baseline arms and reuses the saved Cloud/OTLP connection. See the example and CI workflow for build instructions, scoring details, and verification with gcx. This is a new command in this checkout, not a promise about older installed releases.

The binary carries agent skills: markdown workflows a coding agent reads and follows. They ship inside the binary, so there is nothing to fetch and no second CLI to install. Upgrading agento11y upgrades them.

agento11y skills list                          # name and one-line description
agento11y skills show setup-coding-agent       # the raw SKILL.md on stdout

get is accepted as an alias for show, matching gcx agent skills get.

setup-coding-agent walks a coding agent through the whole setup: reading agento11y doctor --json, installing the binary, saving credentials, wiring the host agent, verifying one session, and diagnosing a broken pipeline. To hand setup to the agent already open in your terminal, paste this:

Run `agento11y skills show setup-coding-agent` and follow it to set up Grafana Agent observability for my coding agent.

agento11y doctor and agento11y login both name that command when they finish.

For help choosing and testing local guard packs, paste this into your coding agent:

Run `agento11y skills show setup-local-guards` and follow it to help me choose, configure, and test local guard packs.

The skills for instrumenting your own application code are separate and ship with gcx instead: gcx agent skills install agento11y-instrument.

agento11y <agent> --local records the session to a JSONL store and starts the local Agent Observability app. The command prints the app URL (tries http://127.0.0.1:8765, then a higher port if needed).

AGENTO11Y_LOCAL=true in the shell or config.env enables local mode for every launch and installed hook. Local only writes AGENTO11Y_LOCAL and SIGIL_LOCAL as true, and writes AGENTO11Y_LOCAL_FORWARD and SIGIL_LOCAL_FORWARD as false. Sessions then stay on the machine. Local web UI = Yes writes all four keys as true, so the daemon keeps a full local copy and also forwards to Grafana Cloud. Local web UI = No writes all four keys as false and sends directly to Cloud. Use --no-local to override a saved local destination for one launcher session without changing those keys.

The daemon always stores full session content locally. It forwards full generation content only when AGENTO11Y_CONTENT_CAPTURE_MODE=full. Every other selected capture mode is reduced to metadata_only for the forwarded copy. Local mode is available on macOS and Linux only; Windows has no local receiver.

Manage the app with agento11y local start|open|status|stop|restart. agento11y local open starts the receiver if needed, prints its address, and tries to open the app.

For the six bundled packs, setup steps, blocked and allowed examples, and enforcement limits, refer to Local guard packs. You can select packs in Settings > Local > Guards, or ask your coding agent to follow agento11y skills show setup-local-guards.

With AGENTO11Y_GUARDS_ENABLED=true, each host POSTs preflight and tool-call checks to the daemon. Put guards.toml next to config.env (~/.config/agento11y/guards.toml). A local deny always denies. AGENTO11Y_GUARDS_FAIL_OPEN only applies to Cloud relay failures. The daemon skips rules that cannot compile and still evaluates valid rules. An unknown action_on_fail is reported and treated as deny. An unreadable, unparsable, or empty file allows every call locally.

reject = true blocks when the pattern matches. config.target = "shell_command" evaluates the decoded command line of a shell tool instead of the JSON-escaped tool-call text. Cloud-only evaluator kinds (llm_judge, heuristic, prompt_guard, json_schema) load and never fire locally.

[[rules]]
rule_id = "block.reset"
phase = "postflight"
action_on_fail = "deny"

  [[rules.evaluators]]
  kind = "regex"
  config.target = "shell_command"
  config.reject = true
  config.patterns = ['(?i)\bgit\s+reset\s+--hard\b']

agento11y doctor reports the file path, compile errors, and how many rules can enforce locally.

agento11y guards test evaluates saved local policy even when host guard requests are disabled. It does not execute the submitted command, contact endpoints, start a daemon, or write files. A local dry run does not predict host enforcement or Cloud decisions.

agento11y guards test 'rm -rf ~/.ssh'
agento11y guards test --json --rules ./guards.toml 'git reset --hard'
printf '%s\n' 'echo first' 'echo second' | agento11y guards test --stdin --tool shell --agent pi

Flags must precede one quoted command argument. Use -- before a command starting with a dash. Alternatively, --stdin reads one command through EOF, preserving newlines. Blank commands and empty tool names are errors.

Without --rules, the command reads guards.toml beside the resolved config.env, including the legacy sigil directory fallback. An explicit path bypasses configuration discovery. A missing default file or a valid file with no enforceable rules returns allow with a notice. A missing explicit file is an error. Disabled rules stay disabled; absent packs stay absent. Unchanged legacy packs receive the daemon's in-memory upgrade without changing the file.

The synthetic request is postflight, with one assistant tool call, ID guards-test. --tool defaults to Bash; another name still receives {"command": ...} arguments. Conditional rules see only --agent, which defaults to empty. There is no model, tags, agent version, preflight history, or system prompt. This command does not replay arbitrary host payloads.

Plain output starts with allow (local dry run), deny (local dry run), or error (local dry run). It reports the rules path, enforceable count, rule-level results, diagnostics, and any transformed input. Failed evaluations can be warnings or dropped redactions, not just denials. A response rule ID does not identify every transform that ran.

--json writes one deterministic document to stdout:

• schema_version: 1 andscope: "local" identify the contract.
• rules containspath ,exists ,compiled (rule count), andenforcing (locally enforceable rule count).
• request containscommand ,tool_name , andagent_name .
• response is the engine response, or null when no evaluation completed. Itsevaluations array preserves engine order.transformed_input uses SDK JSON with raw JSON tool arguments, not the HTTP protobuf encoding.
• errors andnotices are arrays, including when empty.

After flags parse successfully, JSON errors also go to stdout. Stderr stays empty unless writing stdout fails. Flag-parsing errors use stderr.

Exit	Meaning
0	Completed allow, including warnings or transforms
1	Completed deny
2	Invalid input, file-read failure, compilation diagnostics, evaluation failure, or output failure

Errors take precedence over allow or deny. When some rules compile, their partial response accompanies compilation errors. Diagnostic JSON includes the submitted command and rewritten content. Do not use real secrets in shared test output. Run agento11y guards test --help for the full syntax.

The local Agent Observability app starts empty: it only has sessions captured after you installed agento11y. To backfill earlier sessions, prefer the app — a banner on the Sessions page, or Settings → History. Imports run in the background with live progress; you can cancel them, and a cancelled run keeps what it already imported.

You can also use the CLI. Supported agents are claude-code, codex, cursor, opencode, and pi (agento11y history import with no agent lists them):

agento11y history import claude-code --dry-run

agento11y history import claude-code --local

agento11y history import claude-code

Imported sessions are thinner than live capture — host logs omit fields that live hooks see:

• pi — reads$PI_CODING_AGENT_DIR/sessions (default~/.pi/agent/sessions ). One generation per assistant turn. Missing vs live: compaction/branch-summary generations, system prompt and request controls, full tool schemas, andgit.branch . Forks import only the fork's own turns; subagent logs are skipped.
• cursor — reads agent-transcript JSONL under~/.cursor/projects/…/agent-transcripts/ and the olderstore.db under~/.cursor/chats/… . Missing vs live: token usage/cost (turns are marked approximate), reliable timestamps onstore.db sessions, models on transcripts, and tool results on transcripts. Cursor formats can change without notice.
• opencode : Readsopencode*.db under$XDG_DATA_HOME/opencode (default~/.local/share/opencode ).OPENCODE_DB adds another database. Absolute values are used as-is. Relative values are resolved under the OpenCode data directory. History cannot recover the system prompt, host version, configured agent-name prefix, capture-time Git branch, time to first token, or separate tool spans. An assistant message is imported when it has a completion timestamp or an error. Messages with neither are skipped. OpenCode forks copy all messages before the fork point into a new session with new IDs. If you import both sessions, those copied assistant turns are imported twice.

Without --since, an import covers the last 90 days. Pass --since 365d (or a timestamp) to widen the window, and --until to bound the other end. Other useful flags: --workspace, --max-sessions, --max-turns, --all, --yes, --force. Without a terminal, pass --all --yes to import from a script.

Re-running an import is safe: a per-agent ledger under ~/.local/state/agento11y/history/ledger/ skips turns already recorded. --force re-exports those turns under the same generation IDs.

Send sessions to Grafana Cloud Agent Observability. Choose Grafana Cloud during agento11y login, or write Cloud credentials into config.env (see below).

To configure the connection without the prompt, set these in ~/.config/agento11y/config.env:

AGENTO11Y_ENDPOINT=https://agento11y-prod-<region>.grafana.net
AGENTO11Y_AUTH_TENANT_ID=<instance-id>
AGENTO11Y_AUTH_TOKEN=glc_...
AGENTO11Y_OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp-gateway-prod-<region>.grafana.net/otlp

Find these values at https://<your-grafana>.grafana.net/a/grafana-agento11y-app/setup-coding-agent.

The shared agento11y binary defaults to metadata_only: only model, tokens, tool names, timing, and cost ship to Grafana Agent Observability. Prompts, responses, and tool I/O stay local unless guards are enabled. Guards send evaluated messages and tool arguments to AGENTO11Y_ENDPOINT in every capture mode. To send captured content, set AGENTO11Y_CONTENT_CAPTURE_MODE in ~/.config/agento11y/config.env. The shared parser accepts every SDK mode:

AGENTO11Y_CONTENT_CAPTURE_MODE=full

Unknown values fall back to metadata_only with a warning. default is accepted as an alias for metadata_only so the shared binary matches the Go envconfig resolver rather than the JS SDK's client-level default of no_tool_content. The Pi (@grafana/agento11y-pi) and OpenCode (@grafana/agento11y-opencode) plugins ship their own parsers but accept the same set of values.

A plugin can only export fields the host agent passes through to it, so individual plugins may capture less than the SDK matrix shows. See Content Capture Modes for the SDK-level behavior matrix and plugin defaults.

Scripts and devcontainers can pass the Cloud connection values as flags. Preferences have no flags; set them in the config file or answer the prompt.

agento11y login --endpoint https://agento11y-prod-<region>.grafana.net --tenant <instance-id> --token glc_...

Flag	Meaning
--endpoint url	conversations API URL
--tenant id	instance ID
--token value	access-policy token with the sigil:write scope
--token-stdin	read the token from stdin; requires --endpoint and--tenant
--otlp-endpoint url	OTLP endpoint for SDK traces and metrics
--no-verify	write the file without checking the credentials
--yes	save even when the check fails

Passing --endpoint, --tenant, and a token together skips the value prompts. Login verifies the credentials before it writes the file. If verification fails and stdin is a terminal, login asks whether to save anyway. Pass --yes or --no-verify so a script never stops there.

Keep the token out of your shell history with --token-stdin:

printf %s "$TOKEN" | agento11y login --endpoint https://agento11y-prod-<region>.grafana.net --tenant <instance-id> --token-stdin

Shared options that apply whether you use local mode or Grafana Cloud.

Add --tag key=value (repeatable) before any -- to attach tags to every generation the launched session produces. This is shorthand for setting AGENTO11Y_TAGS; flag tags merge onto (and override) any AGENTO11Y_TAGS already in the environment.

agento11y claude --tag project=hackathon --tag team=ai
agento11y claude --tag project=hackathon -- --resume

The same flag works for every launcher (claude, codex, copilot, opencode, pi, vibe) and combines with --local.

AGENTO11Y_AUTO_CODING_AGENT_TAGS resolves the session's user, repository, and branch and attaches them as client tags, which are the tags that also become OTel metric labels. Use it to break usage and cost down by person, repository, or branch. The switch is off by default, and on its own it enables every name:

AGENTO11Y_AUTO_CODING_AGENT_TAGS=true
AGENTO11Y_AUTO_CODING_AGENT_TAGS_NAMES=user,repo

Name	Tag key	Resolved from
user	user	AGENTO11Y_USER_ID , then the identity the host agent knows, then the OS account name
repo	repo	owner/name of the checkout'sorigin remote, or the checkout directory name
branch	git.branch	Branch checked out in the session's directory

A key already set in AGENTO11Y_TAGS wins, an unresolved value leaves its key off, and an unsupported name is logged and skipped. The allowlist does nothing while the switch is off. Because these are client tags, they arrive in Prometheus as agento11y_tag_user, agento11y_tag_repo, and agento11y_tag_git_branch.

agento11y login asks the same question. Choose On under "Automatic tags" and it opens a checklist of the three values, ticked to match the saved allowlist. A first run ticks all three. Login then writes the switch and the allowlist for you. Choosing Off sets the switch to false and deletes the allowlist.

The setting works whether the session starts through agento11y <agent> or through the host agent directly: the installed hooks read it when they build their client.

Before enabling it, read the cardinality and personal-data notes in Tags and Metadata. The user value is commonly an email address, and it is kept for the metric retention period. Run agento11y doctor to see the exact values first.

All integrations read ~/.config/agento11y/config.env. If you only have the old ~/.config/sigil/config.env, that file is read and updated instead.

The stack is saved as AGENTO11Y_STACK_URL. It is used only to build links; the ingest endpoint is a different host. Login never saves the stack as AGENTO11Y_ENDPOINT.

Cloud credentials in this file are documented under Grafana Agent Observability.

AGENTO11Y_EXPORT_TIMEOUT_MS bounds each generation export request. It defaults to 30000 milliseconds and accepts base-10 integers from 1 through 2147483647.

After writing the current user's config.env, a script can register an agent integration without launching the host or opening the credential prompt:

agento11y copilot install --json
agento11y opencode install --json
agento11y pi install --json

Each command prints one secret-free result with installed, already_installed, missing_host, or error. Copilot writes its shared user hook file and does not require the Copilot CLI on PATH; that file is also read by Copilot Chat in VS Code. OpenCode and pi require their respective CLI to be on the current user's PATH; missing_host is a successful deferral, so a later run can configure a host installed after the script.

Claude Code provides the same command as agento11y claude install --json.

For MDM, configuration management, and other unattended rollout tools, reconcile those installers in one shot and receive one JSON result per agent:

agento11y agents reconcile --agents all --json

all includes every noninteractive installer registered in the installed binary, including an agent added by a future release. To target a fixed allowlist, pass names instead: --agents claude,cursor. The command never launches a coding agent or opens a login prompt. Its receipt reports installed, already_installed, missing_host, or a per-agent descriptive error; it exits non-zero only when an installer fails. This command contains no MDM-vendor, credential, or device-policy assumptions.

agento11y claude, agento11y codex, and agento11y opencode refresh the installed host plugin automatically. Set AGENTO11Y_AUTO_UPDATE=false to opt out.

AGENTO11Y_AUTO_UPDATE does not apply to the other launchers. agento11y copilot rewrites its own agento11y.json hooks file, and agento11y vibe re-upserts its three entries into vibe's hooks.toml, so both always point at the installed binary. agento11y pi leaves upgrades to pi's own installer.

agento11y help (or --help, or -h) prints the full command list on stdout and exits 0. An unknown subcommand, or a command given the wrong number of arguments, prints a one-line usage form on stderr and exits 2.

Run agento11y doctor first. It's a read-only diagnostic that reports both export pipelines, config, and installed host-agent plugins in one place. It sends a lightweight request to each endpoint and reports the HTTP status, so a wrong endpoint or a token missing a scope shows up as a broken pipeline:

agento11y doctor

The two pipelines are independent and fail independently:

• Conversations (the chat transcripts) export overAGENTO11Y_ENDPOINT +AGENTO11Y_AUTH_TENANT_ID +AGENTO11Y_AUTH_TOKEN . The token needs thesigil:write scope.
• Analytics (the Agent Observability metrics and traces) export overAGENTO11Y_OTEL_EXPORTER_OTLP_ENDPOINT (orOTEL_EXPORTER_OTLP_ENDPOINT ). The token needsmetrics:write andtraces:write .

The common failure is conversations showing up while the analytics page stays empty: the OTLP endpoint is unset, or the token lacks the metrics/traces scopes. agento11y doctor flags that case explicitly and exits non-zero when a pipeline is broken.

A 403 means the token is missing a write scope. A 401 means the endpoint refused the credentials without saying why: the token may be invalid or expired, or AGENTO11Y_AUTH_TENANT_ID may be wrong.

For support, capture the machine-readable report — it never includes the auth token value:

agento11y doctor --json

If you need lower-level detail, hooks always exit 0, so problems only show up in the debug log. Set AGENTO11Y_DEBUG=true in ~/.config/agento11y/config.env and tail ~/.local/state/agento11y/logs/agento11y.log. Installs that still have the pre-rename ~/.local/state/sigil directory keep using it (with the new agento11y.log file name) until it is removed.

This page is the setup guide for the agento11y CLI. Product docs: Instrument coding agents.

source & further reading

github.com — original article Show HN: AutoBot – live voice control for long-running AI work Sub-ms deterministic parsing vs. LLM-based policy for agent safety? Show HN: I forked JetBrains' new agent IDE to host Claude Code