Grafana Coding Agent Observability

Grafana released agento11y, a coding-agent observability tool that monitors usage, cost, tokens, and tools across Cursor, Claude Code, Codex, Copilot CLI, OpenCode, Pi, and Vibe in one place. The tool installs via curl, Homebrew, mise, or Go 1.25+ (go install github.com/grafana/agento11y/plugins/agento11y/cmd/agento11y@latest), and users configure it with `agento11y login` to keep sessions local or forward them to Grafana Cloud Agent Observability. Grafana renamed the command from sigil; the old name still works but will be removed.

Monitor the coding agents you already use — Cursor, Claude Code, Codex, Copilot CLI, OpenCode, Pi, Vibe, and others. Observe usage, cost, tokens, and tools across all of them in one place. Keep sessions on your machine with the local Agent Observability app, or send them to Grafana Agent Observability https://grafana.com/docs/grafana-cloud/machine-learning/agent-observability/ . 1. Install install agento11y . 2. Configure configure with agento11y login use local Agent Observability app or Grafana Cloud Agent Observability . 3. Launch a coding agent launch-a-coding-agent with agento11y <agent for example agento11y claude . 4. If something looks wrong, run agento11y doctor troubleshooting . Or hand setup to a coding agent already in your terminal — see Skills skills . Quick install Linux/macOS : curl -fsSL https://raw.githubusercontent.com/grafana/agento11y/main/plugins/agento11y/scripts/install.sh | sh Installs to ~/.local/bin . Put that directory on your PATH if it is not already. Homebrew macOS : brew install grafana/grafana/agento11y mise: mise use -g aqua:grafana/agento11y@latest Make sure mise is activated in your shell https://mise.jdx.dev/cli/activate.html so agento11y is on your PATH . Go install Windows, or any platform with Go 1.25+ : go install github.com/grafana/agento11y/plugins/agento11y/cmd/agento11y@latest Installs to $ go env GOPATH /bin or GOBIN . Put that directory on your PATH . Windows prebuilt binary : download the windows amd64 or windows arm64 zip from the releases page https://github.com/grafana/agento11y/releases , extract agento11y.exe , and put it on your PATH . Verify with agento11y --version . Note: The command was renamed from sigil ; the old name still works but will be removed. Run agento11y login to configure capture. On first run it asks where sessions go: Local only , or Grafana Cloud . Local only needs no Cloud credentials and does not forward sessions. Grafana Cloud prints your stack's coding-agent setup page and asks you to paste the connection block from that page. Interactive Cloud setup on macOS and Linux also asks for Local web UI in the preferences. The question starts on Yes by default and remembers an earlier answer. Yes keeps a full local copy and forwards a copy to Grafana Cloud. No sends directly to Grafana Cloud without a local copy. Windows has no local receiver, so it does not show either local choice. Complete credential flags also skip the preferences and keep direct Cloud behavior. Run agento11y login again to change the Cloud connection, local web UI, content capture, tags, or guard settings. With a complete saved Cloud connection, a rerun shows its Grafana stack first. Keep this connection jumps to preferences without changing the connection. Change connection opens the stack and credential questions. A rerun asks where sessions go only when neither a destination nor credentials are saved. For scripts and unattended rollout register an agent without prompts , see Noninteractive agent setup noninteractive-agent-setup and Fleet reconciliation fleet-reconciliation . Run agento11y <agent with your coding agent's command name: agento11y claude | Agent | How to run | |---|---| | Claude Code https://docs.anthropic.com/en/docs/claude-code | agento11y claude | | Codex https://developers.openai.com/codex | agento11y codex | | Copilot CLI https://docs.github.com/en/copilot/github-copilot-in-the-cli/using-github-copilot-in-the-cli | agento11y copilot | | Cursor https://cursor.com | agento11y cursor install , then start Cursor | | OpenCode https://opencode.ai | agento11y opencode | | Pi https://github.com/earendil-works/pi | agento11y pi | | Vibe https://github.com/mistralai/vibe | agento11y vibe | Cursor has no launcher. Run agento11y cursor install once, then start Cursor normally. Remove its hooks with agento11y cursor uninstall . See also cursor/README.md https://github.com/grafana/agento11y/blob/main/plugins/cursor/README.md . Per-agent notes and glue live under plugins/ https://github.com/grafana/agento11y/blob/main/plugins . Export claude plugin eval results, conversations, traces, and usage without another runner or judge. Run Claude with --keep-temp , then: python agento11y claude eval import results.json --trace-root /tmp --include-content Use --dry-run to preview and --expect-tenant <stack-id to guard the destination. --trace-root authorizes retained trace files under that directory; omit it for results-only import. Content requires --include-content ; without it, trajectory capture preserves identities, model/tool spans and usage but not message bodies. The command imports both baseline arms and reuses the saved Cloud/OTLP connection. See the example and CI workflow https://github.com/grafana/agento11y/blob/main/examples/experiments/claude-plugin-evals for build instructions, scoring details, and verification with gcx . This is a new command in this checkout, not a promise about older installed releases. The binary carries agent skills: markdown workflows a coding agent reads and follows. They ship inside the binary, so there is nothing to fetch and no second CLI to install. Upgrading agento11y upgrades them. agento11y skills list name and one-line description agento11y skills show setup-coding-agent the raw SKILL.md on stdout get is accepted as an alias for show , matching gcx agent skills get . setup-coding-agent walks a coding agent through the whole setup: reading agento11y doctor --json , installing the binary, saving credentials, wiring the host agent, verifying one session, and diagnosing a broken pipeline. To hand setup to the agent already open in your terminal, paste this: Run agento11y skills show setup-coding-agent and follow it to set up Grafana Agent observability for my coding agent. agento11y doctor and agento11y login both name that command when they finish. For help choosing and testing local guard packs, paste this into your coding agent: Run agento11y skills show setup-local-guards and follow it to help me choose, configure, and test local guard packs. The skills for instrumenting your own application code are separate and ship with gcx https://github.com/grafana/gcx instead: gcx agent skills install agento11y-instrument . agento11y <agent --local records the session to a JSONL store and starts the local Agent Observability app. The command prints the app URL tries http://127.0.0.1:8765 , then a higher port if needed . AGENTO11Y LOCAL=true in the shell or config.env enables local mode for every launch and installed hook. Local only writes AGENTO11Y LOCAL and SIGIL LOCAL as true, and writes AGENTO11Y LOCAL FORWARD and SIGIL LOCAL FORWARD as false. Sessions then stay on the machine. Local web UI = Yes writes all four keys as true, so the daemon keeps a full local copy and also forwards to Grafana Cloud. Local web UI = No writes all four keys as false and sends directly to Cloud. Use --no-local to override a saved local destination for one launcher session without changing those keys. The daemon always stores full session content locally. It forwards full generation content only when AGENTO11Y CONTENT CAPTURE MODE=full . Every other selected capture mode is reduced to metadata only for the forwarded copy. Local mode is available on macOS and Linux only; Windows has no local receiver. Manage the app with agento11y local start|open|status|stop|restart . agento11y local open starts the receiver if needed, prints its address, and tries to open the app. For the six bundled packs, setup steps, blocked and allowed examples, and enforcement limits, refer to Local guard packs https://github.com/grafana/agento11y/blob/main/docs/local-guards.md . You can select packs in Settings Local Guards , or ask your coding agent to follow agento11y skills show setup-local-guards . With AGENTO11Y GUARDS ENABLED=true , each host POSTs preflight and tool-call checks to the daemon. Put guards.toml next to config.env ~/.config/agento11y/guards.toml . A local deny always denies. AGENTO11Y GUARDS FAIL OPEN only applies to Cloud relay failures. The daemon skips rules that cannot compile and still evaluates valid rules. An unknown action on fail is reported and treated as deny . An unreadable, unparsable, or empty file allows every call locally. reject = true blocks when the pattern matches. config.target = "shell command" evaluates the decoded command line of a shell tool instead of the JSON-escaped tool-call text. Cloud-only evaluator kinds llm judge , heuristic , prompt guard , json schema load and never fire locally. rules rule id = "block.reset" phase = "postflight" action on fail = "deny" rules.evaluators kind = "regex" config.target = "shell command" config.reject = true config.patterns = ' ?i \bgit\s+reset\s+--hard\b' agento11y doctor reports the file path, compile errors, and how many rules can enforce locally. agento11y guards test evaluates saved local policy even when host guard requests are disabled. It does not execute the submitted command, contact endpoints, start a daemon, or write files. A local dry run does not predict host enforcement or Cloud decisions. agento11y guards test 'rm -rf ~/.ssh' agento11y guards test --json --rules ./guards.toml 'git reset --hard' printf '%s\n' 'echo first' 'echo second' | agento11y guards test --stdin --tool shell --agent pi Flags must precede one quoted command argument. Use -- before a command starting with a dash. Alternatively, --stdin reads one command through EOF, preserving newlines. Blank commands and empty tool names are errors. Without --rules , the command reads guards.toml beside the resolved config.env , including the legacy sigil directory fallback. An explicit path bypasses configuration discovery. A missing default file or a valid file with no enforceable rules returns allow with a notice. A missing explicit file is an error. Disabled rules stay disabled; absent packs stay absent. Unchanged legacy packs receive the daemon's in-memory upgrade without changing the file. The synthetic request is postflight, with one assistant tool call, ID guards-test . --tool defaults to Bash ; another name still receives {"command": ...} arguments. Conditional rules see only --agent , which defaults to empty. There is no model, tags, agent version, preflight history, or system prompt. This command does not replay arbitrary host payloads. Plain output starts with allow local dry run , deny local dry run , or error local dry run . It reports the rules path, enforceable count, rule-level results, diagnostics, and any transformed input. Failed evaluations can be warnings or dropped redactions, not just denials. A response rule ID does not identify every transform that ran. --json writes one deterministic document to stdout: - schema version: 1 and scope: "local" identify the contract. - rules contains path , exists , compiled rule count , and enforcing locally enforceable rule count . - request contains command , tool name , and agent name . - response is the engine response, or null when no evaluation completed. Its evaluations array preserves engine order. transformed input uses SDK JSON with raw JSON tool arguments, not the HTTP protobuf encoding. - errors and notices are arrays, including when empty. After flags parse successfully, JSON errors also go to stdout. Stderr stays empty unless writing stdout fails. Flag-parsing errors use stderr. | Exit | Meaning | |---|---| | 0 | Completed allow, including warnings or transforms | | 1 | Completed deny | | 2 | Invalid input, file-read failure, compilation diagnostics, evaluation failure, or output failure | Errors take precedence over allow or deny. When some rules compile, their partial response accompanies compilation errors. Diagnostic JSON includes the submitted command and rewritten content. Do not use real secrets in shared test output. Run agento11y guards test --help for the full syntax. The local Agent Observability app starts empty: it only has sessions captured after you installed agento11y. To backfill earlier sessions, prefer the app — a banner on the Sessions page, or Settings → History. Imports run in the background with live progress; you can cancel them, and a cancelled run keeps what it already imported. You can also use the CLI. Supported agents are claude-code , codex , cursor , opencode , and pi agento11y history import with no agent lists them : See what would be imported. Nothing is decoded, exported, or stored. agento11y history import claude-code --dry-run Import into the local store on this machine. agento11y history import claude-code --local Import into Grafana Cloud, the default without --local. agento11y history import claude-code Imported sessions are thinner than live capture — host logs omit fields that live hooks see: - pi — reads $PI CODING AGENT DIR/sessions default ~/.pi/agent/sessions . One generation per assistant turn. Missing vs live: compaction/branch-summary generations, system prompt and request controls, full tool schemas, and git.branch . Forks import only the fork's own turns; subagent logs are skipped. - cursor — reads agent-transcript JSONL under ~/.cursor/projects/…/agent-transcripts/ and the older store.db under ~/.cursor/chats/… . Missing vs live: token usage/cost turns are marked approximate , reliable timestamps on store.db sessions, models on transcripts, and tool results on transcripts. Cursor formats can change without notice. - opencode : Reads opencode .db under $XDG DATA HOME/opencode default ~/.local/share/opencode . OPENCODE DB adds another database. Absolute values are used as-is. Relative values are resolved under the OpenCode data directory. History cannot recover the system prompt, host version, configured agent-name prefix, capture-time Git branch, time to first token, or separate tool spans. An assistant message is imported when it has a completion timestamp or an error. Messages with neither are skipped. OpenCode forks copy all messages before the fork point into a new session with new IDs. If you import both sessions, those copied assistant turns are imported twice. Without --since , an import covers the last 90 days. Pass --since 365d or a timestamp to widen the window, and --until to bound the other end. Other useful flags: --workspace , --max-sessions , --max-turns , --all , --yes , --force . Without a terminal, pass --all --yes to import from a script. Re-running an import is safe: a per-agent ledger under ~/.local/state/agento11y/history/ledger/ skips turns already recorded. --force re-exports those turns under the same generation IDs. Send sessions to Grafana Cloud Agent Observability. Choose Grafana Cloud during agento11y login configure , or write Cloud credentials into config.env see below . To configure the connection without the prompt, set these in ~/.config/agento11y/config.env : AGENTO11Y ENDPOINT=https://agento11y-prod-<region .grafana.net AGENTO11Y AUTH TENANT ID=<instance-id AGENTO11Y AUTH TOKEN=glc ... AGENTO11Y OTEL EXPORTER OTLP ENDPOINT=https://otlp-gateway-prod-<region .grafana.net/otlp Find these values at https://<your-grafana .grafana.net/a/grafana-agento11y-app/setup-coding-agent . The shared agento11y binary defaults to metadata only : only model, tokens, tool names, timing, and cost ship to Grafana Agent Observability. Prompts, responses, and tool I/O stay local unless guards are enabled. Guards send evaluated messages and tool arguments to AGENTO11Y ENDPOINT in every capture mode. To send captured content, set AGENTO11Y CONTENT CAPTURE MODE in ~/.config/agento11y/config.env . The shared parser accepts every SDK mode: valid values: full | no tool content | metadata only | full with metadata spans AGENTO11Y CONTENT CAPTURE MODE=full Unknown values fall back to metadata only with a warning. default is accepted as an alias for metadata only so the shared binary matches the Go envconfig resolver rather than the JS SDK's client-level default of no tool content . The Pi @grafana/agento11y-pi and OpenCode @grafana/agento11y-opencode plugins ship their own parsers but accept the same set of values. A plugin can only export fields the host agent passes through to it, so individual plugins may capture less than the SDK matrix shows. See Content Capture Modes https://github.com/grafana/agento11y/blob/main/docs/concepts/content-capture-modes.md for the SDK-level behavior matrix and plugin defaults. Scripts and devcontainers can pass the Cloud connection values as flags. Preferences have no flags; set them in the config file or answer the prompt. agento11y login --endpoint https://agento11y-prod-<region .grafana.net --tenant <instance-id --token glc ... | Flag | Meaning | |---|---| | --endpoint url | conversations API URL | | --tenant id | instance ID | | --token value | access-policy token with the sigil:write scope | | --token-stdin | read the token from stdin; requires --endpoint and --tenant | | --otlp-endpoint url | OTLP endpoint for SDK traces and metrics | | --no-verify | write the file without checking the credentials | | --yes | save even when the check fails | Passing --endpoint , --tenant , and a token together skips the value prompts. Login verifies the credentials before it writes the file. If verification fails and stdin is a terminal, login asks whether to save anyway. Pass --yes or --no-verify so a script never stops there. Keep the token out of your shell history with --token-stdin : printf %s "$TOKEN" | agento11y login --endpoint https://agento11y-prod-<region .grafana.net --tenant <instance-id --token-stdin Shared options that apply whether you use local mode or Grafana Cloud. Add --tag key=value repeatable before any -- to attach tags to every generation the launched session produces. This is shorthand for setting AGENTO11Y TAGS ; flag tags merge onto and override any AGENTO11Y TAGS already in the environment. agento11y claude --tag project=hackathon --tag team=ai forward args to the underlying CLI after -- agento11y claude --tag project=hackathon -- --resume The same flag works for every launcher claude , codex , copilot , opencode , pi , vibe and combines with --local . AGENTO11Y AUTO CODING AGENT TAGS resolves the session's user, repository, and branch and attaches them as client tags, which are the tags that also become OTel metric labels. Use it to break usage and cost down by person, repository, or branch. The switch is off by default, and on its own it enables every name: ~/.config/agento11y/config.env AGENTO11Y AUTO CODING AGENT TAGS=true Optional: narrow it to some of the names. Defaults to all of them. AGENTO11Y AUTO CODING AGENT TAGS NAMES=user,repo | Name | Tag key | Resolved from | |---|---|---| | user | user | AGENTO11Y USER ID , then the identity the host agent knows, then the OS account name | | repo | repo | owner/name of the checkout's origin remote, or the checkout directory name | | branch | git.branch | Branch checked out in the session's directory | A key already set in AGENTO11Y TAGS wins, an unresolved value leaves its key off, and an unsupported name is logged and skipped. The allowlist does nothing while the switch is off. Because these are client tags, they arrive in Prometheus as agento11y tag user , agento11y tag repo , and agento11y tag git branch . agento11y login asks the same question. Choose On under "Automatic tags" and it opens a checklist of the three values, ticked to match the saved allowlist. A first run ticks all three. Login then writes the switch and the allowlist for you. Choosing Off sets the switch to false and deletes the allowlist. The setting works whether the session starts through agento11y <agent or through the host agent directly: the installed hooks read it when they build their client. Before enabling it, read the cardinality and personal-data notes in Tags and Metadata https://github.com/grafana/agento11y/blob/main/docs/concepts/tags-and-metadata.md cardinality-and-personal-data . The user value is commonly an email address, and it is kept for the metric retention period. Run agento11y doctor to see the exact values first. All integrations read ~/.config/agento11y/config.env . If you only have the old ~/.config/sigil/config.env , that file is read and updated instead. The stack is saved as AGENTO11Y STACK URL . It is used only to build links; the ingest endpoint is a different host. Login never saves the stack as AGENTO11Y ENDPOINT . Cloud credentials in this file are documented under Grafana Agent Observability grafana-agent-observability . AGENTO11Y EXPORT TIMEOUT MS bounds each generation export request. It defaults to 30000 milliseconds and accepts base-10 integers from 1 through 2147483647 . After writing the current user's config.env , a script can register an agent integration without launching the host or opening the credential prompt: agento11y copilot install --json agento11y opencode install --json agento11y pi install --json Each command prints one secret-free result with installed , already installed , missing host , or error . Copilot writes its shared user hook file and does not require the Copilot CLI on PATH ; that file is also read by Copilot Chat in VS Code. OpenCode and pi require their respective CLI to be on the current user's PATH ; missing host is a successful deferral, so a later run can configure a host installed after the script. Claude Code provides the same command as agento11y claude install --json . For MDM, configuration management, and other unattended rollout tools, reconcile those installers in one shot and receive one JSON result per agent: agento11y agents reconcile --agents all --json all includes every noninteractive installer registered in the installed binary, including an agent added by a future release. To target a fixed allowlist, pass names instead: --agents claude,cursor . The command never launches a coding agent or opens a login prompt. Its receipt reports installed , already installed , missing host , or a per-agent descriptive error; it exits non-zero only when an installer fails. This command contains no MDM-vendor, credential, or device-policy assumptions. agento11y claude , agento11y codex , and agento11y opencode refresh the installed host plugin automatically. Set AGENTO11Y AUTO UPDATE=false to opt out. AGENTO11Y AUTO UPDATE does not apply to the other launchers. agento11y copilot rewrites its own agento11y.json hooks file, and agento11y vibe re-upserts its three entries into vibe's hooks.toml , so both always point at the installed binary. agento11y pi leaves upgrades to pi's own installer. agento11y help or --help , or -h prints the full command list on stdout and exits 0. An unknown subcommand, or a command given the wrong number of arguments, prints a one-line usage form on stderr and exits 2. Run agento11y doctor first. It's a read-only diagnostic that reports both export pipelines, config, and installed host-agent plugins in one place. It sends a lightweight request to each endpoint and reports the HTTP status, so a wrong endpoint or a token missing a scope shows up as a broken pipeline: agento11y doctor The two pipelines are independent and fail independently: - Conversations the chat transcripts export over AGENTO11Y ENDPOINT + AGENTO11Y AUTH TENANT ID + AGENTO11Y AUTH TOKEN . The token needs the sigil:write scope. - Analytics the Agent Observability metrics and traces export over AGENTO11Y OTEL EXPORTER OTLP ENDPOINT or OTEL EXPORTER OTLP ENDPOINT . The token needs metrics:write and traces:write . The common failure is conversations showing up while the analytics page stays empty: the OTLP endpoint is unset, or the token lacks the metrics/traces scopes. agento11y doctor flags that case explicitly and exits non-zero when a pipeline is broken. A 403 means the token is missing a write scope. A 401 means the endpoint refused the credentials without saying why: the token may be invalid or expired, or AGENTO11Y AUTH TENANT ID may be wrong. For support, capture the machine-readable report — it never includes the auth token value: agento11y doctor --json If you need lower-level detail, hooks always exit 0, so problems only show up in the debug log. Set AGENTO11Y DEBUG=true in ~/.config/agento11y/config.env and tail ~/.local/state/agento11y/logs/agento11y.log . Installs that still have the pre-rename ~/.local/state/sigil directory keep using it with the new agento11y.log file name until it is removed. This page is the setup guide for the agento11y CLI. Product docs: Instrument coding agents https://grafana.com/docs/grafana-cloud/machine-learning/agent-observability/guides/instrument-coding-agents/ .