MCP CLI Bridge and Usage Analytics

Signet is building a CLI bridge to expose installed MCP servers as subcommands and adding usage analytics to track invocations. The new `signet mcp` command group will let operators test tools directly from the terminal, while a new `mcp_invocations` table and dashboard panel will provide per-server, per-tool, and per-agent breakdowns. This gives operators debugging capabilities and visibility into agent tool usage for capacity planning.

MCP CLI Bridge and Usage Analytics Expose installed MCP servers as CLI subcommands and track invocation analytics in the dashboard. Problem Statement Signet manages MCP servers for agents marketplace routes, policy/scope controls, enable/disable lifecycle . But using those servers requires the agent harness to invoke them. There is no way for a human operator to call an MCP tool from the terminal, inspect available tools, or see which tools agents actually use and how often. This creates two gaps: 1 operators cannot test or debug MCP servers without running a full agent session, and 2 there is no visibility into which MCP capabilities agents rely on, making capacity planning and deprecation decisions blind. Goals - Expose every installed MCP server’s tools as signet mcp <server <tool CLI subcommands. - Record every MCP tool invocation CLI and agent with timestamp, agent id, latency, and outcome. - Surface usage analytics in the dashboard: per-server, per-tool, and per-agent breakdowns. - Support signet mcp list and signet mcp <server --help for discoverability. - Feed invocation frequency into the predictive scorer as a behavioral signal. Proposed Capability Set A. CLI Bridge signet mcp New CLI command group under surfaces/cli/src/commands/ : signet mcp list — queries GET /api/marketplace/servers to show installed servers with enabled/disabled status. signet mcp <server list-tools — calls the server’s MCP endpoint to enumerate available tools via the tools/list method. signet mcp <server <tool --param key=value... — invokes a tool via POST /api/marketplace/servers/:name/call or directly via MCP transport . Params are passed as key=value pairs, JSON auto-parsed for non-string values. signet mcp <server --help — shows server metadata and tool schemas from the MCP tools/list response. The bridge does not maintain its own MCP client connection. It calls the daemon API which already manages server lifecycle and transport platform/daemon/src/routes/marketplace.ts . B. Invocation Tracking Table New migration adding mcp invocations to memories.db : CREATE TABLE mcp invocations id INTEGER PRIMARY KEY AUTOINCREMENT, server TEXT NOT NULL, tool TEXT NOT NULL, agent id TEXT NOT NULL DEFAULT 'default', source TEXT NOT NULL CHECK source IN 'cli', 'agent', 'mcp' , latency ms INTEGER NOT NULL, success INTEGER NOT NULL DEFAULT 1, error text TEXT, created at TEXT NOT NULL DEFAULT datetime 'now' ; CREATE INDEX idx mcp inv server ON mcp invocations server, created at ; CREATE INDEX idx mcp inv agent ON mcp invocations agent id, created at ; The daemon records a row on every tool call — both CLI-originated via the bridge API and agent-originated via MCP transport hooks in platform/daemon/src/mcp/tools.ts . C. Analytics API New daemon endpoints: GET /api/mcp/analytics — aggregated stats: total calls, top servers, top tools, failure rate, p50/p95 latency. Accepts ?agent id= , ?server= , ?since= query params. GET /api/mcp/analytics/:server — per-server breakdown by tool. D. Dashboard Usage Panel New dashboard component in the MCP/Marketplace section showing: - Invocation timeline sparkline per server, last 7 days . - Top tools by call count with success/failure ratio. - Per-agent usage breakdown when multi-agent is active. - Latency distribution p50/p95 bars . E. Scorer Feature Export Invocation frequency calls per day, recency of last call exported as scorer features alongside existing behavioral signals. The mcp invocations table is queryable by the predictor feature pipeline the same way session memories is today. Non-Goals - MCP server installation or marketplace browsing see git-marketplace-monorepo . - Modifying MCP protocol transport or adding new transport types. - Real-time streaming of MCP tool output in the CLI v1 is request/response . - Policy enforcement already handled by mcp server policy set . Integration Contracts Signet Runtime : runtime records invocations via the same tracking table. The source column distinguishes 'agent' from 'cli' . Predictive Scorer : invocation frequency becomes a scorer feature cross-cutting invariant 4 . High-frequency tools score higher for related entity contexts. Multi-Agent : agent id column on mcp invocations scoped per cross-cutting invariant 1. Analytics API filters by agent. Git Marketplace : marketplace install/uninstall events are not tracked here — only tool invocations. Rollout Phases Phase 1: CLI Bridge + Tracking safe defaults Ship signet mcp list , signet mcp <server <tool , and the mcp invocations migration. Agent-side tracking off by default CLI tracking always on . Dashboard panel not yet built. Phase 2: Agent Tracking + Dashboard Enable agent-side invocation recording in the MCP transport hooks. Ship the dashboard analytics panel. Export scorer features. Phase 3: Advanced Analytics Add latency alerting thresholds, tool deprecation warnings no invocations in 30 days , and per-session tool usage summaries in the session timeline. Validation and Tests - CLI integration test: install a mock MCP server, call a tool via signet mcp , verify invocation row in mcp invocations . - Analytics API test: seed invocation rows, verify aggregation math counts, latency percentiles, per-agent filtering . - Agent tracking test: simulate an agent MCP tool call, verify the source='agent' row is recorded with correct agent id . - Scorer feature test: verify invocation frequency appears in the feature vector for entities associated with the MCP server. Success Metrics - Any installed MCP server’s tools are callable from signet mcp with <500ms overhead vs direct MCP call. - Dashboard shows per-tool invocation counts and latency within 1 minute of the call occurring. - Zero invocation rows with missing agent id scoping invariant . Open Decisions CLI output format — should signet mcp <server <tool output raw JSON or formatted text by default? Leaning toward JSON with --pretty flag for human-readable output. Batch invocation — should the CLI support piping multiple tool calls? Defer to Phase 3 unless demand is clear. Retention policy — how long to keep mcp invocations rows? 30 days rolling window vs unbounded with manual cleanup.