{"slug": "one-rails-request-one-event-production-context-for-coding-agents", "title": "One Rails request, one event: production context for coding agents", "summary": "A developer has released Wide Events, a Rails gem that attaches production context to a single OpenTelemetry root span per request or job, enabling coding agents to quickly diagnose issues. In one example, the root event revealed a 30-second request with 29.4 seconds spent on outbound HTTP, pinpointing an external dependency. The gem compiles a flat map of attributes onto the root span, which can be stored in ClickHouse for querying, and also supports logging to JSON for setups without tracing.", "body_md": "[Wide Events](https://github.com/adammiribyan/wide_events) is a Rails gem that puts the production context a coding agent needs onto one OpenTelemetry root span per request or job.\n\nIn one production search request, the root event showed 30.0 seconds total duration, 446 ms of Postgres time, and 29.4 seconds of outbound HTTP time. That was enough to focus the investigation on an external dependency. The trace then identified a `POST`\n\nthat took 28.9 seconds.\n\nThe trace contained 82 spans and 20,261 bytes of attribute JSON. The root event contained 40 attributes and 1,420 bytes. This is not a token benchmark, but it shows why the root event is a more compact starting point for an agent.\n\nA coding agent starts with an unusual advantage: it can search every model, controller, job, migration, and test in a few seconds.\n\nIt also starts with a serious blind spot. The repository cannot tell it:\n\nThose answers often exist somewhere, but “somewhere” might mean a trace waterfall, application logs, a feature-flag service, product analytics, and a database console. Pulling all of that into a context window is expensive and usually requires several joins that were never designed in advance.\n\nA wide event changes the starting point. The app accumulates the context it learns while processing one unit of work, then attaches the completed flat map to the OpenTelemetry root span. The span is marked `main=true`\n\n, so every request or job can be queried as one row.\n\n``` php\nrequest or job\n  -> Rails and domain context accumulate\n  -> child spans contribute dependency counts and timings\n  -> one flat map is flushed onto the root span\n  -> ClickHouse stores one queryable row\n```\n\nThe trace still exists. Wide Events gives it an application-shaped index.\n\nIf you do not run tracing, the same map can be emitted as one JSON log line.\n\nWide Events requires Ruby 3.2 or newer and Rails 7.1 or newer.\n\n```\n# Gemfile\ngem \"wide_events\"\nbundle install\nbin/rails generate wide_events:install\n```\n\nFor a first local event, no tracing setup is required:\n\n```\n# config/initializers/wide_events.rb\nWideEvent.configure do |config|\n  config.enabled = true\n  config.sink = :log\nend\n```\n\nHit a route and the event appears as a single JSON line. In production, the default sink writes the attributes onto the current OpenTelemetry root span. The deployment behind this article exports those spans to self-hosted ClickStack, which combines ClickHouse and HyperDX. The gem itself works with an OTLP backend you already use.\n\nThe gem supplies request and Active Job boundaries. The useful application context is deliberately yours to define.\n\nRails knows more about a request than generic instrumentation ever can. After authentication, it knows the actor and account. When a flag is evaluated, it knows the exact variant. After a search completes, it knows whether semantic search ran and what survived its thresholds.\n\nThat is where I add attributes:\n\n``` js\nWideEvent.set(\n  \"user.type\" => Current.person&.class&.name,\n  \"user.account.id\" => Current.account&.id\n)\n```\n\nOpaque identifiers can still be sensitive. Whether to record them is a policy decision, not something the gem can decide. Names, email addresses, request parameters, and free text do not belong in this event by default.\n\nFor a hybrid search:\n\n``` js\nWideEvent.set(\n  \"search.result_count\" => results.size,\n  \"search.semantic\" => search.semantic?,\n  \"search.semantic_fetched\" => search.semantic_fetched,\n  \"search.semantic_kept\" => search.semantic_kept,\n  \"search.semantic_top_cosine\" => search.semantic_top_cosine\n)\n```\n\nFor a phase whose duration matters:\n\n```\nreport = WideEvent.phase(\"report_upload\") do\n  upload_report\nend\n```\n\nThat adds `report_upload.duration_ms`\n\nand returns the block's result.\n\nHandled failures get stable names:\n\n``` js\nrescue Search::Unavailable => error\n  WideEvent.error!(\n    slug: \"err-search-unavailable\",\n    exception: error,\n    expected: true\n  )\n  keyword_results\nend\n```\n\nAn exception that escapes the request or job is recorded automatically with `error=true`\n\nand no slug. The missing slug distinguishes an escaped exception from a failure the application intentionally handled and named.\n\nInstrumentation has a never-raise contract. Attribute writes outside an open request or job are safe no-ops, and failures inside the telemetry code are reported without being raised into application code. The block passed to `phase`\n\nbehaves like normal application code, including propagating its own exceptions.\n\nFree-form attributes become a mess when nobody owns their meaning. Wide Events therefore treats its registry as executable schema rather than documentation.\n\n```\nsearch.semantic:\n  type: boolean\n  set_by: SearchController#index\n  pii: none\n  notes: False when the keyword-only fallback ran.\n\nsearch.semantic_top_cosine:\n  type: float\n  set_by: Search\n  pii: none\n```\n\nThe production workflow is intentionally strict:\n\nThis costs more than inventing a key in a dashboard. In return, both humans and agents get stable names, declared types, ownership, and a place to review privacy.\n\n“One event per request” should not be confused with “one event per user action.” Asynchronous work crosses a real execution boundary and should remain visible.\n\nI inspected one production reply-draft flow. The initiating request and the generation job were linked by an opaque generation identifier, but they were correctly emitted as separate events:\n\n| Field | Request | Generation job |\n|---|---|---|\n| Duration | 230.6 ms | 3,159.7 ms |\n| HTTP status | 204 | n/a |\n| Postgres queries | 59 | 14 |\n| Model latency | n/a | 2,995.96 ms |\n| Input tokens | n/a | 3,035 |\n| Output tokens | n/a | 112 |\n| Cost | n/a | $0.009268 |\n\nThe job event also recorded the prompt version, model, input size, counts of thread messages, notes, goals and todos, and the output character count. It did not record the prompt or completion.\n\nThat distinction is important in an application subject to healthcare privacy requirements. Shape and outcome are useful for production analysis. Content is both risky and usually unnecessary.\n\nThe generation job's full trace contained 35 spans and 288 attributes. Its root event contained 42 attributes. The attribute JSON was 11,534 bytes across the trace and 1,384 bytes on the root event. Again, this does not replace the trace. It gives an agent a compact first read of what happened.\n\nThe gem ships two plain-Markdown skills:\n\n```\nbin/rails generate wide_events:skills\n```\n\nOne skill teaches an agent how to add instrumentation, including naming, registry, testing, and privacy conventions. The other teaches the read path: start with root events, narrow the cohort, inspect individual traces, and verify a deployed change against a new build.\n\nA first query can be small:\n\n```\nSELECT\n  Timestamp,\n  TraceId,\n  SpanAttributes['http.route.controller'] AS route,\n  round(Duration / 1e6, 1) AS duration_ms,\n  SpanAttributes['stats.postgres_query_count'] AS query_count,\n  SpanAttributes['stats.postgres_query_duration_ms'] AS postgres_ms,\n  SpanAttributes['stats.http_call_duration_ms'] AS http_ms,\n  SpanAttributes['error'] AS error\nFROM otel_traces\nWHERE ResourceAttributes['deployment.environment'] = 'production'\n  AND SpanAttributes['main'] = 'true'\n  AND Timestamp > now() - INTERVAL 15 MINUTE\nORDER BY Duration DESC\nLIMIT 20\n```\n\nThe loop is:\n\nThe last step matters. An agent that can edit code but cannot check the running result is still handing the experiment back to a person halfway through.\n\nThe skills do not magically grant production access. The agent still needs a read-only query path, appropriate credentials, and limits on what it may retrieve. In this case, ClickStack exposes ClickHouse through MCP, and the production queries explicitly exclude identifiers and free text.\n\nI originally described unnamed exceptions as a ready-made queue of failures that nobody had handled. The real data made that claim less neat.\n\nThe query was dominated by expected framework noise, especially routing errors. The missing slug still tells you that application code did not name the path, but it does not prove every row deserves engineering attention.\n\nA practical query needs local exclusions or an explicit expected-error policy:\n\n```\nSELECT\n  SpanAttributes['exception.type'] AS exception_type,\n  SpanAttributes['http.route.controller'] AS route,\n  count() AS occurrences\nFROM otel_traces\nWHERE ResourceAttributes['deployment.environment'] = 'production'\n  AND SpanAttributes['main'] = 'true'\n  AND SpanAttributes['error'] = 'true'\n  AND SpanAttributes['exception.slug'] = ''\n  AND SpanAttributes['exception.type'] != ''\n  AND Timestamp > now() - INTERVAL 7 DAY\n  AND SpanAttributes['exception.type'] NOT IN (\n    'ActionController::RoutingError',\n    'ActionController::TooManyRequests'\n  )\nGROUP BY exception_type, route\nORDER BY occurrences DESC\nLIMIT 50\n```\n\nThat is one reason I wanted the article grounded in production rows. A tidy API design can still produce a noisy operational query.\n\nI also checked the basic grain before using the data. In the seven-day production slice ending August 4, 2026, the table contained 1,185,563 root events and exactly 1,185,563 distinct trace IDs. There were no duplicate root rows. That does not prove every deployment will be configured correctly, but it does verify the one-root-row invariant in this deployment at meaningful volume.\n\nWide events are deliberately high-cardinality. You need a backend that handles that shape, plus an explicit retention and privacy policy. ClickHouse is a good fit for my deployment, but operating it is still work.\n\nThe registry adds friction. Teams that will not review attribute names or data classification will not get much value from pretending they have a schema.\n\nRoot events also cannot contain every detail. They should contain the dimensions and outcomes you expect to group, filter, compare, and hand to an agent. Traces, logs, and profiles remain better for lower-level evidence.\n\nFinally, “compact” is workload-dependent. The two traces in this article had roughly 8x and 14x more serialized span-attribute data than their root events. Those are two measured examples, not a universal compression ratio and not a claim about exact model-token usage.\n\n[Wide Events is MIT licensed on GitHub](https://github.com/adammiribyan/wide_events). The log sink produces the first event without requiring an observability stack; the OpenTelemetry sink works with an OTLP backend you already operate.\n\nI extracted the gem from the production Rails application described here. I am especially interested in feedback on three things: the registry workflow, the attribute naming conventions, and whether the generated skills describe an agent verification loop you would actually trust against production.\n\n*Disclosure: I used OpenAI Codex to run sanitized, read-only queries against production telemetry, challenge the claims, and edit this article. I reviewed the source code, query results, and every number before publication.*", "url": "https://wpnews.pro/news/one-rails-request-one-event-production-context-for-coding-agents", "canonical_source": "https://dev.to/adammiribyan/one-rails-request-one-event-production-context-for-coding-agents-47n3", "published_at": "2026-08-04 21:24:09+00:00", "updated_at": "2026-08-04 21:47:11.632702+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "mlops"], "entities": ["Wide Events", "OpenTelemetry", "Rails", "ClickHouse", "HyperDX", "ClickStack"], "alternates": {"html": "https://wpnews.pro/news/one-rails-request-one-event-production-context-for-coding-agents", "markdown": "https://wpnews.pro/news/one-rails-request-one-event-production-context-for-coding-agents.md", "text": "https://wpnews.pro/news/one-rails-request-one-event-production-context-for-coding-agents.txt", "jsonld": "https://wpnews.pro/news/one-rails-request-one-event-production-context-for-coding-agents.jsonld"}}