# Armature turns MCP sessions into analytics and staged evals > Source: > Published: 2026-08-03 17:37:50+00:00 [Theodore Otzenberger (@Totzenberger)](https://x.com/Totzenberger?ref=runtimewire) and Louis Scremin launched [Armature](https://armature.tech/?ref=runtimewire) on July 22 to show software makers what happens when customers use their products through Claude, ChatGPT and other AI clients. Armature reconstructs sessions from Model Context Protocol servers, groups them by user intent and ranks the failures that break the most workflows. The founders arrived at the problem from opposite sides of the same production gap. Otzenberger worked at Palantir and later built observability infrastructure at Tsuga. Scremin led AI automation at Joko, where he helped put agents and MCP servers in front of a consumer product with 6 million users, according to [Y Combinator's profile of Armature](https://www.ycombinator.com/companies/armature?ref=runtimewire). YC lists Armature as a three-person Spring 2026 company. Their founding thesis is grounded in an operational problem Scremin encountered at Joko: an MCP server that performs correctly in internal testing can still fail across millions of unpredictable requests. Otzenberger brought the tracing background needed to turn those interactions into something a product or engineering team can inspect. Otzenberger framed the problem in a launch thread: "You get the raw tool calls, never the user intent." [Theodore Otzenberger on X](https://x.com/Totzenberger/status/2082474871527751892?ref=runtimewire) That distinction is the opening Armature is pursuing. ### From tool calls to product behavior Traditional product analytics instruments the interface controlled by the software vendor. Events such as page views, button clicks and completed funnels can be tied together because the interaction happens inside the vendor's application. An agent-mediated session happens elsewhere. A customer may ask Claude or ChatGPT to create an invoice, update a subscription or reconcile a payment. The software provider receives calls to its tools, but those calls alone may not show the original request, the agent's plan or whether the finished result met the customer's goal. Armature's [launch announcement](https://armature.tech/blog/armature-launch-product-analytics-for-agent-sessions?ref=runtimewire) describes three layers: reconstructed sessions, clustered use cases and grouped issues. The dashboard replays tool calls and outcomes, then Armature's models classify what users were trying to accomplish and identify loops, dead ends and unsupported requests. A workflow can be marked as unsuccessful even when every underlying API request returns a 200 status code. The implementation adds an Armature SDK around an existing MCP server. Armature currently documents SDKs for [TypeScript, Python, Go and PHP](https://docs.armature.tech/?ref=runtimewire). Its [telemetry documentation](https://docs.armature.tech/sdks/telemetry?ref=runtimewire) says the SDK adds an optional telemetry object to each instrumented tool's input schema, with fields for user intent, agent-reported thinking and perceived user frustration. The same documentation says agents that ignore those optional fields still produce sessions containing tool calls, timing and outcomes. That caveat matters. Armature is not independently extracting a complete internal reasoning record from every model. Some of the richer session context depends on the calling agent supplying the optional telemetry fields that Armature adds to the schema. Armature's [telemetry documentation](https://docs.armature.tech/sdks/telemetry?ref=runtimewire) also says analytics-enabled SDKs register an optional capability-request tool for cases in which a user needs something the existing tools cannot do. Those calls give product teams a structured view of unmet demand, turning failed requests into possible roadmap inputs. ### Production evidence becomes regression tests The July 22 announcement said a testing product would follow. By August 3, Armature's [evaluation documentation](https://docs.armature.tech/evaluation/overview?ref=runtimewire) described evaluations as a workspace-by-workspace rollout, with access enabled separately for each workspace. When enabled, the evaluation product runs a real agent against a deployed MCP server using a defined user goal. A separate judge model scores the resulting trace against criteria written by the customer, producing a score from zero to five and an outcome of passed, partial or failed, according to Armature's [scoring documentation](https://docs.armature.tech/evaluation/suites-and-runs?ref=runtimewire). The [evaluation overview](https://docs.armature.tech/evaluation/overview?ref=runtimewire) says runs can start manually, on a schedule or from a continuous integration pipeline. Armature's sharper product decision is the connection between analytics and testing. A recurring production use case can become an eval case. A failure discovered in live sessions can become a regression test intended to prove that a later fix holds. Armature says customers review the generated prompt and criteria before saving the test. That creates a feedback loop spanning observation, prioritization and verification. It also gives Armature a wider surface than a session-replay dashboard alone. The analytics product identifies how externally controlled agents use an MCP server, while the evaluation layer checks whether those same jobs continue to work after a release. Access remains staged. Armature's [documentation](https://docs.armature.tech/evaluation/overview?ref=runtimewire) says evaluation is enabled workspace by workspace, with customers able to request access if the feature does not appear in their account. Free workspaces receive 100 completed runs per month, while scheduling begins on paid plans. ### A category built around the missing interface Armature positions itself between product analytics products such as PostHog, Amplitude and Mixpanel and agent observability products such as LangSmith and Langfuse. Armature's pitch is that the first group measures behavior inside interfaces a vendor owns, while the second is generally oriented toward agents the vendor builds. Armature is focused on outside agents operating the vendor's product through MCP. The distinction gives Otzenberger and Scremin a specific wedge, though established analytics, observability and MCP infrastructure vendors can add overlapping features. Armature's defense will depend on whether its session reconstruction and production-to-eval workflow become difficult enough to reproduce, and whether product teams treat agent behavior as a separate discipline with its own budget. Armature calls that discipline "Agent Experience," or AX. The label is ambitious, covering future questions around conversion, support and retention when an agent stands between the software and the customer. The first product is narrower and easier to evaluate: tell developers what people asked their agents to do, show where the tools failed and convert the costly failures into tests. Armature's [pricing page](https://armature.tech/?ref=runtimewire) lists the first 1,000 analytics sessions each month as free, followed by $50 per additional 1,000 sessions, with seven days of retention on the free plan. Armature says its system scans sessions for personal information and secrets before storage. Its [telemetry documentation](https://docs.armature.tech/sdks/telemetry?ref=runtimewire) says actor identities are represented by SHA-256 hashes derived on the customer's server and that captured input and result previews are limited to 8 KiB each. For Otzenberger and Scremin, the timing rests on a simple bet: MCP servers are becoming a customer-facing surface before most software teams have learned how to measure them. Armature is trying to own that measurement layer while the operating habits, tooling choices and category boundaries are still being written.