{"slug": "rd-signal-2-frontier-classification-at-production-scale", "title": "Rd-Signal-2: Frontier Classification at Production Scale", "summary": "Raindrop launched Signals 2.0, powered by rd-signal-2, a model pipeline that approaches GPT-5.6 Sol xhigh accuracy while costing 1600x less and 260x less than GPT-5.6 Luna xhigh, available to all customers at no additional cost. The company also released Signal Builder, a platform for training and hosting custom classifiers with Zero Data Retention, targeting strict environments including healthcare.", "body_md": "# rd-signal-2: Frontier Classification at Production Scale\n\nToday we're launching Signals 2.0, powered by rd-signal-2, our new model pipeline for building task-specific binary classifiers from production traces.\n\nrd-signal-2 approaches **GPT-5.6 Sol xhigh accuracy** while costing **1600x less, and 260x less than GPT-5.6 Luna xhigh**.\n\nThis model is available to all Raindrop customers today (at no additional cost).\n\nWe are also releasing Signal Builder: a platform for training and hosting custom classifiers with Zero Data Retention. Signal Builder brings the power of Signals to the strictest of enviroments (including healthcare).\n\n## Everything is a binary classification problem\n\nLast year, OpenAI published a paper called [\"Why Language Models Hallucinate\"](https://arxiv.org/abs/2509.04664). Hallucination, they said, is simply a binary classification problem: any statement is either true or not.\n\nThe LinkedIn headlines were immediately victorious: \"OpenAI solved hallucination.\"\n\nAnd, if being binary made classification easy, they would have. But, as it would turn out, everything in life is just binary classification too. You either should or should not get married; the UI is either good or not.\n\n...and a given agent behavior is either good or bad.\n\nBinary classification is a hard alignment problem. You need to align the humans within a company on the definition of good/bad, and then explore all of the edge cases, and then - and only then - you must align a model or pipeline to that definition.\n\nRaindrop covers the entire journey of training accurate classifiers for agent behavior.\n\n## Classifiers for Agents\n\nIn June 2025, we launched [the first version of Signals](https://x.com/benhylak/status/1935794106493722702): an automated pipeline for training tiny classification models.\n\nAt the time, our classifiers evaluated a single input-and-output pair for a given behavior. That worked for the chatbot era, when the relevant evidence was contained within a single turn. Our competitors still have that limitation.\n\nBut agent failures now unfold across multiple turns, tool calls, and subagents, sometimes spanning hundreds of thousands of tokens. These failures are often nuanced and sparse. Finding them requires both deterministic filtering to assemble the relevant evidence and semantic judgment to interpret it.\n\nRunning a frontier model over every trace is the obvious solution, but, among other problems, it becomes prohibitively expensive and slow at production scale.\n\nSmaller models are affordable, but they struggle with complex behaviors and are often limited by how much context they can consume.\n\n*rd-signal-2* solves this through an automated research loop. For each behavior, it studies production traces, writes code to assemble the context that matters, and uses that context to train a task-specific model.\n\n## How rd-signal-2 builds Signals\n\nWhen an agent produces this trace:\n\nThe failure is not contained in any single step. It is the relationship between the repeated tool failures and the assistant's final response. *rd-signal-2* can express that relationship:\n\nThe code finds the relevant tool calls, compares their inputs, and checks whether they failed. If those conditions are not met, the Signal returns a non-match without calling a model.\n\nIf they are met, the Signal extracts the failed attempts and the assistant's final response from the trace. The remaining semantic judgment goes to a combination of a \"task-specific classification head\" and our in-house semantic reasoning model, optimized for binary classification.\n\nThis was an easy example, constrained to a single turn. Signals are built to efficiently and dynamically collect context from sessions that span days and weeks.\n\n## Reasoning at build time\n\n*rd-signal-2* separates the reasoning required to **construct** a classifier from the computation required to **execute** it.\n\nTraditional LLM Judges waste reasoning tokens rediscovering the same evidence and making slightly different judgments each time.\n\n```\nPrompt classifier cost = traffic × full-trace reasoning\n```\n\n*rd-signal-2* pays a one-time cost that can be spread across every future trace:\n\n```\nrd-signal-2 cost = build once + deterministic execution\n                    + ambiguous candidates × compact context\n```\n\nBy iterating and enforcing strict self-verification, the system can try several approaches before finding one that matches the customer's intent. This pattern is similar to the advisor-executor pattern used by frontier coding harnesses, where a smarter planner constructs a narrow task that a smaller model can perform efficiently.\n\n**Model calls should scale with uncertainty, not traffic.**\n\nAs a result, Signals are inexpensive enough to include with the Raindrop platform and run across billions of traces per month. This stands in stark contrast to platforms like Braintrust and Langchain that require customers to pay out-of-pocket for inference.\n\n## Binary classification is an alignment problem\n\nThe hardest part of building an effective classifier is often discovering what the user actually means. We quickly realized that this is as much a product problem as it is a machine-learning problem.\n\nReturn to the repeated tool-call example. Should the Signal match if:\n\n- the arguments changed slightly but the strategy did not?\n- the first three attempts failed but the fourth succeeded?\n- the agent admitted that the operation failed?\n- the agent handed the task to a subagent that completed it?\n\nEach decision is rooted in our customers' judgment. Translating their nuanced requirements into a faithful classifier is a non-trivial product problem.\n\nTo illustrate how challenging finding this line can be, we wrote four variations of the same one-sentence behavior and ran all four over the same 2,000 production traces. Match rates ranged from 0.9% to 4.6%, and 67% of matched traces were contested by at least one variation. Human-model alignment will remain an active area of research at Raindrop.\n\n## A Signal is never truly finished\n\nBuilding an accurate classifier is just the first step. Raindrop helps keep it accurate.\n\nAs models and harnesses change, the shape of traces changes too. Customers also discover specification errors as they see their Signal operate on more data. A Signal can become less accurate even when its code has not changed.\n\nAfter deployment, we continue watching. Each day we randomly sample production Signals using frontier models. When these evaluations uncover drift or a regression, we rerun prompt optimization and retune.\n\nThis creates a continuous loop:\n\nWe also give customers the power to tune all this themselves. They can directly inspect and tweak the code and prompt that power their Signal.\n\n## Running Signals safely at scale\n\nEvaluating 10,000 traces is easy. Running more than two million a day requires careful model serving, queueing, retries, and isolation.\n\nEvery generated Signal is treated as untrusted by default. Each customer's Signals run in an isolated environment with no credentials or internet egress. An organization's runtime can access only that organization's data.\n\nTrace context is fetched once and cached close to the evaluation workers, so multiple Signals can reuse it instead of repeating expensive queries. This keeps the hot path focused on deterministic execution and the small set of cases that need model judgment.\n\nAt production scale, the median trace is classified in 100 ms. This infrastructure currently evaluates over 20 billion traces per month.\n\n## How rd-signal-2 builds a better Raindrop\n\n*rd-signal-2* already powers more than user-created classifiers inside Raindrop.\n\nOur Issue Detection system shares the same architecture to identify, track, and monitor emerging failure modes in our customers' data distributions. This allows our customers to insantly turn any issue detected into a long-standing Signal. From there, they can refine the policy, use Experiments to A/B test, and more.\n\nSignals can be created by a user, constructed through our Triage Agent, generated by your coding agent via MCP, or used by systems like Raindrop Issue Detection.\n\nOnce deployed, they all run through the same evaluation and monitoring infrastructure.\n\n## Signals 2.0 API\n\nSignals aren’t limited to just the Raindrop platform. Developers can now call the Signals API directly to build domain-specific detection into their own systems.\n\nFor teams with stricter data requirements, ZDR Signals supports training and running task-specific classifiers without retaining production data.\n\n## Try Signals 2.0\n\nSignals 2.0 is available today in Raindrop.\n\nCreate your first Signal in Raindrop: [Get started](https://app.raindrop.ai/signup), or [schedule a call with our team](https://cal.com/team/raindrop-ai/chat-15-min) to build one together.", "url": "https://wpnews.pro/news/rd-signal-2-frontier-classification-at-production-scale", "canonical_source": "https://www.raindrop.ai/blog/signals-2-frontier-classification/", "published_at": "2026-08-11 20:55:44+00:00", "updated_at": "2026-08-11 21:11:39.432083+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-products", "ai-tools", "ai-infrastructure"], "entities": ["Raindrop", "rd-signal-2", "GPT-5.6 Sol xhigh", "GPT-5.6 Luna xhigh", "Signal Builder", "OpenAI"], "alternates": {"html": "https://wpnews.pro/news/rd-signal-2-frontier-classification-at-production-scale", "markdown": "https://wpnews.pro/news/rd-signal-2-frontier-classification-at-production-scale.md", "text": "https://wpnews.pro/news/rd-signal-2-frontier-classification-at-production-scale.txt", "jsonld": "https://wpnews.pro/news/rd-signal-2-frontier-classification-at-production-scale.jsonld"}}