The question I am asked most frequently about WisdomAI is around the "secret sauce" behind the technology. Is it a fine-tuned model? Is it a custom reinforcement learning (RL) loop?
The answer is, in fact, a series of optimizations that are only possible because we co-design every layer of the stack to function cohesively as a single product: the Context Layer, the Chat Agent, and the Presentation Layer, which consists of Live Apps and Agents.
This pattern is quite familiar:
- Why did DeepSeek reach frontier-level performance at a fraction of the expected training cost? Its model architecture and training framework were engineered around the hardware it had available (e.g., FP8 training, custom communication kernels).
- Why are Apple products so seamless? Apple builds the hardware and software together.
- Why does Netflix stream high-quality video at lower bandwidth? It controls the encoding, its own CDN (Open Connect), and the client player.
The answer in each case is that these companies are able to co-design the various layers of their stack in order to deliver the most efficient solution.
In this post, we examine some key examples of how this co-design enables WisdomAI to perform considerably more effectively than a system in which the context, harness, and consumption layers are designed and built separately.
Learning loop #
The learning loop is the mechanism through which WisdomAI extracts relevant context from user interactions, and it is critical to accuracy and latency. In many cases, the learning loop improves accuracy considerably (e.g., from ~45–50% to ~85–95%) without any explicit human input.
Data teams want control over their context. In practice, though, reviewing every change gets tiring, open items go untouched, and the system ends up with very few user signals to learn from. So we built a learning loop that learns from how people actually use the product, and context managers decide when to turn it on, much as Google Search derives a substantial portion of its accuracy gains from implicit clicks.
These optimizations (how the agent harness writes context back, checks for conflicts on the fly, and learns from implicit interactions within the product) are effective only because the context layer and the harness are co-designed. Furthermore, learning occurs not only through implicit signals in the chat interface but also through the LiveApps as users create them or interact with them.
Similarly for faster responses, WisdomAI maintains an index of past queries and user interactions. When a new question is asked, rather than regenerating the full response from scratch, the system reuses much of the work from similar previous queries. As a result, thinking and output tokens are used far more efficiently. For example, suppose a past query was "Which product categories drove the most revenue in the last 90 days?" and a new query is "Which product categories drove the most revenue in the last 30 days, by region?" The second query can reuse the result of the first, with only minor edits to the response.
Context indexing and retrieval #
We design our indexes and retrieval tools together, with each directly shaping the other. When we run our in-house evaluations, we optimize the system end to end. In a recent blog post, we described how we evaluated the Claude Code harness against an export of the same context that WisdomAI uses, and WisdomAI was 3x faster and 3.4x cheaper.
One of the key ideas behind this approach is that, when designing the retrieval tool, we can always ask how context is best represented. Unlike many systems that rely only on a filesystem interface (e.g., OKF or OSI), we build additional custom internal embedding indexes for context. As with any search system, aggregating ranking signals from the harness (e.g., which items are used most frequently) allows us to be more accurate.
For instance, suppose we observe that user queries asking for "Revenue for Acme" do not return a valid result when the query is run against "Acme Corporation" in the "Company" column, but do when it is run against "Acme Corp" in the "Customer" column. As a result, "Acme Corp" in the "Customer" column receives a ranking boost during value retrieval. This saves the chat agent multiple retries or, in the worst case, prevents an incorrect response.
Context health #
Finally, even if your use cases remain constant, the context does not. It gradually degrades as your data, business definitions, and use cases evolve. As Anthropic found in its internal data analysis experience, "We watched our offline accuracy drift from ~95% at launch to ~65% over a month before we treated this as an engineering problem."
One of the major benefits of co-designing the harness and the context engine is that the harness can participate in the context management loop. The harness’s key contribution to context health is the evaluation loop. When the context health agent identifies a potential error or conflict in the context, an effective way to determine whether the suggested fix is sound is to trigger a targeted evaluation loop on the change.
Ideally, one would have a large evaluation set with manually labeled ground-truth responses; however, anyone who has attempted to build internal agents knows how difficult this is in practice. In our case, we rerun the harness on a selection of relevant previous user sessions and perform an A/B comparison against the earlier responses using our session grader. If the session grader prefers the responses produced with the new context, this indicates that the fixes are generally improving context health.
The session grader merits a discussion of its own, but at a high level, its goal is to assess the user’s experience by inspecting a chat session. For instance, an empty or partial response, where the user clearly expected more, would be graded lower than a complete response. Similarly, a session in which the user had to try multiple prompts to obtain an answer would be rated lower than one in which they did not.
Conclusion #
At WisdomAI, we are solving various interesting harness, context, model, and infra problems. Co-designing is a key insight behind a lot of what we do, but it’s just one of the various such insights.
If these problems look like your type, join us.