Currently in industry we are seeing a shift in narrative from tokenmaxxing–the idea that squeezing the maximum tokens out of frontier models equates to being productive– towards ROI (return on investment) driven AI spend. In this blog, we want to help you build a mental model for thinking about resource-efficient agentic development, and zoom in to understand where open-weight models fit into this.
We like to think of resource efficiency as a switchback trail up a steep mountain. You zigzag back and forth to turn an impossible incline into a manageable climb. Resource efficiency works in a similar way: it asks you to alternate between two motions instead of pushing in one direction the whole time.
Think of this as a continuous cycle, not necessarily a one-time effort. Periodically, pay the upfront cost to push capabilities forward: add new tool calls, introduce a new subagent, or switch to a more intelligent model. Then level out and optimize: delegate easier tasks to open-weight models, reduce prompt context, or run knowledge distillation.
This switchback mental model asks you to think of resource efficiency as an ongoing discipline rather than a one-off cleanup project. Additionally, it frames resource efficiency not as mere cost reduction; rather as active cost management, where we spend resources to buy new capabilities while keeping the overall cost of the system within bounds.
Within this broader framework, we want to talk about our belief that open-weight models will play a significant role in the resource efficiency effort. July and August 2026 have been notable for open-weight models, with GLM-5.2 and Kimi K3 shipping. Both GLM-5.2 and Kimi K3 are now considered state of the art within their class: Kimi K3 landed at #3 overall on the Artificial Analysis Intelligence Index, overtaking GLM-5.2 as the top open-weight model and landing within range of Opus 4.8 and GPT-5.5. Industry is also expecting this trend to continue accelerating with multiple labs doubling down on open-weight strategy.
If you’ve been tracking any of this, you’re probably feeling pressure to decide how you should be using open-weight models in your own workflow. This blog is our attempt to help you answer that, and to make the case that having your own private eval is crucial to answering it. A private eval is built from your own production traces and task types, not from public benchmarks. It answers one question: can this model handle this task in your workflow?
The Limits of Aggregate Benchmarks: Why You Need a Private Eval #
What is considered state of the art in aggregate benchmarks is not necessarily the same as state of the art for your product. A leaderboard isn’t an accurate representation of the actual distribution of queries the production agent sends to the model every day. Some of those queries will not notice a swap to an open-weight model, while others may break in ways public leaderboards generally won’t be able to catch.
We wrote about this before, in Your AI Systems Need an Eval Loop, where we laid out an observe, evaluate and optimize framework for catching and fixing an agent’s failure modes. Here we’re using that same eval loop for a different job: to choose the most efficient model for your workflow.
Stage 1: Task-Level Model Swapping #
The easiest move is to swap a frontier call for an open-weight model and see what happens. Plenty of production tasks don’t need frontier reasoning at all. A solid answer, fast and cheap, is enough, and that’s a nice sweet spot for open-weight models.
You don’t need a fancy eval for this stage. Take your real production traces, run the candidate model against them, and look at the results task by task rather than relying on a single aggregate number. You want open-weight models to be the right fit at the task level, not necessarily at the workflow level. Otherwise, if a model struggles with a single complex task, that mistake can compound across the entire agentic workflow.
For instance, OpenNash shared a study where a logistics company swapped document classification and line-item extraction from GPT-4 to a fine-tuned Llama 3.3 70B and cut costs by 73% with no complaints. However, they kept exception routing on GPT-4, because that task needed multi-step reasoning about contract terms. When applied to that harder task, the open-weight model kept hallucinating clause references that weren’t in the document. To figure out the right balance between open-weight and frontier models, you need an eval on your own traces that generates accuracy, cost, and latency metrics per task. For tasks where the cost-accuracy tradeoff makes the most sense, adopt open-weight models, and delegate rest to the frontier models.
An illustrative example of a scorecard that can help with a model swap decision.
Illustrative example, not real reported figures.
Stage 2: Query-Level Dynamic Routing #
In most production workflows, the reality is often messier. For a given task, the majority of queries are fine on an open-weight model, but the long tail of harder requests presents a much greater challenge. Since those edge cases are often exactly what differentiates our products, we can’t afford to lose efficacy when handling them. In practice, that is why a simple task-level swap often falls short. Instead of making a blanket choice for an entire task, it is generally much more effective to route between models at the query level.
Pattern A: Model Cascading
One way to approach this is to send every request to the smaller, open-weight model first, and escalate to a frontier model only when the smaller model’s internal uncertainty signal triggers an alert.
We tackled this challenge in a project we worked on with Sublime Security*. As we detailed in The Craft of Designing Resource-Efficient Agents, we built a system where an open-weight model handles email security triage by default, passing edge cases up to a frontier model only when its confidence drops below a set threshold. This setup yielded a 16x latency reduction and a 70% cost reduction with no apparent drop in accuracy, because the expensive model only ran when necessary.
The tradeoff here is that escalated queries pay for two model calls instead of one, so cascading helps average latency but adds a delay for the queries that do escalate.
Pattern B: Custom Predictive Routers
The other pattern is a custom router. This approach relies on a classifier trained up front that scores each query and sends it straight to the model most likely to handle it, skipping the escalation step entirely. Folks are taking various technical approaches here, and we believe the right recipe for designing the perfect router is still in the research phase.
For instance, Applied Compute designed an agentic router trained through reinforcement learning (RL). They ran three rollouts each from Nemotron 3 Ultra, Opus 4.7, and GPT-5.5 across 497 SWE-bench Verified tasks. The model that gave the correct answer at the cheapest cost got a reward of 1, while the other two received a reward of 0. They then trained a router on those labels with GRPO (Group Relative Policy Optimization). The result beat the individual frontier models on both cost and accuracy, delivering about a 76% pass rate at a roughly 25% lower cost than running GPT-5.5 alone. Dynamic routing is rapidly becoming an industry-standard feature. For instance, Ramp Router sends every API call to the cheapest model that clears a strict quality bar. Similarly, Cursor’s router, which was trained on over 600,000 live coding requests, cuts costs by 30 to 60% compared to defaulting to Opus 4.8.
Pattern C: Frontier Model as a Callable Advisor
A third pattern: instead of a router deciding upfront which model handles a query, you run the open-weight model as the primary worker and give it the frontier model as a tool it can call mid-task, whenever it decides it needs a second opinion.
A joint study from Fireworks and Harvey tested this on legal drafting tasks, pairing an open-weight GLM-5.1 worker with Claude Opus 4.7 as an on-demand advisor. The worker called the advisor only 0.83 times per task on average, but that sparse use added six more fully passing tasks over the GLM-5.1 baseline. It also beat Opus running alone on both cost and accuracy: 18 tasks fully passed versus 14 for Opus by itself, at $368 total cost versus Opus’s $954.
Source: Fireworks and Harvey, open-source agents with frontier advisors We shared these three patterns to give you a sense of how the industry is thinking about designing model routers. Router design remains a highly active research area. We plan to share more blogs on routing strategies in the future, but for now, we would like to point you to a useful survey paper on the topic: Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey.
Stage 3: Owning your intelligence #
After running Stage 1 and Stage 2 evals for a while, you will likely have a labeled dataset showing exactly where the failure modes are for both frontier and open-weight models on your specific workloads. These failure modes can take various forms: the model fails on certain hard queries, the cost blows up on a set of queries, or the model is wasting significant context by reasoning the same things repeatedly. In these scenarios, consider building a strong agentic harness that can help steer the model in the right direction. Even with a strong harness, however, some failure modes persist. It is at this point that owning your own model weights may become the economical solution.
Now, owning your intelligence can mean a lot of things depending on who you ask. This can range from supervised fine-tuning (SFT) and LoRA (low-rank adaptation), to on-policy distillation or modern continual learning methods on top of open-weight models. While there is no single right recipe here, this is another area where your private evals can help you decide which technique you should rely on.
Cursor’s Composer 2.5 is one public example of this: it’s a fine-tune of Kimi K2.5, an open-weight MoE (mixture-of-experts) model, trained with RL on long-horizon coding tasks using 25x more synthetic data than the base checkpoint. On coding benchmarks it matches GPT-5.5 and Opus 4.7, at a fraction of their API cost.
Post-training isn’t free either. For instance, if you decide to fine-tune your model, that tuning does not automatically carry over every time the base model ships a new version. You will likely have to redo the tuning work from the new checkpoint. That is a real, ongoing tech debt that needs to be understood and carefully managed.
There might also be unintended side effects from these tunings. While fine-tuning can bring your model closer to the target task, performance can degrade on tasks you might not be testing for. For instance, a model fine-tuned for code generation will probably work fine for code review too. But it may become risky if you push it onto something unrelated, like using that same model for your support chatbot without testing it against evals built for that specific task. Once again, the emphasis here is to build your evals for the actual tasks the model is catering to, not just the task you originally fine-tuned it for.
Open-weight models are opening interesting pathways for us to utilize intelligence across different points on the Pareto frontier. The three-stage framework, swap, route and own, climbs that switchback mountain one step at a time. Each stage adds technical complexity, and each one gives you more control over the cost-accuracy tradeoff.
Artificial Analysis Intelligence Index vs cost, representing the Pareto frontier.
There are a lot of compelling reasons for owning your intelligence. But how you get there requires technical deliberation, and there is no single one-size-fits-all recommendation. This choice of adaptation is dictated by the complexity, scale, and, most importantly, the accuracy and price sensitivity of your workflows. Those are questions only your private eval can answer. Our hope is that this post gives you a clear mental model for working through that transition.
Additional Reading #
If you’re interested in this topic, here is a list of readings we find useful.
- [Applied Compute’s agentic router research](https://www.appliedcompute.com/research/training-an-agentic-router) , Applied Compute
- [Ramp Router](https://router.com/) , Ramp
- [Cursor’s Composer 2.5](https://cursor.com/blog/composer-2-5) , Cursor
- [Fireworks and Harvey’s frontier-advisor study](https://fireworks.ai/blog/open-source-agents-frontier-advisors) , Fireworks AI
- [Your AI Systems Need an Eval Loop](https://georgianailab.substack.com/p/your-ai-systems-need-an-eval-loop) , Georgian AI Lab
- [The Craft of Designing Resource-Efficient Agents](https://georgianailab.substack.com/p/the-craft-of-designing-resource-efficient) , Georgian AI Lab
- [AI-based source code security scanning](https://georgianailab.substack.com/p/ai-source-code-analysis-in-the-new) , Georgian AI Lab
- [From Static to Adaptive: Scaling AI Reasoning Without the Waste](https://georgianailab.substack.com/p/from-static-to-adaptive-scaling-ai) , Georgian AI Lab
- [Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey](https://arxiv.org/abs/2603.04445) , Moslem and Kelleher
- [DeepSeekMath: the paper that introduced GRPO](https://arxiv.org/pdf/2402.03300) , DeepSeek-AI
Kshitij Jain is an AI Tech Lead in the Georgian AI Lab, where he helps portfolio companies to implement scalable AI solutions and refine their technical roadmaps. Prior to joining Georgian, he spent five years at Google working on Search Ads CTR prediction models, where he led the team’s resource-efficient ML efforts.
Grateful to David Poole, Aryan Khurana and Siddharth Arya for their thoughtful feedback on this piece.
** Sublime Security is a Georgian portfolio company.*
This blog is provided for informational purposes only and should not be relied upon as legal, business, investment, or tax advice. Nothing in this blog constitutes investment advice, nor is it intended for use by any investors or prospective investors in any Georgian funds. This blog may include links to external websites or information obtained from third-party sources. Georgian has not independently verified and makes no representations regarding the accuracy or completeness of such information, whether current or ongoing. If this content includes third-party advertisements, Georgian has not reviewed such materials and does not endorse any advertising content or the companies referenced.
Any investments or portfolio companies mentioned are for illustrative purposes only and may not be representative of all investments made by funds managed by Georgian. Please contact Georgian for more information.