Expert Opinion Open models are ready for agents, but the infrastructure around them isn't. Otari is a gateway that restores what hosted APIs quietly handle, like web search and code execution, so you keep control of your providers, credentials, and sandboxes while your app sees the same tool interface.
Back in July, I argued that open models are ready for agents but their APIs aren't. Migrating our agent app off a proprietary frontier API meant rebuilding a surprising amount of underlying infrastructure around the model itself. This piece dives into the part I didn't cover back then: what it actually takes to craft that missing half.
The goal #
Our goal is “straightforward”: building on open models should feel just as seamless as using closed alternatives.
The hardest parts aren't the obvious feature gaps people talk about; they are the subtle, invisible functions hosted APIs quietly perform in the background that your code unconsciously relies on.
We built Otari as the gateway layer to bring those missing capabilities back, sitting neatly between your application and whatever model host you choose.
Why open #
Consider web search as an example. When enabled in Claude, queries execute somewhere behind the scenes on a search engine operating under terms you never directly agreed to. You can't select the underlying provider, track what data gets retained, or swap services if domain results deteriorate. The same applies to code execution: user files run inside sandboxes you didn't configure, hosted in locations you can't verify.
With Otari, these remain your architectural decisions. Web queries route through services you explicitly configure using your own credentials and contracts. Code runs within sandbox environments you specify, governed by an open published protocol that anyone can implement. Your application experiences the exact same tool interface, but you retain full visibility and ownership.
What's next #
While there are still things to implement and polish, we’ve already managed to do this in octonous.com, which has been an important milestone. The remaining technical challenges might not be obvious if you don’t build around different frontier APIs: context compaction, prompt caching, and persistent memory that developers can inspect and migrate across models instead of locking them inside proprietary silos.
The key benchmark we evaluate is whether an agent designed for commercial frontier APIs can point directly at open models through Otari and operate flawlessly.
Feel free to open an issue in our repository on GitHub to share your findings.