Show HN: Building a full agentic harness around a 4B model is hard

A developer built a full agentic harness for on-device iPhone LLM inference, using Qwen 3.5 4B after testing 8 models, and released it for iPhone 15 Pro and above. The harness includes inference, agentic loop, context management, tools, and MCP servers, and supports optional OpenRouter or OpenAI API keys.

Around 3 months ago, we were thinking why none of the iPhone apps running an LLM are built as a full harness as in inference + agentic loop + context management + tools + MCP servers and etc. . It became more interesting when we noticed even the new Siri is not fully on device and not available in EU for that matter . Having built a few agentic products around a custom harness in the past, we thought this shouldn't be that hard. well, we underestimated how "dumb" a 4B model can be, especially when it comes to tool calling. :D We tried 8 different models and we settled on Qwen 3.5 4B and we used every trick we knew to make this model behave. well, it works It's not gonna win in any intelligence or speed benchmark, but it can do actual useful work and it really is an on device, private, full agentic harness. That said, you can still connect your OpenRouter or OpenAI API keys if that's what you prefer. Please go check it out : You need an iPhone 15 pro or above. Comments URL: https://news.ycombinator.com/item?id=49359341 https://news.ycombinator.com/item?id=49359341 Points: 1 Comments: 0