MCPservers, the works) without shipping anything to a server. Most "local AI" apps just wrap a chat completion endpoint. This one actually runs the whole loop on the Neural Engine.
Tried eight different 4B-class models. Most fell apart at tool calling: hallucinating function names, mangling JSON arguments, getting stuck in loops. Qwen 3.5 4B was the only one that consistently produced valid tool calls without heavy prompt engineering. Still needed a few tricks — strict JSON schema enforcement, few-shot examples baked into the system prompt, a tiny retry wrapper that re-prompts on parse failure — but it works.
Performance is what you'd expect. Not winning MMLU. Token generation sits around 12-15 tok/s on a 15 Pro Max, slower on base 15 Pro. Cold start adds ~2 seconds for model load. But it does useful work: file ops, web search via a local MCP server, calendar reads, shortcuts triggers. All private, all offline after first launch.
The app also lets you swap in OpenRouter or OpenAI keys if you want cloud fallback. I've been running it hybrid — local for privacy-sensitive stuff, cloud when I need deeper reasoning.
Biggest surprise: context management on a 4B model with a 32K window. Had to implement sliding window + summarization because the model starts losing the thread after ~8 turns of tool-use history. Summarization step adds latency but keeps the loop coherent.
If you've got a 15 Pro/Max and want to poke at on-device agents, it's worth a look. Not a toy — actual harness. Just don't expect GPT-4o levels of planning. Curious if anyone's gotten smaller models (1.5B, 3B) to do reliable tool calling. The quantization hit seems brutal below 4B.
Next Exit codes lie when PDF extraction yields nothing →