Qwen 3.5 4B actually runs a full agent loop on iPhone 15 Pro

Qwen 3.5 4B, a large language model from Alibaba, successfully runs a full agent loop on an iPhone 15 Pro, including tool calling via MCP servers, without server-side processing. The developer tested eight 4B-class models and found Qwen 3.5 4B the only one to consistently produce valid tool calls, achieving 12-15 tokens per second on a 15 Pro Max. The app supports local execution for privacy and optional cloud fallback via OpenRouter or OpenAI keys.

Qwen 3.5 4B actually runs a full agent loop on iPhone 15 Pro MCP /en/tags/mcp/ servers, the works without shipping anything to a server. Most "local AI" apps just wrap a chat completion endpoint. This one actually runs the whole loop on the Neural Engine. Tried eight different 4B-class models. Most fell apart at tool calling: hallucinating function names, mangling JSON arguments, getting stuck in loops. Qwen 3.5 4B was the only one that consistently produced valid tool calls without heavy prompt engineering. Still needed a few tricks — strict JSON schema enforcement, few-shot examples baked into the system prompt, a tiny retry wrapper that re-prompts on parse failure — but it works. Performance is what you'd expect. Not winning MMLU. Token generation sits around 12-15 tok/s on a 15 Pro Max, slower on base 15 Pro. Cold start adds ~2 seconds for model load. But it does useful work: file ops, web search via a local MCP server, calendar reads, shortcuts triggers. All private, all offline after first launch. The app also lets you swap in OpenRouter or OpenAI keys if you want cloud fallback. I've been running it hybrid — local for privacy-sensitive stuff, cloud when I need deeper reasoning. Biggest surprise: context management on a 4B model with a 32K window. Had to implement sliding window + summarization because the model starts losing the thread after ~8 turns of tool-use history. Summarization step adds latency but keeps the loop coherent. If you've got a 15 Pro/Max and want to poke at on-device agents, it's worth a look. Not a toy — actual harness. Just don't expect GPT-4o levels of planning. Curious if anyone's gotten smaller models 1.5B, 3B to do reliable tool calling. The quantization hit seems brutal below 4B. Next Exit codes lie when PDF extraction yields nothing → /en/threads/7065/