{"slug": "kinetic-4b-vs-claude-haiku-4-5-the-4b-model-wins-tools", "title": "Kinetic-4B vs Claude Haiku 4.5: The 4B Model Wins Tools", "summary": "Bengaluru lab Conscious Engines released Kinetic-4B, a 4-billion-parameter model that scored 82.33% accuracy at 1.61s p95 latency on a 300-sample Composio tool-calling benchmark, edging out Anthropic's Claude Haiku 4.5 (80.0% at 4.02s) and OpenAI's GPT-OSS-120B (76.33% at 7.99s). The model was produced by LoRA fine-tuning at rank 64 on 13,694 samples, training 132 million parameters in about 4.5 hours on a single rented GPU. Conscious Engines reports the key gain is a halved failed-call rate of 4.67% versus Haiku's 9.67%, though it notes the base Qwen3-4B already carried most of the capability.", "body_md": "Kinetic-4B wins on tool calling. On a 300-sample Composio evaluation, the 4-billion-parameter model from Bengaluru lab Conscious Engines scored 82.33% accuracy at 1.61s p95 latency, against 80.0% and 4.02s for Anthropic's Claude Haiku 4.5, and 76.33% at 7.99s for OpenAI's GPT-OSS-120B ([Conscious Engines, 1 April 2026](https://consciousengines.com/blog/kinetic-4b-a-4-billion-parameter-model-that-outperforms-claude-haiku-at-tool-calling)). That is 2.5x lower tail latency at slightly higher accuracy. Haiku remains the better buy if you need one model to also write, summarise, reason and code; Kinetic-4B wins only the narrow job it was trained for, which happens to be the job most agents actually do.\n\nThe honest answer in 2026 is both, in different slots. A frontier or mid-tier LLM handles open-ended reasoning, ambiguous instructions and code. A small, fine-tuned model handles the high-volume, narrow, structured step: pick the right tool from a list, fill the arguments correctly, return JSON, do it in under two seconds.\n\nTool calling is exactly that kind of step. There is a correct answer, the output shape is fixed, and latency compounds because agents chain several calls per user action. A 4-billion-parameter model that gets the schema right is more useful there than a 120-billion-parameter model that gets it right slightly less often and four times slower.\n\nThe evaluation used 300 samples drawn from Composio, an agent tool-integration platform with more than 800 integrations covering GitHub, Slack, Salesforce, Stripe, Notion and Jira. Each prompt offered 10 candidate tools: one correct tool plus nine distractors drawn from the same toolkit, which is the realistic failure mode. Every model saw the same set with the same seed and `tool_choice=auto` ([Conscious Engines](https://consciousengines.com/blog/kinetic-4b-a-4-billion-parameter-model-that-outperforms-claude-haiku-at-tool-calling)).\n\n| Model | Params | Accuracy | Tool-name accuracy | Failed calls | p95 latency | \n|---|---|---|---|---|---|\n| Kinetic-4B | 4B | 82.33% | 95.33% | 4.67% | 1.61s | \n| Claude Haiku 4.5 | undisclosed | 80.0% | 90.33% | 9.67% | 4.02s | \n| Qwen3-4B-Instruct-2507 (base) | 4B | 78.67% | 95.0% | 5.0% | 1.84s | \n| GPT-OSS-120B | 120B | 76.33% | 94.67% | 5.33% | 7.99s | \n| FunctionGemma | 270M | 49.67% | — | 19.67% | 0.84s | \n\nAll figures from the [Conscious Engines benchmark writeup](https://consciousengines.com/blog/kinetic-4b-a-4-billion-parameter-model-that-outperforms-claude-haiku-at-tool-calling), published 1 April 2026.\n\nThe gap that matters is not the 2.33-point accuracy lead. It is the failure rate: 4.67% versus 9.67%. A failed call is a malformed or unusable request that your agent has to catch and retry, and retries are where latency budgets die. Halving that rate changes the shape of the error handling you need to write.\n\nAgainst the base model it was fine-tuned from, the improvement is smaller than the headline suggests. On the same set, 222 samples were answered correctly by both Kinetic-4B and stock Qwen3-4B. Fine-tuning fixed 25 and broke 14, a net gain of 11. That is a real improvement, and also a reminder that most of the capability was already in the open base model.\n\nLoRA at rank 64, alpha 128, targeting all projections. That trained 132 million parameters, about 3.18% of the 4.15-billion total, at a learning rate of 2e-4 for two epochs, in roughly 4.5 hours on a single rented GPU. The dataset was 13,694 samples built from Composio's top-20 toolkits with the same one-correct-plus-nine-distractors structure ([Conscious Engines](https://consciousengines.com/blog/kinetic-4b-a-4-billion-parameter-model-that-outperforms-claude-haiku-at-tool-calling)).\n\nAn earlier attempt at a full fine-tune on generic function-calling data reached only about 71%, below the untouched base model. Freezing the base weights and adapting through a low-rank update preserved general instruction-following that the full fine-tune had eroded. If you are doing this yourself, that is the transferable lesson: on a narrow task with a strong base, LoRA is the safer default.\n\nThe output ships as an adapter, `consciousengines/Kinetic-FC-LoRA`, applied to Qwen3-4B-Instruct-2507. You can reproduce the model rather than take the numbers on trust, which is more than most benchmark claims offer. For the wider picture on small open models, see our comparisons of [GLM-5.3 Flash, Qwen 3.8 Flash Next and DeepSeek V4 Flash](https://dev.to/articles/glm-5-3-flash-vs-qwen-3-8-flash-next-vs-deepseek-v4-flash) and [the best local LLMs for coding](https://dev.to/articles/best-local-llm-for-coding-qwen3-8-27b-vs-glm-5-3-vs-gemma-4).\n\nConscious Engines works out of Indiranagar in Bengaluru, founded by Kautuk Kundan, previously of Ethereum-scaling company Stackr Labs. The lab is backed by a16z and, per the founder's own public statements as of 10 September 2026, is running close to $1 million in annualised revenue ([founder bio](https://x.com/Kautukkundan/highlights)). The Kinetic-4B work came out of a LossFunk residency, authored with Ritam Pal.\n\nThe commercial thesis is a right-sized model per use case, with frontier models used as teachers for distillation rather than as the thing you ship. Deployments described by the founder sit in healthcare and manufacturing, including models running alongside production lines, with robotics the fastest-growing area of interest. The team also reports running a language model doing real-time on-device inference on an Apple Watch. Treat that as a capability demonstration, not a benchmark.\n\nThe cost argument is more contestable. Claude Haiku 4.5 lists at $1 per million input and $5 per million output tokens ([Anthropic](http://anthropic.com/claude/haiku)), which is cheap in absolute terms. The counter-argument from Conscious Engines is that large enterprise discounts on frontier APIs, in the 75-80% range, hide the true unit cost and encourage lock-in. The founder also describes spending around $35 of API tokens on a task that a local model on his own machine ran at no marginal cost - a self-reported anecdote from his 10 September 2026 AIM interview ([founder statements](https://x.com/Kautukkundan/highlights)), not a measured comparison. Our [distillation coverage](https://dev.to/articles/white-house-moonshot-ai-distillation-anthropic-fable-kimi-k3) covers the same teacher-student pattern at policy scale.\n\nThree limits are worth stating plainly. First, it is a single 300-sample eval set, self-published by the party with an interest in the outcome, and not independently replicated. Second, it measures one capability. Haiku 4.5 reports 73.3% on SWE-bench Verified ([Anthropic](http://anthropic.com/claude/haiku)); Kinetic-4B makes no comparable claim, and a tool-router is not a coding model. Third, the distractor design, nine same-toolkit alternatives, is the setup Kinetic-4B was trained on. Real production traffic is messier.\n\nThe practical reading: route the structured, repetitive tool calls to a small fine-tuned model, keep a general model behind it for anything ambiguous, and measure your own p95 rather than anyone's table. If raw speed is your constraint, our [fastest-LLM comparison for 2026](https://dev.to/articles/fastest-llm-2026-mercury-2-5-vs-gpt-5-6-luna-vs-gemini-3-5-flash-lite-vs-claude-haiku-4-5) covers the hosted options, and [open-source coding models](https://dev.to/articles/best-open-source-llm-for-coding-deepseek-vision-update-2026) covers the self-hosted end.\n\n**Q: Does Kinetic-4B replace Claude Haiku 4.5?**\n\n**A:** No. It beats Haiku 4.5 on one benchmark measuring tool-call accuracy and latency. Haiku remains the general-purpose choice for reasoning, writing and code, and Kinetic-4B makes no claim on those tasks.\n\n**Q: Can I run Kinetic-4B myself?**\n\n**A:** Yes. It is published as a LoRA adapter, `consciousengines/Kinetic-FC-LoRA`, applied on top of Qwen3-4B-Instruct-2507. Applying the adapter to that base reproduces the model, so the benchmark is checkable rather than closed.\n\n**Q: In a small language model vs LLM decision, what should drive the choice?**\n\n**A:** Task breadth. If the output shape is fixed and the volume is high, a fine-tuned small model usually wins on latency and cost. If inputs are open-ended or the task changes weekly, a larger general model is cheaper than maintaining fine-tunes.\n\n**Q: How much did the fine-tune cost to train?**\n\n**A:** The published run used LoRA on a single rented GPU for about 4.5 hours over 13,694 training samples, with 132 million trainable parameters ([Conscious Engines](https://consciousengines.com/blog/kinetic-4b-a-4-billion-parameter-model-that-outperforms-claude-haiku-at-tool-calling)). No cluster and no multi-week schedule were involved.\n\n**Q: Is the 82.33% figure independently verified?**\n\n**A:** Not yet. It comes from the lab's own April 2026 writeup using 300 Composio samples with a fixed seed. The methodology is documented and the adapter is public, but no third party has published a replication.\n\n**Q: What is Composio's role in the benchmark?**\n\n**A:** Composio is an agent tool-integration platform with over 800 integrations across services like GitHub, Slack, Stripe and Jira. Its tool schemas supplied both the training data and the evaluation prompts, which is also why the result is domain-specific.\n\nLast verified: 10 September 2026.", "url": "https://wpnews.pro/news/kinetic-4b-vs-claude-haiku-4-5-the-4b-model-wins-tools", "canonical_source": "https://dev.to/shaam_ai/kinetic-4b-vs-claude-haiku-45-the-4b-model-wins-tools-53h4", "published_at": "2026-09-23 03:43:49+00:00", "updated_at": "2026-09-23 03:52:46.820489+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "ai-research", "ai-tools", "mlops"], "entities": ["Conscious Engines", "Kinetic-4B", "Claude Haiku 4.5", "Anthropic", "OpenAI", "GPT-OSS-120B", "Qwen3-4B-Instruct-2507", "Composio"], "alternates": {"html": "https://wpnews.pro/news/kinetic-4b-vs-claude-haiku-4-5-the-4b-model-wins-tools", "markdown": "https://wpnews.pro/news/kinetic-4b-vs-claude-haiku-4-5-the-4b-model-wins-tools.md", "text": "https://wpnews.pro/news/kinetic-4b-vs-claude-haiku-4-5-the-4b-model-wins-tools.txt", "jsonld": "https://wpnews.pro/news/kinetic-4b-vs-claude-haiku-4-5-the-4b-model-wins-tools.jsonld"}}