Berkeley researchers synthesize Mac-specific LLM engines up to 2.72x faster than tested alternatives

UC Berkeley EECS PhD student Mert Cemri and six co-authors released JITEngine on November 4th, an open-source set of six LLM serving engines synthesized for specific Apple Silicon Macs and specific mixture-of-experts models, which Cemri said run up to 2.7x faster than the fastest alternatives the team tested. The project from UC Berkeley's Sky Computing Lab used coding agents to generate the model-and-Mac-specific engines, with the headline benchmark reaching 2.72x faster single-user decoding. The work tests whether coding agents can replace months of specialist inference engineering with a repeatable synthesis loop tuned to one model, one chip and one memory budget.

Berkeley researchers synthesize Mac-specific LLM engines up to 2.72x faster than tested alternatives UC Berkeley's Sky Computing Lab used coding agents to synthesize six model-and-Mac-specific serving engines, reporting up to 2.72x faster single-user decoding. By Ryan Merket https://runtimewire.com/author/ryan-merket · Published Primary source: x.com https://x.com/mertcemri/status/2118057463488589863 Why it matters JITEngine tests whether coding agents can replace months of specialist inference engineering with a repeatable synthesis loop tuned to one model, one chip and one memory budget. If the approach generalizes beyond these six builds, the advantage in local AI shifts from owning the best general-purpose server to generating the right engine for the hardware already on a developer's desk. Mert Cemri @mertcemri https://x.com/mertcemri , a UC Berkeley EECS PhD student, and six co-authors released JITEngine on November 4th, a set of LLM serving engines synthesized for specific Apple Silicon Macs and specific mixture-of-experts models. Cemri said in a thread on X https://x.com/mertcemri/status/2118057463488589863 that the engines run up to 2.7x faster than the fastest alternatives the team tested. The project comes from UC Berkeley's Sky Computing Lab and is released as open source, not as a commercial product. When you purchase through links in our articles, we may earn a small commission https://runtimewire.com/affiliate-monetization-standards . This doesn’t affect our editorial independence.