{"slug": "run-many-agents-3x-faster-on-your-mac", "title": "Run many agents 3x faster on your Mac", "summary": "Generated Computers released Locals, a local inference server for running AI agents on a Mac that it says completes work up to 3x faster than the standard way of running models locally, and 2.5x sooner than mlx-lm with llguidance for sixteen tool-calling agents. Locals is built on Arip Asadulaev's 2026 paper \"Breaking the Space Barrier and its Application to Language Model Inference,\" which constrains tool-call and JSON output on the GPU using about 3 MB of memory per format instead of up to 1.5 GB. The server works with OpenCode, OpenHands, Cline, Aider, Claude Code and any OpenAI, Anthropic or Ollama app, keeps agent system prompts of 3,000 to 16,000 tokens on disk across restarts, and starts the next turn of a 138,000-token conversation after 2.4 seconds.", "body_md": "**Run many AI agents at once on your Mac, up to 3x faster.**\n\n  A local server made for agents rather than plain chat: tool calls, long\n  conversations and many agents working at the same time. For OpenCode,\n  OpenHands, Cline, Aider, Claude Code and any OpenAI, Anthropic or Ollama app.\n\n[**Getting started**](#getting-started) ·\n  [**Agents**](#works-with-your-agents) ·\n  [**Benchmarks**](https://github.com/Generated-Computers/Locals/blob/main/docs/benchmarks/README.md) ·\n  [**Documentation**](#documentation) ·\n  **Paper**\n\n```\ncurl -LsSf https://raw.githubusercontent.com/Generated-Computers/Locals/main/install.sh | sh\n```\n\nOr, if you use [uv](https://docs.astral.sh/uv/), install it from [PyPI](https://pypi.org/project/locals/):\n\n```\nuv tool install --python 3.12 locals\n```\n\nThen, in a new Terminal window:\n\n```\nlocals code ~/my-project\n```\n\nThat is all. The first time, Locals downloads a model that suits your Mac (about 3 GB), starts it in the background and opens the OpenCode assistant in your project. Nothing leaves your computer.\n\nLocals runs AI models on your own Mac for agents, not for chatting with a raw model. An agent works differently from a person in a chat. It calls tools, and every call has to be right. It sends a long prompt that grows a little every turn. And it rarely works alone: coding assistants run helpers in parallel, and teams of agents split one task between them. Locals is built for this work, from the way it computes a batch of requests on your Mac's graphics chip to the way it writes a tool call, so a team of agents gets its answers up to three times faster than with the standard way of running models on a Mac.\n\nIt works with the coding agents you may already use, such as OpenCode, Cline, Aider, OpenHands and Claude Code, and with any app or program that talks to OpenAI, Anthropic or Ollama. Everything stays on your computer.\n\nLocals is built on the paper\n[Breaking the Space Barrier and its Application to Language Model Inference](https://arxiv.org/abs/2610.09139)\n(Arip Asadulaev, 2026). A JSON format or a tool call can be described as a\ngraph, and checking a reply against it as finding a path in that graph. The\npaper does this in less memory than Savitch's classical bound and turns it into\nan inference engine: text the format already decides is written without running\nthe model, the checks run on the graphics chip, and a format needs about 3 MB of\nmemory instead of up to 1.5 GB. With it, sixteen tool-calling agents finish 2.5\ntimes sooner than with mlx-lm and llguidance.\n\n- **Many agents at once.** Requests from every agent are computed together, with\nGPU kernels written for a few requests at a time, so eight agents working at\nonce finish in about the time two would need one after the other.\n- **Tool calls that always parse.** Replies are written in each model's own\ntool-call format under a grammar applied on the GPU, so an agent can call only\nthe tools it offered, every argument arrives in the type its schema asks for,\nand a reply cut off by its length limit still closes its call. Requests for\nJSON follow their schema the same way.\n- **Each turn picks up where the last one ended.** Locals keeps every\nconversation between turns, and the long system prompt an agent tool starts\nwith (3,000 to 16,000 tokens) on disk across restarts, so an agent's next turn\nreads only what it added: the next turn of a 138,000-token conversation starts\nafter 2.4 seconds.\n- **Guards made for agents.** A small model can repeat itself or call the same\ntool forever; Locals notices and stops both. A memory guard keeps many agents\nwith long prompts from ending in an out-of-memory error, and an agent that\nhangs up frees the server at once.\n- **One command per agent tool.**`locals code` opens OpenCode in your project,\nand`locals launch` connects Hermes Agent, OpenClaw, Claude Code, Kilo Code,\nCline, OpenHands and Aider.\n- **A team of agents for bigger tasks.**`locals run` splits a task into parts,\ngives each part to several agents at once, each in its own copy of your\nproject, and lets your tests decide.\n\nmlx-lm, the standard way of running models on a Mac, answers several requests together, but it gets no faster from four of them to eight: it writes 87 tokens per second for four requests and the same 87 for eight. Locals keeps getting faster as requests are added, up to three times mlx-lm's speed with eight at once.\n\nIt is faster for a single agent too: when the reply repeats a file it is rewriting, after a restart, and in long conversations. The table has these cases, on a MacBook Pro with an M2 Pro and 16 GB and Qwen3.5-4B, the default model.\n\n| On a MacBook Pro with an M2 Pro and 16 GB, Qwen3.5-4B | Without Locals | With Locals |  | \n|---|---|---|---|\n| 8 requests at once | 87 tokens/s (mlx-lm) | 271 tokens/s | 3.1x | \n| 8 agents asking at once: time to the first word | 42 s (Ollama) | 1.6 s | 26x sooner | \n| One agent rewriting a 100-line file | 43 tokens/s (mlx-lm) | 128 tokens/s | 3.0x | \n| An agent's first request after a restart | 16 s (mlx-lm) | 1.7 s | 9.6x sooner | \n| A follow-up question about a 48,000-token document | out of memory (mlx-lm) | 0.6 s |  | \n| 16 coding agents calling tools (Qwen3.5-2B, from the paper) | 75 s (mlx-lm) | 29 s | 2.5x | \n| 40 programming problems from LiveCodeBench, 8 at a time (from the paper) | 20 min (mlx-lm) | 8 min | 2.5x | \n\nBigger models gain more on the M2 Max: 3.0x for Qwen3.5-9B and 3.1x for a 27B\nmodel with eight requests at once. On the M5 Pro the gain is smaller, because\nmlx-lm is much faster there: MLX has its own kernels for the M5's neural\naccelerators, and these numbers were measured before Locals had its own. M1, M3\nand M4 chips have not been measured yet: results from them are welcome as\n[issues](https://github.com/Generated-Computers/Locals/issues), with the output\nof `locals bench` and `locals tune --report`.\n\nEvery table, and how each was measured, is in\n[Benchmarks](https://github.com/Generated-Computers/Locals/blob/main/docs/benchmarks/README.md). Unless a chart says otherwise, it was\nmeasured on a MacBook Pro with an M2 Pro (19-core GPU) and 16 GB, with\nQwen3.5-4B (`mlx-community/Qwen3.5-4B-MLX-4bit`), MLX 0.32 and mlx-lm 0.32.\n\nModels quantized to 4 bits in the usual way (in groups of 64) gain 2.3x to 3.3x\nwith 8 requests at once; other formats run at mlx-lm's speed\n([below](#where-locals-is-not-faster)).\n\nOllama 0.40.5, Magnitude 0.2.9 and llama.cpp `b11724`, one at a time on the GPU,\neach with its own 4-bit build of the model, measured with SemiAnalysis's\nInferenceX client (temperature 0, replies of natural length). The tables for\n1,000- and 8,000-token prompts and for Gemma 4 are in\n[Benchmarks](https://github.com/Generated-Computers/Locals/blob/main/docs/benchmarks/README.md#3-against-other-servers).\n\n**No pause while a long prompt is read.** When an agent sends an 8,000-token\nprompt while others are getting answers, Locals reads it in the same passes as\ntheir next tokens, so their replies keep streaming: the longest gap in them went\nfrom 12 s to 3 s, with the same throughput, against reading the prompt on its\nown. A 3,000-token request that arrives during a 30,000-token one gets its first\ntoken after 7 s instead of 41.\n\n- **One request alone, with nothing to reuse** : about the same as mlx-lm (64\nagainst 61 tokens per second on the M2 Pro). It is faster alone when the reply\nrepeats the prompt (file rewrites) or the prompt is cached.\n- **Prompt-bound work** : reading a prompt is limited by the GPU's arithmetic.\nWith 8,000-token prompts and 8 requests at once the server writes 57 tokens\nper second; most of the time goes into reading.\n- **Formats outside the tuned kernels** : 4-bit affine quantization in groups of\n64 is covered; mxfp4 (a mixture-of-experts model, Mellum) ran at mlx-lm's\nspeed.\n- **The M5's neural accelerators** : MLX's own kernels already use them, so the\ngain there is 1.5x, not 3x (before Locals' own`nax` kernel, which is\nexperimental).\n\nLocals works with the coding agents and assistants people already use, on a model running on your Mac. All of these but Roo Code were tested end to end with Qwen3.5-4B on a 16 GB Mac (Cline and Kilo Code through their command-line versions), on real tasks such as writing a program, running it and reporting what it printed. One command starts Locals and connects the tool to it:\n\n| Tool | Run | \n|---|---|\n| [Hermes Agent](https://github.com/NousResearch/hermes-agent) | `locals launch hermes` | \n| [Kilo Code](https://github.com/Kilo-Org/kilocode) | `locals launch kilo` | \n| [OpenCode](https://github.com/anomalyco/opencode) | `locals code ~/my-project` | \n| [OpenHands](https://github.com/OpenHands/OpenHands) | `locals launch openhands` | \n| [Cline](https://github.com/cline/cline) | `locals launch cline` | \n| [Aider](https://github.com/Aider-AI/aider) | `locals launch aider` | \n| [Roo Code](https://github.com/RooCodeInc/Roo-Code) | in VS Code, the settings from `locals launch editor` ([how](#use-it-in-apps-and-editors) ) | \n| [OpenClaw](https://github.com/openclaw/openclaw) | `locals launch openclaw` | \n| [Claude Code](https://github.com/anthropics/claude-code) | `locals launch claude` (through Anthropic's Messages API, which Locals also answers) | \n\n`locals launch list` shows every tool it can open. Locals keeps running in the\nbackground after you close a tool: `locals status` shows what it is doing and\n`locals stop` stops it.\n\n<sub>The logos belong to their projects and are shown only to say that Locals\nworks with these tools.</sub>\n\nYou need a Mac with Apple silicon (M1 or newer) and macOS 14 or newer. A Mac with 16 GB of memory gives the best results; with 8 GB, use one of the smaller models.\n\nOpen the Terminal app and run:\n\n```\ncurl -LsSf https://raw.githubusercontent.com/Generated-Computers/Locals/main/install.sh | sh\n```\n\nThen open a new Terminal window (the installer adds Locals to your shell's PATH, which only new windows read) and start a coding assistant on one of your projects:\n\n```\nlocals code ~/my-project\n```\n\nAfter the first start, which downloads the model, Locals starts in seconds.\n\n- **OpenCode** uses Locals as its default model from then on, so plain`opencode` uses it too;`locals disconnect opencode` gives OpenCode its own\nsettings back.\n- **To update** Locals, run the install command again, or`uv tool upgrade locals` .\n- **To remove** it, run`locals stop` ,`locals disconnect opencode` and`uv tool uninstall locals` .\n- **With the uv package manager** , you can install from PyPI with`uv tool install --python 3.12 locals` instead.\n\nLocals installs next to them and changes nothing of theirs: it has its own port (8321) and its own folders. Only the memory is shared, so quit the other one while Locals runs; two models at once rarely fit. Locals remembers the model and port you start it with for later commands.\n\n**Ollama.** Quit Ollama, then let Locals answer on Ollama's port. Apps set up\nfor Ollama (Open WebUI, Raycast, Continue) and the `ollama` command itself keep\nworking unchanged, whatever model name they ask for:\n\n```\nosascript -e 'quit app \"Ollama\"'      # or: brew services stop ollama\nlocals launch --model qwen3.5:4b ollama\n```\n\nOllama's model names work in Locals, and for a model Locals does not know it\nnames the MLX version on Hugging Face to use instead. Ollama's own downloads\nare in its own format (GGUF for most models), so a model is downloaded once\nmore, in MLX's. To\ngo back, run `locals stop` and start Ollama.\n\n**LM Studio.** LM Studio's MLX models run in Locals where they are, with no\nsecond download. Stop LM Studio's server and start Locals on its port, so apps\nthat use it keep working:\n\n```\nlocals start --model ~/.lmstudio/models/mlx-community/Qwen3.5-4B-MLX-4bit --port 1234\n```\n\n**mlx-lm.** The same models and the same download folder, so nothing downloads\nagain. `locals serve` takes the place of `mlx_lm.server`, with the same API:\n\n```\nlocals serve --model mlx-community/Qwen3.5-4B-MLX-4bit --port 8080\n```\n\n**llama.cpp.** Locals runs MLX models, not GGUF files; give it one and it names\nthe MLX version of the same model. Start it on `llama-server`'s port to keep\nyour clients:\n\n```\nlocals start --model qwen --port 8080\n```\n\n**Any other server with the OpenAI API** (vLLM, LocalAI): point your client at\n`http://127.0.0.1:8321/v1`, or start Locals on the port the client already uses\nwith `--port`.\n\n**Work on a project with an assistant.** This opens OpenCode in your project, where\nyou can ask for changes in plain words and watch it read, edit and test your code:\n\n```\nlocals code ~/my-project\n```\n\n**Give it one task and get the result.** The assistant does the task from start\nto finish without opening a window, and prints what it did:\n\n```\nlocals code ~/my-project --run \"add a --verbose flag to cli.py and run the tests\"\n```\n\n**Let a team of agents do a bigger task.** Several agents work on the parts of\nthe task at the same time, each in its own copy of your project, and your tests\ndecide when it is done. Your project is changed only when the tests pass:\n\n```\nlocals run \"write wordcount.py with a function count_words(text) and tests in test_wordcount.py\" \\\n    --dir ~/my-project --check \"python3 -m pytest -q\"\n```\n\n**Ask a question about your code.** An agent reads the project and prints the\nanswer:\n\n```\nlocals run \"Which function parses dates, and which format does it expect?\" --dir ~/my-project\n```\n\n**Chat with the model** in the Terminal, like a private chatbot:\n\n```\nlocals chat\n```\n\n**Turn text into structured data.** Describe the fields you want in a JSON schema,\nfor example in a file called `person.json`:\n\n```\n{\"type\": \"object\",\n \"properties\": {\"name\": {\"type\": \"string\"}, \"age\": {\"type\": \"integer\"}, \"email\": {\"type\": \"string\"},\n                \"status\": {\"enum\": [\"pending\", \"approved\", \"rejected\"]}, \"tags\": {\"type\": \"array\", \"items\": {\"type\": \"string\"}}},\n \"required\": [\"name\", \"age\", \"email\", \"status\", \"tags\"]}\n```\n\nThen the answer always follows it, field by field:\n\n```\nlocals generate --prompt \"Ana Silva, 29, ana@example.com, application approved, tags: vip, early\" \\\n    --schema person.json\n{\"name\":\"Ana Silva\",\"age\":29,\"email\":\"ana@example.com\",\"status\":\"approved\",\"tags\":[\"vip\",\"early\"]}\n```\n\nStart Locals once with the model you want; it keeps running in the background:\n\n```\nlocals start --model qwen\n```\n\n**The OpenCode app.** Run `locals setup --model qwen` once, then choose Locals in\nOpenCode's model list. This works in the desktop app, the editor extension and\nthe Terminal version alike.\n\n**Kilo Code, Cline and Roo Code in VS Code**, and **Continue, Zed and JetBrains.**\nChoose the provider called \"OpenAI Compatible\" and enter these values. Run\n`locals launch editor` to print them for the model that is running.\n\n| Setting | What to enter | \n|---|---|\n| Base URL | `http://127.0.0.1:8321/v1` | \n| API key | any text, for example `local` | \n| Model | the model's name from the table below, for example `locals/qwen3.5-4b` | \n| Context window | the number `locals launch editor` prints, if the extension asks for it | \n\n**Apps that work with Ollama**, such as Open WebUI or Raycast, need no settings:\nrun `locals launch ollama`.\n\nLocals picks a model that fits your Mac's memory, so most people never need to\nchange it. To use another one, start Locals with its name, for example\n`locals start --model lfm`. Locals runs one model at a time. The last two\ncolumns are speeds in tokens per second (tps) with 8 requests at once, on the\nM2 Pro with 16 GB above.\n\n| Model | Start it with | Name in your editor | Size | mlx-lm | Locals | \n|---|---|---|---|---|---|\n| Qwen3.5-4B (the default) | `--model qwen` | `locals/qwen3.5-4b` | 2.8 GB | 87 tps | 271 tps | \n| LFM2.5-2.6B | `--model lfm` | `locals/lfm2.5-2.6b` | 1.5 GB | 144 tps | 404 tps | \n| MiniCPM5-2B | `--model minicpm` | `locals/minicpm5-2b` | 1.3 GB | 160 tps | 478 tps | \n| Gemma 4 E4B | `--model gemma` | `locals/gemma-4-e4b-it` | 4.8 GB | 80 tps | 224 tps | \n\nOn a Mac with less than 12 GB of memory, the default is Qwen3.5-2B instead\n(`--model qwen3.5-2b`; `locals/qwen3.5-2b` in your editor).\n\nEach model also has a thinking mode, in which it reasons before it answers: add\n`-thinking` to the name, for example `locals/lfm2.5-2.6b-thinking`. The smaller\nmodels are good choices for a Mac with 8 GB of memory, and other models made for\nApple silicon on [Hugging Face](https://huggingface.co/mlx-community) work too.\n\nConversations can be long: up to about 178,000 tokens with the default model on a 16 GB Mac, a few hundred pages of text. On the M2 Pro, reading a 170,000-token document takes about 12 minutes the first time, and follow-up questions about it start in 2 seconds. OpenCode is set up for 32,000 tokens, and OpenClaw, Hermes Agent and Kilo Code for 64,000, which keeps each of their steps quick.\n\nLocals understands the same requests as the OpenAI and Ollama services, so\nprograms written for them work with it unchanged. Start Locals with\n`locals start`, then point your program at `http://127.0.0.1:8321/v1`:\n\n``` python\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://127.0.0.1:8321/v1\", api_key=\"local\")\nreply = client.chat.completions.create(\n    model=\"locals\",  # the model the server runs\n    messages=[{\"role\": \"user\", \"content\": \"Write a haiku about the sea.\"}],\n)\nprint(reply.choices[0].message.content)\n```\n\nSend many requests at once, from threads or with an async client, and Locals\nanswers them together; that is where it is fastest. Streaming, answers in a\nfixed JSON format, function calls, the Ollama library and plain `curl` all work\nas well; the [examples](https://github.com/Generated-Computers/Locals/blob/main/examples/README.md) show each of them.\n\nEverything runs on your Mac. Locals only accepts requests from apps on your own\ncomputer, so a website you visit cannot use it. When a team of agents works on a\ntask (`locals run`, `locals solve`), the commands they run can change files only\nin their own copy of your project, and in the temporary and cache folders that\ntools write to. `locals code` runs OpenCode in your project itself, as OpenCode\nalways runs.\n\nRun `locals status`, or open [http://127.0.0.1:8321](http://127.0.0.1:8321) in your browser, to see whether\nLocals is running and what it is doing. On a chip other than the M2 Pro and\nM2 Max, whose tuning ships with Locals, it tunes itself to your chip the first\ntime, in the background while it is idle: 10 to 20 minutes of idle time in all.\nYou can use it during that time, and it gets faster once it finishes.\n\nQuestions and problems are welcome as\n[issues on GitHub](https://github.com/Generated-Computers/Locals/issues).\n\n| Topic | Page | \n|---|---|\n| How Locals works: the server, the batch engine, the prompt cache, the team of agents behind `locals run` , and every setting | [How Locals works](https://github.com/Generated-Computers/Locals/blob/main/docs/README.md) | \n| Every benchmark, and how it was measured | [Benchmarks](https://github.com/Generated-Computers/Locals/blob/main/docs/benchmarks/README.md) | \n| Every table with its date, commit and command; the M2 Max | [Measurements](https://github.com/Generated-Computers/Locals/blob/main/docs/measurements/README.md) ,[M2 Max](https://github.com/Generated-Computers/Locals/blob/main/docs/measurements/m2-max.md) | \n| Why Locals is faster than mlx-lm, part by part | [Why Locals is faster](https://github.com/Generated-Computers/Locals/blob/main/docs/why-faster/README.md) | \n| Streaming, JSON, function calls, the Ollama library and `curl` | [Examples](https://github.com/Generated-Computers/Locals/blob/main/examples/README.md) | \n| The tests, and the end-to-end suite with real agent tools | [Tests](https://github.com/Generated-Computers/Locals/blob/main/tests/README.md) ,[End to end](https://github.com/Generated-Computers/Locals/blob/main/tests/e2e/README.md) | \n| Working on Locals itself | [CONTRIBUTING.md](https://github.com/Generated-Computers/Locals/blob/main/CONTRIBUTING.md) | \n\nTo work on Locals:\n\n```\ngit clone https://github.com/Generated-Computers/Locals\ncd Locals\nuv sync --extra dev\nuv run pytest tests -q -m \"not model\"\n```\n\nIf you use Locals in your research, please cite the paper:\n\n```\n@article{asadulaev2026breaking,\n  title   = {Breaking the Space Barrier and its Application to Language Model Inference},\n  author  = {Asadulaev, Arip},\n  journal = {arXiv preprint arXiv:2610.09139},\n  year    = {2026}\n}\n```\n\nLocals is released under the MIT license. It is made by\n[Generated Computers](https://github.com/Generated-Computers).", "url": "https://wpnews.pro/news/run-many-agents-3x-faster-on-your-mac", "canonical_source": "https://github.com/Generated-Computers/Locals", "published_at": "2026-10-21 18:28:28+00:00", "updated_at": "2026-10-21 18:43:21.738978+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-infrastructure", "large-language-models", "developer-tools"], "entities": ["Generated Computers", "Locals", "OpenCode", "OpenHands", "Cline", "Aider", "Claude Code", "Arip Asadulaev"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/run-many-agents-3x-faster-on-your-mac", "markdown": "https://wpnews.pro/news/run-many-agents-3x-faster-on-your-mac.md", "text": "https://wpnews.pro/news/run-many-agents-3x-faster-on-your-mac.txt", "jsonld": "https://wpnews.pro/news/run-many-agents-3x-faster-on-your-mac.jsonld"}}