{"slug": "openworker-andrew-ng-s-local-first-ai-coworker-explained-for-developers", "title": "OpenWorker: Andrew Ng's Local-First AI Coworker, Explained for Developers", "summary": "Andrew Ng's OpenWorker, a local-first AI coworker agent, launched in late July 2026 under an MIT license. The desktop app runs on the user's machine, takes their own API key, and delivers finished work artifacts rather than chat transcripts. Built on Ng's aisuite library, it classifies tool calls into four risk classes and enforces granular permission modes, with a shipped operator persona that treats external content as untrusted data.", "body_md": "[OpenWorker](https://github.com/andrewyng/openworker) shipped in late July 2026. It is MIT-licensed, runs on your own machine, and takes your API key instead of selling you inference.\n\nThe pitch is narrow and worth repeating exactly: it is an agent that hands you **finished work**, not a chat transcript. A drafted document on disk. A Slack reply with the real numbers in it. A calendar that has actually been rearranged.\n\nThere are a lot of desktop agents right now. This post is about what makes this one structurally different, what state it is actually in, and how to get it running.\n\nOpenWorker is a desktop app: a Tauri shell around a React UI, sitting on top of a local Python agent server. You give it an outcome (\"prepare a customer brief from these three files and the Jira tickets\"). It decomposes that into steps, reaches into your files, terminal, and connected SaaS apps, and produces an artifact. Before anything consequential happens - sending a message, running a shell command, writing to your calendar - it stops and asks.\n\nThe engine is built on [aisuite](https://github.com/andrewyng/aisuite), Ng's provider-agnostic LLM library. That matters more than it sounds like: OpenWorker is explicitly positioned as a reference implementation of what you can build on aisuite, so the codebase doubles as a worked example if you are building your own harness.\n\nYou paste a key, or you point it at Ollama and use none at all. The curated list covers OpenAI, Anthropic, Google, plus OpenAI-compatible vendors like DeepSeek, GLM, Kimi, Qwen, MiniMax, Mistral, and Grok, plus open-weight models through Together and Fireworks. Roughly thirty models are marked as verified for tool-calling work; you can point it at any other model string and accept the risk yourself.\n\nThe practical consequence: your cost is your provider bill, and swapping models is a dropdown, not a migration.\n\nThis is the part I would actually read the source for. Most agent projects treat approvals as UI polish. OpenWorker classifies every tool call into one of four risk classes:\n\n| Risk class | Meaning |\n|---|---|\n`read` |\nNo side effects |\n`write_local` |\nMutates the workspace, path-scoped |\n`exec` |\nRuns commands |\n`external` |\nSide effects that leave your machine |\n\nFive permission modes then decide what happens to each class. `discuss`\n\nand `plan`\n\nare read-only. `interactive`\n\nis the default and asks before writes, commands, and external actions. `auto`\n\nallows everything but stays path-scoped. `custom`\n\nauto-approves a list of tools you name.\n\nTwo design choices stand out. First, unattended runs do not raise the autonomy ceiling - they change *where the human is reached*. Prompts that would have appeared inline get routed to an inbox and the session suspends until you answer. The agent does not quietly gain permissions because you walked away. Second, standing \"always allow\" rules are restricted to the `external`\n\nclass only. Shell commands ask every single time, on purpose.\n\nThe shipped operator persona instructs the model to treat content coming from tools, logs, web pages, files, and incoming messages as untrusted *data*, not as instructions. That is written into the persona the app ships with, rather than being left as an exercise for the user. For an agent that reads your Slack and your inbox, this is not optional, and it is good to see it stated rather than assumed.\n\nConversations, connector tokens, and model keys live in a local secret store. Model calls go straight from your machine to whichever provider you configured. The only cloud component is a small optional broker that handles OAuth handshakes for one-click connectors, and tokens are handed to your machine rather than stored server-side. You can skip sign-in entirely and wire up connectors with manually created credentials.\n\n`@OpenWorker`\n\nin a channel, a session opens on your desktop, the work runs with your local tools, and the answer comes back as a thread reply.Leave it in `interactive`\n\nmode for the first few sessions. Watching what it asks permission for is the fastest way to build an accurate mental model of what it is doing.\n\nPrerequisites: Python 3.10+, Node 20+, and the Rust toolchain via [rustup](https://rustup.rs/) if you want the desktop shell rather than the browser UI.\n\n```\ngit clone https://github.com/andrewyng/openworker\ncd openworker\n\n# 1. One-time bootstrap - creates the Python venv at .venv\n#    On Windows, run this from Git Bash or WSL\nbash packaging/setup_dev_env.sh\n\n# 2. Start the local agent server\n.venv/bin/openworker-server --cwd ~/some/project --port 8765\n# Windows: .venv\\Scripts\\openworker-server.exe\n```\n\nThen in a second terminal:\n\n```\ncd surfaces/gui\nnpm install\nnpm run dev        # browser UI on the Vite dev port\n```\n\nFor the full desktop app instead of the browser UI, replace that last step with `npm run tauri dev`\n\nfrom `surfaces/gui/`\n\n. The Tauri shell launches the window and supervises the Python server itself, so you can skip step 2 in that case.\n\nThe server is FastAPI on uvicorn, bound to `127.0.0.1:8765`\n\nby default.\n\nRunning the tests:\n\n```\n.venv/bin/pytest              # backend\ncd surfaces/gui\nnpm test                      # GUI unit tests\nnpm run e2e                   # hermetic end-to-end\n```\n\nNote the `--cwd`\n\nflag on the server. That is the workspace root the agent's path-scoped file access is anchored to. Point it at a scratch directory for your first runs, not at your home folder.\n\n| Directory | Contents |\n|---|---|\n`coworker/` |\nPython backend - agent engine, model providers, connectors, MCP client, memory, automations |\n`surfaces/gui/` |\nReact UI plus the Tauri shell |\n`stt/` |\nRust speech-to-text sidecar for voice input |\n`packaging/` |\nmacOS DMG and Windows installer builds, auto-update manifest |\n`docs/` |\nDesign specs and decision logs |\n`tests/` |\nBackend test suite |\n\nIf you are here to learn rather than to use, `docs/`\n\nand the permission engine in `coworker/`\n\nare the highest-value reads. The design logs explain choices rather than just documenting APIs.\n\n**Yes, if any of these describe you:**\n\n**Hold off if:**\n\nThe interesting claim here is not \"agent that does tasks.\" Everyone is shipping that. The interesting claim is that the boring infrastructure - who is allowed to do what, where secrets live, what counts as an instruction versus data - has been designed as a typed system rather than bolted on. That is the part worth reading even if you never use the app.\n\nIt is early. Expect rough edges, expect the Windows signing situation to resolve, expect the connector list to shift. But if you have been waiting for a credible open, local, model-agnostic option in this category, this is the first one that looks like it was built by people who thought about the failure modes first.\n\nRepo: [github.com/andrewyng/openworker](https://github.com/andrewyng/openworker)\n\nSite: [openworker.com](https://openworker.com)", "url": "https://wpnews.pro/news/openworker-andrew-ng-s-local-first-ai-coworker-explained-for-developers", "canonical_source": "https://dev.to/arshtechpro/openworker-andrew-ngs-local-first-ai-coworker-explained-for-developers-3hc9", "published_at": "2026-07-29 12:10:20+00:00", "updated_at": "2026-07-29 12:37:36.324806+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-tools", "ai-products", "developer-tools"], "entities": ["Andrew Ng", "OpenWorker", "aisuite", "Ollama", "OpenAI", "Anthropic", "Google", "DeepSeek"], "alternates": {"html": "https://wpnews.pro/news/openworker-andrew-ng-s-local-first-ai-coworker-explained-for-developers", "markdown": "https://wpnews.pro/news/openworker-andrew-ng-s-local-first-ai-coworker-explained-for-developers.md", "text": "https://wpnews.pro/news/openworker-andrew-ng-s-local-first-ai-coworker-explained-for-developers.txt", "jsonld": "https://wpnews.pro/news/openworker-andrew-ng-s-local-first-ai-coworker-explained-for-developers.jsonld"}}