{"slug": "how-sentience-ships-60-ai-tools-in-one-local-desktop-app-without-locking-you-one", "title": "How Sentience ships 60+ AI tools in one local desktop app — without locking you into one provider", "summary": "A developer built Sentience, a PySide6 desktop AI assistant that runs locally and integrates 60+ tool functions across multiple AI providers including Groq, OpenAI, Anthropic, and local Ollama instances. The key technical challenge was creating a unified tool schema adapter that works identically across providers without provider-specific dispatchers, achieved with a 60-line provider layer.", "body_md": "I wanted Cursor's UX and Zo Computer's tool breadth in a desktop app I could close the laptop lid on. So I built **Sentience** — a PySide6 desktop AI assistant that runs entirely on your machine, brings its own browser, its own email client, its own voice controller, and exposes 60+ tool functions to the model.\n\nThe hard part was never the tools. The hard part was making 60+ tool schemas work identically across **Groq, OpenAI, Anthropic, and a local Ollama instance** — without writing a provider-specific tool dispatcher and without forcing the user to think about which model they happen to be on today.\n\nThis is the part of the codebase I'm actually proud of, and it's the part nobody ships as a tutorial.\n\nOpenAI's `/v1/chat/completions`\n\nformat is now a de facto standard. Groq implements it. Ollama implements it. Localai implements it. So three out of four providers I wanted to support \"just work\" with one HTTP call — if you're willing to accept the OpenAI tool schema as ground truth.\n\nAnthropic doesn't. The Messages API uses:\n\n`system`\n\nfield instead of a system message in the array`x-api-key`\n\ninstead of `Authorization: Bearer`\n\n`anthropic-version: 2023-06-01`\n\nas a required header`tool_use`\n\n/ `tool_result`\n\ncontent block format on the response sideSo the question is: do you write two tool dispatchers and two execution paths, or do you write a thin adapter that lets you keep one unified tool list and one dispatcher?\n\nI chose the adapter. The whole provider layer is 60 lines.\n\n```\nPROVIDERS = {\n    \"groq\": {\n        \"name\": \"Groq\",\n        \"base_url\": \"https://api.groq.com/openai/v1\",\n        \"models\": [\"llama-3.3-70b-versatile\", \"llama-3.1-70b-versatile\",\n                   \"llama-3.2-90b-vision-preview\", \"mixtral-8x7b-32768\"],\n        \"free_tier\": True,\n    },\n    \"openai\": {\n        \"name\": \"OpenAI\",\n        \"base_url\": \"https://api.openai.com/v1\",\n        \"models\": [\"gpt-4o\", \"gpt-4o-mini\", \"gpt-4-turbo\", \"gpt-3.5-turbo\"],\n        \"free_tier\": False,\n    },\n    \"anthropic\": {\n        \"name\": \"Anthropic\",\n        \"base_url\": \"https://api.anthropic.com/v1\",\n        \"models\": [\"claude-3-5-sonnet-20241022\", \"claude-3-opus-20240229\",\n                   \"claude-3-haiku-20240307\"],\n        \"free_tier\": False,\n    },\n    \"ollama\": {\n        \"name\": \"Ollama (Local)\",\n        \"base_url\": os.getenv(\"OLLAMA_HOST\", \"http://localhost:11434/v1\"),\n        \"models\": [\"llama3.2\", \"llama3.1\", \"codellama\", \"mistral\", \"qwen2.5\"],\n        \"free_tier\": True,\n    },\n}\n```\n\nFree tier is a first-class field, not a comment. The settings dialog lights up the \"free\" badge for Groq and Ollama, and the README leads with those two for new users.\n\n`chat()`\n\nThe whole class has one entry point and two private methods. The entry point picks a path based on `self.provider`\n\n. That's it.\n\n``` python\nclass AIClient:\n    def __init__(self, provider: str, model: str, api_key: str = \"\"):\n        self.provider = provider\n        self.model = model\n        self.api_key = api_key\n        self.config = PROVIDERS.get(provider, PROVIDERS[\"groq\"])\n\n    def chat(self, messages, tools=None):\n        if self.provider == \"anthropic\":\n            return self._chat_anthropic(messages, tools)\n        return self._chat_openai_compatible(messages, tools)\n\n    def _chat_openai_compatible(self, messages, tools=None):\n        headers = {\"Content-Type\": \"application/json\"}\n        if self.api_key:\n            headers[\"Authorization\"] = f\"Bearer {self.api_key}\"\n        payload = {\n            \"model\": self.model,\n            \"messages\": messages,\n            \"max_tokens\": 4096,\n            \"temperature\": 0.7,\n        }\n        if tools:\n            payload[\"tools\"] = tools\n            payload[\"tool_choice\"] = \"auto\"\n        try:\n            resp = requests.post(\n                f\"{self.config['base_url']}/chat/completions\",\n                headers=headers, json=payload, timeout=60,\n            )\n            resp.raise_for_status()\n            return resp.json()\n        except Exception as e:\n            return {\"error\": str(e)}\n\n    def _chat_anthropic(self, messages, tools=None):\n        headers = {\n            \"Content-Type\": \"application/json\",\n            \"x-api-key\": self.api_key,\n            \"anthropic-version\": \"2023-06-01\",\n        }\n        # System message is a separate field in Messages API\n        system_msg = None\n        anthropic_messages = []\n        for msg in messages:\n            if msg[\"role\"] == \"system\":\n                system_msg = msg[\"content\"]\n            else:\n                anthropic_messages.append(msg)\n        payload = {\n            \"model\": self.model,\n            \"messages\": anthropic_messages,\n            \"max_tokens\": 4096,\n        }\n        if system_msg:\n            payload[\"system\"] = system_msg\n        if tools:\n            payload[\"tools\"] = tools\n        try:\n            resp = requests.post(...)\n            ...\n```\n\nThe Anthropic branch has *exactly* three differences from the OpenAI branch: header names, system-message location, and the path. Everything else — the `tools`\n\nlist, the `messages`\n\narray, the `max_tokens`\n\nfield — is identical. So the same 60 tool schemas I registered for Groq work on Claude without rewriting a single function definition.\n\nThis means I can hand the user a dropdown that says \"switch to Claude Sonnet\" and the *next message* the user types goes to Anthropic's API with the exact same tool surface. The model can call `read_file`\n\n, `list_directory`\n\n, `browser_navigate`\n\n, `send_email`\n\n, and `oauth_github_login`\n\non any of the four providers. The dispatcher doesn't care.\n\nThe tools live in their own modules and are aggregated with a single `*`\n\nspread:\n\n``` python\nfrom browser.automation import BROWSER_TOOLS, PLAYWRIGHT_AVAILABLE\nfrom email_agent.client import EMAIL_TOOLS, init_email, execute_email_tool\nfrom oauth_manager.manager import OAUTH_TOOLS, get_oauth_manager, execute_oauth_tool\nfrom voice.controller import VOICE_TOOLS, get_voice_controller, execute_voice_tool\nfrom skills.registry import SKILL_TOOLS, get_skill_registry, execute_skill_tool\nfrom hosting.server import HOSTING_TOOLS, get_hosting_server\n\nTOOLS = [\n    {\"type\": \"function\", \"function\": {\"name\": \"read_file\", ...}},\n    {\"type\": \"function\", \"function\": {\"name\": \"write_file\", ...}},\n    {\"type\": \"function\", \"function\": {\"name\": \"list_directory\", ...}},\n    {\"type\": \"function\", \"function\": {\"name\": \"run_command\", ...}},\n    {\"type\": \"function\", \"function\": {\"name\": \"search_files\", ...}},\n    *BROWSER_TOOLS,    # browser_navigate, browser_click, browser_screenshot, ...\n    *EMAIL_TOOLS,      # email_read_inbox, email_send, email_search\n    *OAUTH_TOOLS,      # oauth_google_login, oauth_github_login, oauth_notion_login\n    *VOICE_TOOLS,      # voice_listen, voice_speak, voice_set_wake_word\n    *SKILL_TOOLS,      # skill_list, skill_load, skill_run\n    *HOSTING_TOOLS,    # hosting_start, hosting_stop, hosting_status, hosting_logs\n]\n```\n\nEach submodule exports both the **schema** (`BROWSER_TOOLS`\n\n, `EMAIL_TOOLS`\n\n, etc.) and the **executor** (`execute_browser_tool`\n\n, `execute_email_tool`\n\n). The schemas are OpenAI function-calling dicts. The executors are the actual Python functions the dispatcher calls when the model invokes the tool.\n\n``` php\ndef execute_tool(name: str, args: dict, workspace: str) -> dict:\n    try:\n        if name == \"read_file\":\n            path = Path(args.get(\"path\", \"\"))\n            if not path.is_absolute():\n                path = Path(workspace) / path\n            if path.exists():\n                return {\"success\": True, \"content\": path.read_text()[:10000]}\n            return {\"success\": False, \"error\": \"File not found\"}\n\n        elif name == \"run_command\":\n            result = subprocess.run(\n                args.get(\"command\", \"\"), shell=True,\n                capture_output=True, text=True, timeout=30,\n                cwd=args.get(\"cwd\", workspace),\n            )\n            return {\"success\": True, \"stdout\": result.stdout[:5000],\n                    \"stderr\": result.stderr[:5000], \"exit_code\": result.returncode}\n\n        # ... 60+ branches, each returning {\"success\": bool, ...}\n        elif name.startswith(\"browser_\"):\n            return execute_browser_tool(name, args)\n        elif name.startswith(\"email_\"):\n            return execute_email_tool(name, args)\n        elif name.startswith(\"oauth_\"):\n            return execute_oauth_tool(name, args)\n        elif name.startswith(\"voice_\"):\n            return execute_voice_tool(name, args)\n        elif name.startswith(\"skill_\"):\n            return execute_skill_tool(name, args)\n        elif name.startswith(\"hosting_\"):\n            return execute_hosting_tool(name, args)\n        else:\n            return {\"success\": False, \"error\": f\"Unknown tool: {name}\"}\n    except Exception as e:\n        return {\"success\": False, \"error\": str(e)}\n```\n\nTwo patterns I want to call out:\n\n**Every executor returns {\"success\": bool, ...}.** The model gets a uniform response shape, no matter which tool blew up. The model can then decide to retry, escalate, or just tell the user \"I couldn't read that file.\" This is what makes the system actually usable when one of the 60 tools fails mid-conversation.\n\n**All file paths are resolved against the workspace.** The model never gets to specify an absolute path that the user didn't authorize. `Path(workspace) / path`\n\nis a tiny line, but it's the line that means \"I can run this app on a stranger's laptop and not worry about `~`\n\n-escape exploits.\"\n\nThe killer feature, in practice, is the **settings dropdown**. The user can:\n\n`llama-3.3-70b-versatile`\n\n)The tool dispatcher doesn't know or care which provider is in `self.provider`\n\n. The tool list doesn't change. The chat history doesn't get a special \"this is the Anthropic thread\" branch. The 800-line `main.py`\n\nhas one `AIClient`\n\nand one `execute_tool`\n\nand the rest is PySide6 widgets and message-routing glue.\n\nTwo things:\n\n**The Anthropic tool result format is still different on the response side.** When Claude calls a tool, the response uses `tool_use`\n\nblocks and you have to send back `tool_result`\n\nblocks in the next turn. I handle this in the message-routing layer, not the dispatcher. If I were starting over I'd move the tool result translation *into* the `_chat_anthropic`\n\nmethod, so the dispatcher could pretend all four providers return identical shapes.\n\n**Streaming is half-done.** Groq and OpenAI stream identically; Anthropic streams differently (event types, not data-only SSE). The current build buffers the full response and shows it once. That's fine for a 4k-token answer; for a 64k reasoning trace it's not. The fix is the same shape as the chat dispatcher — one streaming method per provider family, one normalized chunk iterator for the UI.\n\n`imaplib`\n\n+ `smtplib`\n\nfrom stdlib\n\n```\ngit clone https://github.com/AmSach/sentience\ncd sentience\npip install -r requirements.txt\nplaywright install chromium\nexport GROQ_API_KEY=your_key_here   # or OPENAI / ANTHROPIC / OLLAMA_HOST\npython src/main.py\n```\n\nFor 100% offline use:\n\n```\nollama pull llama3.2\nOLLAMA_HOST=http://localhost:11434 python src/main.py\n```\n\nMIT licensed. PRs welcome — especially on the streaming layer, a new provider, or another tool module (calendar? GitHub? Linear?).", "url": "https://wpnews.pro/news/how-sentience-ships-60-ai-tools-in-one-local-desktop-app-without-locking-you-one", "canonical_source": "https://dev.to/aman_sachan_126d19c4a2773/how-sentience-ships-60-ai-tools-in-one-local-desktop-app-without-locking-you-into-one-provider-26e6", "published_at": "2026-06-17 01:41:21+00:00", "updated_at": "2026-06-17 02:21:38.526590+00:00", "lang": "en", "topics": ["artificial-intelligence", "developer-tools", "large-language-models", "ai-tools"], "entities": ["Sentience", "Groq", "OpenAI", "Anthropic", "Ollama", "PySide6"], "alternates": {"html": "https://wpnews.pro/news/how-sentience-ships-60-ai-tools-in-one-local-desktop-app-without-locking-you-one", "markdown": "https://wpnews.pro/news/how-sentience-ships-60-ai-tools-in-one-local-desktop-app-without-locking-you-one.md", "text": "https://wpnews.pro/news/how-sentience-ships-60-ai-tools-in-one-local-desktop-app-without-locking-you-one.txt", "jsonld": "https://wpnews.pro/news/how-sentience-ships-60-ai-tools-in-one-local-desktop-app-without-locking-you-one.jsonld"}}