{"slug": "nobodywho-is-an-inference-engine-that-lets-you-run-llms-locally-and-efficiently", "title": "NobodyWho is an inference engine that lets you run LLMs locally and efficiently", "summary": "NobodyWho released an open-source inference engine that runs large language models locally and offline, built on llama.cpp and distributed as version 2.0.0 across Kotlin, Swift, Python, Flutter, React Native/Expo and Godot. The engine supports any GGUF-format model including Gemma, Qwen and Mistral, adds multimodal image and audio input, Whisper speech-to-text, Kokoro, Pocket TTS and Supertonic text-to-speech, and GPU acceleration via Vulkan or Metal. NobodyWho said the tool requires no API keys or hidden fees and generates structured grammars automatically from function signatures for type-safe tool calling.", "body_md": "**On-device AI for any device.**\n\n  NobodyWho is an inference engine that lets you run LLMs locally and efficiently.\n\n- **Run locally, offline** — no API keys needed or hidden fees\n- **Run any chat LLM** — Gemma, Qwen, Mistral and more\n- **Fast, type-safe tool calling** — automatically generates structured grammars from your function signatures, no schema writing needed\n- **Multimodal input** — provide image and audio information to your LLM\n- **Text-to-speech** — synthesize local WAV audio with Kokoro, Pocket TTS and Supertonic backends\n- **Speech-to-text** — Transcribe audio into text with Whisper\n- **Model downloading** — load models directly from[Hugging Face](https://huggingface.co/models?library=gguf&sort=trending) or any URL\n\n- Conversation-aware preemptive context shifting — retain full conversation memory without any message length limits\n- GPU-accelerated inference via Vulkan or Metal — runs fast on any OS\n- Compatible with thousands of pre-trained LLMs — use any LLM in the GGUF format\n- Powered by the wonderful [llama.cpp](https://github.com/ggml-org/llama.cpp)\n\nWe support Kotlin, Swift, Python, Flutter, React Native, Expo and Godot.\n\n| Platform | Installation | Documentation | \n|---|---|---|\n| **Kotlin** | [Maven Central](#kotlin) | [docs.nobodywho.ooo/kotlin](https://docs.nobodywho.ooo/kotlin/) | \n| **Swift** | [SPM](#swift) | [docs.nobodywho.ooo/swift](https://docs.nobodywho.ooo/swift/) | \n| **React Native / Expo** | [npm](#react-native--expo) | [docs.nobodywho.ooo/react-native](https://docs.nobodywho.ooo/react-native/) | \n| **Flutter** | [pub.dev](#flutter) | [docs.nobodywho.ooo/flutter](https://docs.nobodywho.ooo/flutter/) | \n| **Python** | [pypi](#python) | [docs.nobodywho.ooo/python](https://docs.nobodywho.ooo/python/) | \n| **Godot** | [AssetLib](#godot) | [docs.nobodywho.ooo/godot](https://docs.nobodywho.ooo/godot/) | \n\n``` python\nimport ai.nobodywho.Chat\n\nval chat = Chat.fromPath(\n    modelPath = \"hf://NobodyWho/Qwen_Qwen3-0.6B-GGUF/Qwen_Qwen3-0.6B-Q4_K_M.gguf\"\n)\n\nval response = chat.ask(\"What is the capital of Denmark?\").completed()\nprintln(response) // The capital of Denmark is Copenhagen.\n```\n\nAdd to your `build.gradle.kts`:\n\n```\n// Android\nimplementation(\"ai.nobodywho:nobodywho-android:2.0.0\")\n\n// Desktop JVM (Linux, macOS, Windows)\nimplementation(\"ai.nobodywho:nobodywho:2.0.0\")\npython\nimport NobodyWho\n\nlet chat = try await Chat.fromPath(\n    modelPath: \"hf://NobodyWho/Qwen_Qwen3-0.6B-GGUF/Qwen_Qwen3-0.6B-Q4_K_M.gguf\"\n)\n\nlet response = try await chat.ask(\"What is the capital of Denmark?\").completed()\nprint(response) // The capital of Denmark is Copenhagen.\n```\n\nAdd via Swift Package Manager:\n\n```\nhttps://github.com/nobodywho-ooo/nobodywho-swift.git\njs\nimport { Chat } from \"react-native-nobodywho\";\n\nconst chat = await Chat.fromPath({\n  modelPath: \"hf://NobodyWho/Qwen_Qwen3-0.6B-GGUF/Qwen_Qwen3-0.6B-Q4_K_M.gguf\",\n});\n\nconst msg = await chat.ask(\"What is the capital of Denmark?\").completed();\nconsole.log(msg); // The capital of Denmark is Copenhagen.\n```\n\nInstall via npm:\n\n```\n# React Native\nnpm install react-native-nobodywho\n\n# Expo\nnpx expo install react-native-nobodywho\n```\n\n[RN / Expo documentation](https://docs.nobodywho.ooo/react-native/) - [npm](https://www.npmjs.com/package/react-native-nobodywho) - [RN starter example app](https://github.com/nobodywho-ooo/react-native-starter-example) - [Expo starter example app](https://github.com/nobodywho-ooo/expo-starter-example)\n\n```\nimport 'package:nobodywho/nobodywho.dart' as nobodywho;\n\nvoid main() async {\n  await nobodywho.NobodyWho.init();\n\n  final chat = await nobodywho.Chat.fromPath(\n    modelPath: 'huggingface:NobodyWho/Qwen_Qwen3-0.6B-GGUF/Qwen_Qwen3-0.6B-Q4_K_M.gguf',\n  );\n\n  final msg = await chat.ask('What is the capital of Denmark?').completed();\n  print(msg); // The capital of Denmark is Copenhagen.\n}\n```\n\nInstall via pub.dev:\n\n```\nflutter pub add nobodywho\n```\n\n[Flutter documentation](https://docs.nobodywho.ooo/flutter/) - [pub.dev](https://pub.dev/packages/nobodywho) - [starter example app](https://github.com/nobodywho-ooo/flutter-starter-example)\n\n``` python\nfrom nobodywho import Chat\n\nchat = Chat(\"huggingface:NobodyWho/Qwen_Qwen3-0.6B-GGUF/Qwen_Qwen3-0.6B-Q4_K_M.gguf\")\n\nresponse = chat.ask(\"What is the capital of Denmark?\").completed()\nprint(response) # The capital of Denmark is Copenhagen.\n```\n\nInstall via pip:\n\n```\npip install nobodywho\n```\n\nYou can install it from inside the Godot editor: In Godot 4.5+, go to AssetLib and search for \"NobodyWho\".\n\n...or you can grab a specific version from our [github releases page.](https://github.com/nobodywho-ooo/nobodywho/releases) You can install these zip files by going to the \"AssetLib\" tab in Godot and selecting \"Import\".\n\nMake sure that the ignore asset root option is set in the import dialogue.\n\nFor further instructions on how to setup NobodyWho in Godot please refer to our [docs](https://docs.nobodywho.ooo/godot/install/).\n\n[The documentation](https://docs.nobodywho.ooo) has everything you might want to know: [https://docs.nobodywho.ooo/](https://docs.nobodywho.ooo/)\n\n- ⭐ Star the repo and spread the word about NobodyWho!\n- Join our [Discord](https://discord.gg/qhaMc2qCYB) or[Matrix](https://matrix.to/#/#nobodywho:matrix.org) communities\n- Found a bug? Open an issue!\n- Submit your own PR - contributions welcome\n- Help improve docs, write tutorials and demos\n\nDesktop (Linux, macOS, Windows) is supported across all bindings. Android is supported on Kotlin, Godot, Flutter and React Native. iOS is supported on Swift, Flutter and React Native. visionOS and watchOS are supported via the Swift package.\n\nWeb exports will be a bit trickier to get right. See issue [#111](https://github.com/nobodywho-ooo/nobodywho/issues/111).\n\nThere has been some confusion about the licensing terms of NobodyWho. To clarify:\n\nLinking two programs or linking an existing software with your own work does not – at least under European law – produce a derivative or extend the coverage of the linked software licence to your own work. [\\[1\\]](https://interoperable-europe.ec.europa.eu/collection/eupl/licence-compatibility-permissivity-reciprocity-and-interoperability)\n\nYou are allowed to use this plugin in proprietary and commercial projects, free of charge.\n\nIf you distribute modified versions of the code *in this repo*, you must open source those changes.\n\nFeel free to make proprietary projects using NobodyWho, but don't make a proprietary fork of NobodyWho.", "url": "https://wpnews.pro/news/nobodywho-is-an-inference-engine-that-lets-you-run-llms-locally-and-efficiently", "canonical_source": "https://github.com/nobodywho-ooo/nobodywho", "published_at": "2026-09-23 07:33:08+00:00", "updated_at": "2026-09-23 07:54:15.304981+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools", "ai-products", "developer-tools"], "entities": ["NobodyWho", "llama.cpp", "Qwen", "Gemma", "Mistral", "Whisper", "Kokoro", "Hugging Face"], "alternates": {"html": "https://wpnews.pro/news/nobodywho-is-an-inference-engine-that-lets-you-run-llms-locally-and-efficiently", "markdown": "https://wpnews.pro/news/nobodywho-is-an-inference-engine-that-lets-you-run-llms-locally-and-efficiently.md", "text": "https://wpnews.pro/news/nobodywho-is-an-inference-engine-that-lets-you-run-llms-locally-and-efficiently.txt", "jsonld": "https://wpnews.pro/news/nobodywho-is-an-inference-engine-that-lets-you-run-llms-locally-and-efficiently.jsonld"}}