{"slug": "a-new-implementation-for-llms-through-nvidia-gpus-natively-on-apple-silicon", "title": "A new implementation for LLMs through Nvidia GPUs natively on Apple Silicon", "summary": "Hawking Systems Inc. released Hawking Runtime, a local inference and GPU computing project for Apple Silicon Macs that includes a DriverKit driver, host runtime and CUDA kernels for running NVIDIA GPUs over Thunderbolt. The company demonstrated driver initialization, GPU kernel execution, numerical verification and recovery without rebooting on a reference setup pairing an NVIDIA RTX 4000 SFF Ada with an M3 Ultra Mac Studio, with compute through the public C ABI qualified at PCIe Gen1; Gen2 ×4 reached roughly twice the measured Gen1 transfer rate, while Gen3 and Gen4 remain in development. The project also ships a llama.cpp-based extension with an OpenAI-compatible API served by default at http://127.0.0.1:8277/v1, installable via `pip install hawking-runtime` on Python 3.12, with the eGPU implementation published as developer source for inspection and reproduction.", "body_md": "Hawking Runtime is a local inference and GPU computing project for Apple Silicon Macs.\n\nThe ethos is simple: make better use of the hardware you already have.\n\nThe project currently has two parts:\n\n- **Extension:** Local model inference through llama.cpp, with an OpenAI-compatible API, and model management.\n- **eGPU:** A native macOS DriverKit driver, host runtime, CUDA kernels, hardware diagnostics, and developer tools for running NVIDIA GPUs connected over Thunderbolt.\n\nBoth are part of Hawking Runtime, but they can be used independently.\n\nRequires an Apple Silicon Mac and Python 3.12.\n\n```\npython3.12 -m pip install hawking-runtime\nhawking init\nhawking add Qwen/Qwen3-0.6B-GGUF:Q8_0\nhawking serve MODEL\n```\n\nUse the library ID printed by `hawking add` in place of `MODEL`, or a local `.gguf` path.\n\n`hawking init` detects your Mac's available resources, sets up the pinned llama.cpp runtime, and configures supported coding agents it finds.\n\n`hawking serve` starts a local OpenAI-compatible server supporting streaming, tool calls, and the Responses API.\n\nDefault endpoint: `http://127.0.0.1:8277/v1`\n\nYou can use it with OpenAI-compatible clients, coding agents, and interfaces such as Open WebUI.\n\nSee [`extension/README.md`](https://github.com/hawking-systems/runtime/blob/main/extension/README.md) for the complete setup and configuration options.\n\nWe have demonstrated real NVIDIA GPU computation on Apple Silicon macOS.\n\nOur reference setup uses an NVIDIA RTX 4000 SFF Ada connected over Thunderbolt to an M3 Ultra Mac Studio. We've demonstrated driver initialization, GPU kernel execution, numerical verification, and recovery without rebooting.\n\nWe have also experimented with PCIe link negotiation. Gen2 ×4 achieved approximately twice the measured Gen1 transfer rate. Gen3 and Gen4 are in development and not part of this release.\n\nIn this release, compute through the public C ABI is qualified on the reference setup at PCIe Gen1.\n\nThe eGPU implementation is available as developer source, including the DriverKit driver, native execution code, CUDA kernel catalogue, diagnostics, and build tooling.\n\n```\ncd egpu/operations/hardware/egpu_runtime\nmake test-fast\n```\n\nThe purpose of publishing the source is to let developers inspect the implementation, reproduce the work, adapt it to their hardware, and build on it.\n\nSee [`egpu/operations/hardware/egpu_runtime/README.md`](https://github.com/hawking-systems/runtime/blob/main/egpu/operations/hardware/egpu_runtime/README.md), `SIGNING.md`, and `docs/LIFECYCLE.md` for the development workflow.\n\nWe encourage using a coding agent to work through the eGPU setup. Hardware configurations differ, and an agent can help inspect your system, build the relevant components, identify bottlenecks, and develop improvements for your particular setup.\n\nYou can start with a prompt like this:\n\nClone the Hawking Runtime repository and read the eGPU documentation. I want to connect and optimize my NVIDIA eGPU with my Apple Silicon Mac. Inspect my GPU, enclosure, PCIe connection, and macOS environment. Help me build and test the driver and runtime, identify compatibility issues, and find opportunities to improve transfer speeds and compute performance. Use reproducible benchmarks and reversible changes. Ask before installing or activating drivers, changing hardware policies, or performing anything that could affect system stability.\n\nSee `LICENSE` and `NOTICE` for the applicable terms and third-party attributions.\n\nHawking Systems Inc. · 2026", "url": "https://wpnews.pro/news/a-new-implementation-for-llms-through-nvidia-gpus-natively-on-apple-silicon", "canonical_source": "https://github.com/hawking-systems/runtime", "published_at": "2026-11-16 19:57:07+00:00", "updated_at": "2026-11-16 20:23:51.876042+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools", "developer-tools", "large-language-models", "ai-chips"], "entities": ["Hawking Systems Inc.", "Hawking Runtime", "Apple Silicon", "NVIDIA", "RTX 4000 SFF Ada", "M3 Ultra Mac Studio", "llama.cpp", "Thunderbolt"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/a-new-implementation-for-llms-through-nvidia-gpus-natively-on-apple-silicon", "markdown": "https://wpnews.pro/news/a-new-implementation-for-llms-through-nvidia-gpus-natively-on-apple-silicon.md", "text": "https://wpnews.pro/news/a-new-implementation-for-llms-through-nvidia-gpus-natively-on-apple-silicon.txt", "jsonld": "https://wpnews.pro/news/a-new-implementation-for-llms-through-nvidia-gpus-natively-on-apple-silicon.jsonld"}}