cd /news/ai-infrastructure/a-new-implementation-for-llms-throug… · home › topics › ai-infrastructure › article
[ARTICLE · art-170890] src=github.com ↗ pub= topic=ai-infrastructure verified=true sentiment=↑ positive

A new implementation for LLMs through Nvidia GPUs natively on Apple Silicon

Hawking Systems Inc. released Hawking Runtime, a local inference and GPU computing project for Apple Silicon Macs that includes a DriverKit driver, host runtime and CUDA kernels for running NVIDIA GPUs over Thunderbolt. The company demonstrated driver initialization, GPU kernel execution, numerical verification and recovery without rebooting on a reference setup pairing an NVIDIA RTX 4000 SFF Ada with an M3 Ultra Mac Studio, with compute through the public C ABI qualified at PCIe Gen1; Gen2 ×4 reached roughly twice the measured Gen1 transfer rate, while Gen3 and Gen4 remain in development. The project also ships a llama.cpp-based extension with an OpenAI-compatible API served by default at http://127.0.0.1:8277/v1, installable via `pip install hawking-runtime` on Python 3.12, with the eGPU implementation published as developer source for inspection and reproduction.

read2 min views12 publishedNov 16, 2026
A new implementation for LLMs through Nvidia GPUs natively on Apple Silicon
Image: Michielbdejong (auto-discovered)

Hawking Runtime is a local inference and GPU computing project for Apple Silicon Macs.

The ethos is simple: make better use of the hardware you already have.

The project currently has two parts:

  • Extension: Local model inference through llama.cpp, with an OpenAI-compatible API, and model management.
  • eGPU: A native macOS DriverKit driver, host runtime, CUDA kernels, hardware diagnostics, and developer tools for running NVIDIA GPUs connected over Thunderbolt.

Both are part of Hawking Runtime, but they can be used independently.

Requires an Apple Silicon Mac and Python 3.12.

python3.12 -m pip install hawking-runtime
hawking init
hawking add Qwen/Qwen3-0.6B-GGUF:Q8_0
hawking serve MODEL

Use the library ID printed by hawking add in place of MODEL, or a local .gguf path.

hawking init detects your Mac's available resources, sets up the pinned llama.cpp runtime, and configures supported coding agents it finds.

hawking serve starts a local OpenAI-compatible server supporting streaming, tool calls, and the Responses API.

Default endpoint: http://127.0.0.1:8277/v1

You can use it with OpenAI-compatible clients, coding agents, and interfaces such as Open WebUI.

See extension/README.md for the complete setup and configuration options.

We have demonstrated real NVIDIA GPU computation on Apple Silicon macOS.

Our reference setup uses an NVIDIA RTX 4000 SFF Ada connected over Thunderbolt to an M3 Ultra Mac Studio. We've demonstrated driver initialization, GPU kernel execution, numerical verification, and recovery without rebooting.

We have also experimented with PCIe link negotiation. Gen2 ×4 achieved approximately twice the measured Gen1 transfer rate. Gen3 and Gen4 are in development and not part of this release.

In this release, compute through the public C ABI is qualified on the reference setup at PCIe Gen1.

The eGPU implementation is available as developer source, including the DriverKit driver, native execution code, CUDA kernel catalogue, diagnostics, and build tooling.

cd egpu/operations/hardware/egpu_runtime
make test-fast

The purpose of publishing the source is to let developers inspect the implementation, reproduce the work, adapt it to their hardware, and build on it.

See egpu/operations/hardware/egpu_runtime/README.md, SIGNING.md, and docs/LIFECYCLE.md for the development workflow.

We encourage using a coding agent to work through the eGPU setup. Hardware configurations differ, and an agent can help inspect your system, build the relevant components, identify bottlenecks, and develop improvements for your particular setup.

You can start with a prompt like this:

Clone the Hawking Runtime repository and read the eGPU documentation. I want to connect and optimize my NVIDIA eGPU with my Apple Silicon Mac. Inspect my GPU, enclosure, PCIe connection, and macOS environment. Help me build and test the driver and runtime, identify compatibility issues, and find opportunities to improve transfer speeds and compute performance. Use reproducible benchmarks and reversible changes. Ask before installing or activating drivers, changing hardware policies, or performing anything that could affect system stability.

See LICENSE and NOTICE for the applicable terms and third-party attributions.

Hawking Systems Inc. · 2026

── more in #ai-infrastructure 4 stories · sorted by recency
── more on @hawking systems inc. 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/a-new-implementation…] indexed:0 read:2min 2026-11-16 · —