# Run a private OpenAI-compatible LLM endpoint on Apple Silicon with one Rust binary

> Source: <https://dev.to/jinxuan_ai/run-a-private-openai-compatible-llm-endpoint-on-apple-silicon-with-one-rust-binary-29h0>
> Published: 2026-09-02 03:12:46+00:00

Disclosure: I maintain [Ferrum](https://github.com/sizzlecar/ferrum-infer-rs), an MIT-licensed local LLM inference server written in Rust.

If you want a private OpenAI-compatible endpoint on an M-series Mac without setting up Python or a container, this is the shortest path I currently recommend.

The commands below were tested end to end with the published Ferrum v0.8.3 Homebrew build on an M1 Max.

```
brew tap sizzlecar/ferrum
brew install ferrum
ferrum doctor qwen3.5:4b-q4_k_m
```

`ferrum doctor`

checks the local setup before inference. In one real install it caught missing Xcode Command Line Tools before the user reached a harder-to-diagnose failure.

**The first run downloads about 2.55 GiB.** The terminal can look quiet during that download, so give it time before assuming it has hung.

```
ferrum run qwen3.5:4b-q4_k_m --disable-thinking
```

Qwen3.5 emits verbose reasoning by default. `--disable-thinking`

gives a more conventional first-chat experience. Omit the flag when you want the reasoning behavior.

```
ferrum serve \
  --model qwen3.5:4b-q4_k_m \
  --served-model-name ferrum \
  --disable-thinking \
  --port 8000
```

In another terminal:

```
curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "ferrum",
    "messages": [{"role": "user", "content": "Reply exactly: ferrum-ok"}],
    "max_tokens": 32
  }'
```

A working setup returns HTTP 200 with a non-empty assistant response.

I am looking for failure-oriented feedback from Apple Silicon users:

`doctor`

, the model download, or the first API request fail?Repository and issue tracker: [github.com/sizzlecar/ferrum-infer-rs](https://github.com/sizzlecar/ferrum-infer-rs)

*This post was drafted with assistance from OpenAI Codex. The commands and stated behavior were checked against the published build; no comparative performance claim is being made here.*