We serve a sovereign inference tier from four Intel Arc Pro B60 — here's what it actually does (and doesn't)

Since this week, SOKKAN Inference has a tier called "Swiss". It is served from a machine in Meyrin, Geneva, on hardware we own, with no NVIDIA card inside. This post is the honest version of what that machine does, with the numbers we measured — including the ones that don't scale.

We are a small Swiss company. Our inference product already had two cloud tiers, EU-sovereign, served from French datacenters. What we didn't have was an answer for the customer who says "the data cannot leave Switzerland, full stop" — and, longer term, for the customer who says "the data cannot leave the building".

That meant buying GPUs, in 2026, in the middle of a memory shortage. We chased second-hand RTX 3090s for a month and lost every auction above market price. Meanwhile the Intel Arc Pro B60 — 24 GB of VRAM, a 2-slot blower, natively x8 — was still selling near MSRP, at CHF 614 a card. Four of them came to about CHF 2,450 for 96 GB of VRAM. That was the whole argument: VRAM per franc, available now.

The bet was on the software stack, not the silicon. The risk with Intel in 2026 is not the hardware; it is whether vLLM, llama.cpp and the XPU runtime hold up under a service you sell.

The chassis is a 2018 X299 workstation (i9-9980XE, 64 GB DDR4) that also carries our production job platform — Postgres, Celery workers, a few web front ends. Not a clean benchmark rig. Every number below is therefore pessimistic; the production appliance is an EPYC board dedicated to inference.

Four B60s, each on its own x8 Gen3 root port, run three resident models:

All three are MoE models with about 3B active parameters, which is what makes the cohabitation possible: 130B of cumulated weights at the speed of a small model.

The Swiss tier is card 0.

gpt-oss-20b under vLLM, warm, on one B60:

Prefill is around 5,000 tok/s on a 4.3k-token prompt. In an earlier run at 8 concurrent requests the same card reached 241 tok/s aggregate. Continuous batching does exactly what it says on a model this size: throughput grows almost linearly with load, and per-request speed barely moves.

Power for the four cards: 163 W idle with all three models loaded, 211 W peak under 12 concurrent requests across the three models. Temperatures 52–62 °C. On the UPS, the whole machine — including the production it also carries — moves from 13% to 19% load. At Swiss electricity prices that is roughly CHF 750 a year for the full box, running around the clock.

Tool calling works on all three models. For gpt-oss you need --enable-auto-tool-choice --tool-call-parser openai; without the harmony parser you get zero tool calls and an empty content field, and you will spend an evening wondering why.

The 80B model is the one people want — it is the better writer, the better analyst. It is also the one that does not scale, and the reason is the runtime, not the model.

llama.cpp has no continuous batching. Under concurrency, the same 80B goes from 32.0 tok/s for one request, to 14.5 tok/s each for two, to 7.0 tok/s each for four. Worse: the aggregate throughput regresses — 32 → 28.8 → 20.7 tok/s. Four users on that model get less total work done than one.

So why is it on llama.cpp? Because vLLM on Intel XPU currently only serves quantisations that have a native XPU kernel. MXFP4 gpt-oss does. Every other quantised MoE we tried routes its expert layers to Marlin, an NVIDIA kernel, and dies with device_capability = -1 regardless of the --quantization flag. Dense AWQ works (int4_gemm_w4a16, and you must pass --dtype float16); AWQ MoE does not. Everything that is not gpt-oss goes through llama.cpp SYCL, and llama.cpp SYCL is honestly good — Qwen3-Next's hybrid attention runs on it — but it serves one user well and four users badly.

We also could not get vLLM tensor-parallel across the four cards to stay up on this platform. four workers in parallel wedged the machine three times in one day, once with 59 GB of RAM free and production stopped, so it is not memory; it looks like a DMA/PCIe storm at load on an old chipset. llama.cpp, which loads sequentially, spreads gpt-oss-120b across the four cards without incident — 18.8 tok/s decode, ~250 tok/s prefill, stable — but that prefill number is why you won't see the 120b behind a public tier yet. We will retry TP on the EPYC board.

Two smaller lessons that cost us real time: models must be loaded sequentially (the memory peak is at load, not at serve — three simultaneous docker start at boot reproduce a crash), and the first request after is 3–4× slower than steady state, so warm the service before a customer touches it.

Given all that, the Swiss tier is deliberately small and deliberately honest:

  export ANTHROPIC_BASE_URL=https://infer.sokkan.ch
  export ANTHROPIC_AUTH_TOKEN=sik_your_token
  export ANTHROPIC_MODEL=sokkan-swiss

Reselling raw tokens from one card will never be a business — at these prices a saturated B60 earns about what it burns in electricity. The point is different: the same machine hosts services on top of the model (a website assistant for SMEs, a finance assistant for our ERP), and the Swiss tier is the door for developers who want to build the same kind of thing with the same residency guarantee.

The same four-card configuration becomes SOKKAN Anchor, an on-prem appliance for organisations that have banned the cloud altogether. The dimensioning rule we learned the hard way goes into the sales sheet: anything real-time — voice, interactive completion — must hit a model served by vLLM; the big llama.cpp model serves two simultaneous users, not a department.

If you are running Arc Pro cards for inference and hit the same walls (or different ones), I would genuinely like to compare notes.

The Swiss tier is live at sokkan.ch/en/inference. SOKKAN is a product of Ninabot Sàrl, Geneva.