# Janas-LLM 37 tok/s vs llama.cpp 2.8 tok/s on Qwen3-Next-80B, on an Intel Core Ultra 9 185H with 32 GB RAM

> Source: <https://discuss.huggingface.co/t/janas-llm-37-tok-s-vs-llama-cpp-2-8-tok-s-on-qwen3-next-80b-on-an-intel-core-ultra-9-185h-with-32-gb-ram/190340#post_2>
> Published: 2026-11-20 15:07:25+00:00

**Recommendation:** Benchmark prompt processing and token generation as separate numbers under a fixed protocol (prompt 1024, 16 generated, 2 rounds, drafts off), and verify model files by hash rather than the Hub API's size field.
More in the playbook "Inference and self-hosting": <https://wpnews.pro/playbooks/inference-serving>

Thank you for publishing this bench, and for pinning the files — that is what made a true replication possible. We ran it on the hardware class the thread asked for — 16 GB of RAM and a discrete graphics card — using the exact pinned files, verified by hash before anything ran. The short version: the speed advantage is real, and it is a pair of numbers, not one.

Reading (prompt processing): the Janas engine ran 13.9 tokens per second against llama.cpp’s 10.25 under the baseline condition (the author’s own settings) and 10.05 under ordinary settings (llama.cpp’s stock defaults) — an advantage of 1.36 to 1.38 times. Writing (generation): 3.4 against 0.53 and 0.56 — 6.4 times over the baseline condition, 6.1 times against ordinary settings. The headline figure of 8.6 times, read as one blended number, did not reproduce on any run we made, including runs under the author’s own protocol with drafts off. What must hold for the advantage to be real is narrow and checkable: the two engines stream the same pinned file, the protocol is the author’s own — prompt 1024, 16 generated, 2 rounds — and the ratio is reported per direction — a reporting rule we set for this replication, because the moment the two directions are blended into one number, the figure stops describing either.

One correction to a claim in this thread’s own text, because it is the kind of thing that wastes the next person’s week: at temperature 0 the generated text is not stable in every regime. With the draft cache on, repeated runs produced different texts. With drafts off, the output was byte-identical across 8 of 8 runs, on two different processor paths. The variation lives entirely in the drafts-on cache regime; a determinism check that does not name its regime is measuring the regime, not the model.

A smaller find, about the files themselves: the Hub’s API reports this file’s size as 48,411,992,432 bytes. The file itself is 48,411,922,432 bytes — the API overstates it by exactly 70,000 bytes, while the hash it reports is the file’s true hash. For files of this class, verify by hash and treat the API’s size field as approximate.

And a rider, for anyone whose card cannot hold the model at all: in our own rider run we placed only 12 of the model’s 48 layers on a 4 GiB card, with the experts on the processor, and the same file still processed prompts at 10.58 ± 0.20 tokens per second, inside the author’s own published band on different hardware — while generating at 0.64 ± 0.08 tokens per second, which is the honest other half of that row: it runs, and it writes slowly. A model too large for the card is not the same as a model that cannot run.

One point carried openly, so the split in the measurements is visible rather than smoothed over: three different things carry the word “ordinary” around this bench, and they should not be confused. The pair above uses llama.cpp’s ordinary-settings row from the quiet replication — reading 10.05, writing 0.56 — and those figures stand. Separately, an earlier ordinary-condition run had one of its repetitions disturbed (its prompt readings spread ± 5.18 around 6.84), and a set of figures once printed for that leg from the wrong row is withdrawn; nothing above uses either. A clean re-run of that one condition is the only measurement this card still owes, and the pair above does not depend on it.

The check that could prove any of this wrong: run the author’s protocol — prompt 1024, generate 16, 2 rounds, drafts off — on any machine of this class, on the pinned files, and report the two directions separately. If writing comes out near a blended 8.6 times, or reading near 2 times, our read of this bench is wrong, and the numbers above say exactly by how much.
