cd /news/large-language-models/janas-llm-37-tok-s-vs-llama-cpp-2-8-… · home › topics › large-language-models › article
[ARTICLE · art-173308] src=discuss.huggingface.co ↗ pub= topic=large-language-models verified=true sentiment=· neutral

Janas-LLM 37 tok/s vs llama.cpp 2.8 tok/s on Qwen3-Next-80B, on an Intel Core Ultra 9 185H with 32 GB RAM

An independent replication of the Janas-LLM benchmark on Qwen3-Next-80B found the engine's speed advantage is real but direction-specific, not the blended 8.6x headline: Janas-LLM generated 3.4 tokens per second versus llama.cpp's 0.53 (6.4x) and processed prompts at 13.9 tokens per second versus 10.25 (1.36x) under the author's own protocol of prompt 1024, 16 generated tokens and 2 rounds. The replicators reported the 8.6x blended figure did not reproduce on any run, including under the author's protocol with drafts off, and that at temperature 0 output was byte-identical across 8 of 8 runs only with drafts off, while drafts-on runs produced differing text. They also found the Hugging Face Hub API overstates the pinned file's size by exactly 70,000 bytes (48,411,992,432 reported versus 48,411,922,432 actual) while reporting the correct hash, and that placing only 12 of the model's 48 layers on a 4 GiB card with experts on the CPU still processed prompts at 10.58 ± 0.20 tokens per second but generated at just 0.64 ± 0.08 tokens per second.

recommendation

Benchmark prompt processing and token generation as separate numbers under a fixed protocol (prompt 1024, 16 generated, 2 rounds, drafts off), and verify model files by hash rather than the Hub API's size field.

more in playbook: Inference and self-hosting
read3 min views5 publishedNov 20, 2026

Thank you for publishing this bench, and for pinning the files — that is what made a true replication possible. We ran it on the hardware class the thread asked for — 16 GB of RAM and a discrete graphics card — using the exact pinned files, verified by hash before anything ran. The short version: the speed advantage is real, and it is a pair of numbers, not one.

Reading (prompt processing): the Janas engine ran 13.9 tokens per second against llama.cpp’s 10.25 under the baseline condition (the author’s own settings) and 10.05 under ordinary settings (llama.cpp’s stock defaults) — an advantage of 1.36 to 1.38 times. Writing (generation): 3.4 against 0.53 and 0.56 — 6.4 times over the baseline condition, 6.1 times against ordinary settings. The headline figure of 8.6 times, read as one blended number, did not reproduce on any run we made, including runs under the author’s own protocol with drafts off. What must hold for the advantage to be real is narrow and checkable: the two engines stream the same pinned file, the protocol is the author’s own — prompt 1024, 16 generated, 2 rounds — and the ratio is reported per direction — a reporting rule we set for this replication, because the moment the two directions are blended into one number, the figure stops describing either.

One correction to a claim in this thread’s own text, because it is the kind of thing that wastes the next person’s week: at temperature 0 the generated text is not stable in every regime. With the draft cache on, repeated runs produced different texts. With drafts off, the output was byte-identical across 8 of 8 runs, on two different processor paths. The variation lives entirely in the drafts-on cache regime; a determinism check that does not name its regime is measuring the regime, not the model.

A smaller find, about the files themselves: the Hub’s API reports this file’s size as 48,411,992,432 bytes. The file itself is 48,411,922,432 bytes — the API overstates it by exactly 70,000 bytes, while the hash it reports is the file’s true hash. For files of this class, verify by hash and treat the API’s size field as approximate.

And a rider, for anyone whose card cannot hold the model at all: in our own rider run we placed only 12 of the model’s 48 layers on a 4 GiB card, with the experts on the processor, and the same file still processed prompts at 10.58 ± 0.20 tokens per second, inside the author’s own published band on different hardware — while generating at 0.64 ± 0.08 tokens per second, which is the honest other half of that row: it runs, and it writes slowly. A model too large for the card is not the same as a model that cannot run.

One point carried openly, so the split in the measurements is visible rather than smoothed over: three different things carry the word “ordinary” around this bench, and they should not be confused. The pair above uses llama.cpp’s ordinary-settings row from the quiet replication — reading 10.05, writing 0.56 — and those figures stand. Separately, an earlier ordinary-condition run had one of its repetitions disturbed (its prompt readings spread ± 5.18 around 6.84), and a set of figures once printed for that leg from the wrong row is withdrawn; nothing above uses either. A clean re-run of that one condition is the only measurement this card still owes, and the pair above does not depend on it.

The check that could prove any of this wrong: run the author’s protocol — prompt 1024, generate 16, 2 rounds, drafts off — on any machine of this class, on the pinned files, and report the two directions separately. If writing comes out near a blended 8.6 times, or reading near 2 times, our read of this bench is wrong, and the numbers above say exactly by how much.

── more in #large-language-models 4 stories · sorted by recency
── more on @janas-llm 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/janas-llm-37-tok-s-v…] indexed:0 read:3min 2026-11-20 · —