Ollaya: the open decision-model runner, with the numbers

Ollaya, an open source local runner for decision models, serves Jev-style models on user hardware with end-to-end HTTP API latency of 8.1ms on laya:multilingual, 9.6ms on laya:en, 14.7ms on gliclass and 20.4ms on nli on an RTX 4090, versus a 236-276ms median for the hosted TypeSafe Jev API across third-party benchmarks. Ollaya serves TypeSafe's /v1/systemone and /v1/models endpoints with official shapes, so TypeSafe Python SDK 0.7.1 works with the base URL pointed at localhost:11435, and it pulls seven model families pinned to a commit and sha256-checked from each author's own Hugging Face repo. The project's thread hit 484 points with 122 comments on September 25, with commenters noting Ollama could add decision-model support and that a trained classifier beats this for teams with an eval set.

TLDR Ollaya is an open source local runner for decision models that serves Jev-style models on your own hardware. It enables private routing, moderation, and guardrails with millisecond answers instead of a hosted LLM API, with no per-token bill. The difference between Ollaya and regular local LLM setups is that decision models return typed answers with confidence numbers attached instead of generated text, and the whole stack answers in a single forward pass. What shipped, in numbers The site’s latency chart has the main numbers. On an RTX 4090, a five-question request end to end through the HTTP API runs 8.1ms on laya:multilingual, 9.6ms on laya:en, 14.7ms on gliclass, and 20.4ms on nli, with laya in fp16 and the others in fp32. The hosted TypeSafe Jev API’s median request across third-party benchmarks sits at 236 to 276ms, network included. The site labels this an order-of-magnitude comparison rather than a head-to-head. The magnitude holds: 25 to 30 times faster, on hardware you own, with zero metering. Ollaya serves TypeSafe’s /v1/systemone and /v1/models endpoints with the official shapes, so the TypeSafe Python SDK 0.7.1 works with the base URL pointed at localhost:11435 and any string as the key. A response on the site’s example carries invoice 0.9698, refund 0.0172, other 0.013 with 43 input tokens and zero output tokens, because there is no generation step to bill. The roster, with authors and sizes Ollaya pulls seven model families, each pinned to a commit and sha256-checked from the author’s own Hugging Face repo, never re-hosted: laya Convai Innovations is the one to start with: hand it a ticket or an email, get back a choice with a confidence number, in English ModernBERT-large 421M or across 100+ languages mmBERT-base 322M , with a typed-decisions checkpoint at 0.766 accuracy. decider Mapika reads its answer from option-letter logits on a Qwen3.5 decoder, 0.75b and 1.9b, scoring 0.591 on typed decisions. nli Moritz Laurer turns every option into an entailment hypothesis, 396m and 435m, the most accurate encoder on typed decisions in Ollaya’s tests. gliclass Knowledgator scores all options in one pass so cost barely grows with option count. qwen3guard the Qwen team takes a message and answers whether it is safe, controversial, or unsafe, in 119 languages, at 0.6b. kev Jared Palmer is a LoRA plus pointer head on Qwen3.5, up to 7.9b. von Victor Hugo Panisa scores every option at its own ModernBERT marker with input-conditioned confidence fitting. Each model ships its own calibration confidence numbers you can put a threshold on , and a Modelfile refits those numbers on your labelled data. Calibration against your own ticket distribution is the difference between a demo and a threshold you can put in production. The wave this landed in Ollaya’s thread hit 484 points with 122 comments on September 25, and the surrounding week shows the scale. Jev Plays Pokemon is at 214 points, a live YouTube run where the harness reads the Game Boy’s RAM, lists legal options with facts, and Jev picks every menu answer, battle action, and even spells nicknames letter by letter the repo is explicit: no scripts, no cheats, hidden items never shown . A single-function Jev-like wrapper using logprobs sits at 97 points, running vision questions at 1 FPS on an RTX 3090 with Gemma 4 12B. Add Jevq for jq, Jevcut for auto-clipping, an emoji-only chatbot, a credit card chooser, Jev.Store, Jevlang, and Tenjin, and the daily sweep counted seven Show HN posts in 24 hours. The comment section’s smartest objection is that Ollama could add decision-model support at any time, and that a trained classifier beats this for teams with an eval set. Both objections are fair. The numbers above answer that: the hosted API round trip costs 25 to 30 times the local encoder, and the runner exists precisely so the routing layer is not a vendor’s API call. Whether Ollama ships this natively is now a question about their roadmap, not a reason the local option did not exist. Ollama did this for LLMs, ComfyUI did it for image models, and Ollaya does it for typed decisions: one binary, Apache-2.0 runtime, weights pinned to commits, 8ms on the card most people already own. Sources: Ollaya site https://ollaya.dev/ latency chart and model roster · ollaya-dev/ollaya on GitHub https://github.com/ollaya-dev/ollaya Apache-2.0 · Ollaya HN thread https://news.ycombinator.com/item?id=49848269 484 points · Laya model card https://huggingface.co/convaiinnovations/laya · Jev Plays Pokemon https://github.com/christianmat/jev-pokemon harness README · Jev-like wrapper https://allanrbo.blogspot.com/2026/09/a-jev-like-wrapper-for-llms-including.html Related on this site: Laya: multilingual decision model https://tokenstead.ai/guides/laya-multilingual-decision-model · What is Jev, in plain English https://tokenstead.ai/guides/what-is-jev-plain-english · Decision models ate routers https://tokenstead.ai/guides/decision-models-ate-routers