TLDR #
Ollaya is an open source local runner for decision models that serves Jev-style models on your own hardware. It enables private routing, moderation, and guardrails with millisecond answers instead of a hosted LLM API, with no per-token bill. The difference between Ollaya and regular local LLM setups is that decision models return typed answers with confidence numbers attached instead of generated text, and the whole stack answers in a single forward pass.
What shipped, in numbers #
The site’s latency chart has the main numbers. On an RTX 4090, a five-question request end to end through the HTTP API runs 8.1ms on laya:multilingual, 9.6ms on laya:en, 14.7ms on gliclass, and 20.4ms on nli, with laya in fp16 and the others in fp32. The hosted TypeSafe Jev API’s median request across third-party benchmarks sits at 236 to 276ms, network included. The site labels this an order-of-magnitude comparison rather than a head-to-head. The magnitude holds: 25 to 30 times faster, on hardware you own, with zero metering.
Ollaya serves TypeSafe’s /v1/systemone and /v1/models endpoints with the official shapes, so the TypeSafe Python SDK 0.7.1 works with the base URL pointed at localhost:11435 and any string as the key. A response on the site’s example carries invoice 0.9698, refund 0.0172, other 0.013 with 43 input tokens and zero output tokens, because there is no generation step to bill.
The roster, with authors and sizes #
Ollaya pulls seven model families, each pinned to a commit and sha256-checked from the author’s own Hugging Face repo, never re-hosted:
laya (Convai Innovations) is the one to start with: hand it a ticket or an email, get back a choice with a confidence number, in English (ModernBERT-large 421M) or across 100+ languages (mmBERT-base 322M), with a typed-decisions checkpoint at 0.766 accuracy. decider (Mapika) reads its answer from option-letter logits on a Qwen3.5 decoder, 0.75b and 1.9b, scoring 0.591 on typed decisions. nli (Moritz Laurer) turns every option into an entailment hypothesis, 396m and 435m, the most accurate encoder on typed decisions in Ollaya’s tests. gliclass (Knowledgator) scores all options in one pass so cost barely grows with option count. qwen3guard (the Qwen team) takes a message and answers whether it is safe, controversial, or unsafe, in 119 languages, at 0.6b. kev (Jared Palmer) is a LoRA plus pointer head on Qwen3.5, up to 7.9b. von (Victor Hugo Panisa) scores every option at its own ModernBERT marker with input-conditioned confidence fitting.
Each model ships its own calibration (confidence numbers you can put a threshold on), and a Modelfile refits those numbers on your labelled data. Calibration against your own ticket distribution is the difference between a demo and a threshold you can put in production.
The wave this landed in #
Ollaya’s thread hit 484 points with 122 comments on September 25, and the surrounding week shows the scale. Jev Plays Pokemon is at 214 points, a live YouTube run where the harness reads the Game Boy’s RAM, lists legal options with facts, and Jev picks every menu answer, battle action, and even spells nicknames letter by letter (the repo is explicit: no scripts, no cheats, hidden items never shown). A single-function Jev-like wrapper using logprobs sits at 97 points, running vision questions at 1 FPS on an RTX 3090 with Gemma 4 12B. Add Jevq for jq, Jevcut for auto-clipping, an emoji-only chatbot, a credit card chooser, Jev.Store, Jevlang, and Tenjin, and the daily sweep counted seven Show HN posts in 24 hours.
The comment section’s smartest objection is that Ollama could add decision-model support at any time, and that a trained classifier beats this for teams with an eval set. Both objections are fair. The numbers above answer that: the hosted API round trip costs 25 to 30 times the local encoder, and the runner exists precisely so the routing layer is not a vendor’s API call. Whether Ollama ships this natively is now a question about their roadmap, not a reason the local option did not exist.
Ollama did this for LLMs, ComfyUI did it for image models, and Ollaya does it for typed decisions: one binary, Apache-2.0 runtime, weights pinned to commits, 8ms on the card most people already own.
Sources: Ollaya site (latency chart and model roster) · ollaya-dev/ollaya on GitHub (Apache-2.0) · Ollaya HN thread (484 points) · Laya model card · Jev Plays Pokemon (harness README) · Jev-like wrapper Related on this site: Laya: multilingual decision model · What is Jev, in plain English · Decision models ate routers