Show HN: Strot – Compile an LLM's intent once, run decisions locally forever

Strot released an open-source Python package, strot-ai, that uses an LLM once at compile time to build a frozen local decision artifact, then classifies, scores and routes records on CPU at roughly 60 ms per record with zero marginal cost and zero LLM calls at runtime. On a 144M-parameter SupersonicLabs/Julia-1 model, Strot reports 92.5% grounded routing accuracy across 398 records versus 74.8% for a naive approach (+18 points) and 86.7% versus 72.7% on AG News with unseen labels (+14 points), with low-confidence records abstaining via split conformal prediction. The package ships with published benchmarks and losses, and its typed decisions matched a leading hosted decision API's own example nearly exactly (sentiment 1.79 versus 1.8).

Compile your intent once with an LLM. Then decide on every record forever — classification, scoring, routing — as type-safe JSON with probabilities and confidence, on your own hardware, at zero marginal cost . Website https://strot.ai · Docs https://github.com/strot-ai/Project-STROT/blob/main/DEVELOPERS.md · Benchmarks https://github.com/strot-ai/Project-STROT/blob/main/benchmarks · PyPI Agents at compile time. Never at runtime. pip install strot-ai python import strot result = strot.run tickets, query="Triage incoming support tickets into the correct workflow.", action registry={ "billing dispute": "Refunds, invoice corrections, double charges", "technical defect": "Crashes, bugs, broken behavior", }, llm="anthropic/claude-opus-4-8", used once, on first compile only for o in result.outcomes: print o "action" , round o "confidence" , 2 billing dispute 0.97 technical defect 0.94 The LLM runs once , at compile time. Production is a frozen, self-contained Python artifact: a small local decision model arbitrates every record, nothing is generated, every output is bounded, and low-confidence records abstain instead of guessing. STROT = Agentic Schema Intent + Compiler & State + Code + Execution + NLU ⟹ JSON Decisions classification · scoring · routing · typed answers COMPILE once per intent + schema intent + sample ─▶ synthesize ─▶ golden gate ─▶ teacher labels ─▶ calibrate labels, your known annotate fit the projection, answers must a sample, confidence handlers pass once threshold │ ▼ frozen artifact .py RUNTIME every record, forever record ─▶ projection ─▶ decision model ─▶ confidence gate ─▶ typed JSON compiled 144M, CPU, decide or decision ~60 ms abstain run — route each record to an action and execute the compiled handler. decide — answer typed questions choice / score / boolean per record: result = strot.decide records, questions={ "is bug report": {"type": "boolean", "instructions": "mentions crashes/errors?"}, "customer sentiment":{"type": "score", "min": 1, "max": 5, "instructions": "1 angry, 5 happy"}, "routing bucket": {"type": "choice", "options": "billing", "engineering", "spam" }, } → {"booleans": {...}, "scores": {...}, "choices": {...}} with probabilities + confidence Real model SupersonicLabs/Julia-1, 144M, CPU , real datasets, clean Docker containers: | | Result | |---|---| | Grounded routing accuracy 398 records | 92.5% compiled vs 74.8% naive — the compiler is worth +18pp on the same model | | Semantic text AG News, unseen labels | 86.7% compiled vs 72.7% naive +14pp | | Typed decisions vs the leading hosted decision API's own example | near-identical sentiment 1.79 vs 1.8 | | Latency / marginal cost / LLM calls per record | ~60 ms · $0 · 0 | Reproduce everything: benchmarks/ https://github.com/strot-ai/Project-STROT/blob/main/benchmarks — with the losses published too. - Compile once, decide forever — content-addressed artifact cache; schema drift triggers exactly one recompile. - Golden-record promotion — artifacts must pass your known answers before they serve. - Fitted guarantees — temperature scaling + split conformal prediction, baked in: dispatched decisions are wrong ≤ α of all records marginal ; the rest abstain. - Measured, not guessed — the compiler scores labelings, and can train a distilled head, keeping only what beats the baseline on held-out data. - 0% structural hallucination — output is selection + computation over your registry; nothing is generated at runtime. - Model-independent — the decision model is a slot nlu= : local Julia-1 by default, hosted APIs, or any drop-in specialist. pip install strot-ai core: pure stdlib pip install strot-ai local + the local decision model torch/transformers pip install strot-ai hosted + openai, for a hosted decision runtime pip install strot-ai anthropic + anthropic, for the synthesizer Bake the local model into your image for offline production: python RUN pip install strot-ai local \ && python -c "import strot; strot.prefetch nlu local dir='/opt/julia-1', install=True " ENV STROT NLU CHECKPOINT=/opt/julia-1 Never pip install julia — unrelated package. DEVELOPERS.md https://github.com/strot-ai/Project-STROT/blob/main/DEVELOPERS.md — architecture, compile lifecycle, calibration semantics and caveats, deployment, model swapping, full API reference. MIT. Default decision model: Apache-2.0 SupersonicLabs/Julia-1 https://huggingface.co/SupersonicLabs/Julia-1 . Implements ideas from STROT arXiv:2505.01636 .