Stop Using LLMs for Simple Decisions: How to Build Fast, Cheap AI Agents with Clef

Cloudflare released Clef and Clef-flash, open-source decision models trained for bounded, typed decisions that run natively on its Workers AI edge network. The models replace autoregressive JSON generation with a single forward pass that slices output logits over declared enum options, cutting decision latency to roughly 38 milliseconds and eliminating schema hallucinations. Clef adds a vision encoder for screenshots and PDFs and doubles the earlier Jev model's context window to 64,000 tokens, with weights published under Apache 2.0 on Hugging Face.

A few weeks ago, we explored a radical shift in artificial intelligence architecture: The rise of “System One” decision models. For three years, the software industry operated under the hypnotic assumption that the future of computing was conversational. We begged chatbots to speak JSON, wrote retry wrappers to catch syntax errors, and waited four seconds while an autoregressive model sequentially emitted tokens: { , "status" , : , "approved" , } . Natural language is an extraordinary interface for humans. But for software? Strings are a catastrophic abstraction. Most of what an autonomous agent does during a multi-step workflow is not writing essays or solving complex mathematical theorems. It is routing, scoring, triaging, and branching . When an agent needs to answer a simple question: - “Does this customer request require an executive refund?” - “Is this inbound URL a phishing domain?” - “Should this workflow branch to the billing database or the support queue?” Using a 70-billion-parameter generative model like Claude Sonnet or GPT-4o is the modern equivalent of chartering a cargo plane to deliver a single postcard. You pay $15 per million tokens, suffer 3,000 milliseconds of latency, and pray the model does not hallucinate a missing key. A couple of weeks back, Cloudflare introduced a major open-source breakthrough that brings decision models directly to the edge: Clef and Clef-flash. Trained specifically for bounded, typed decisions, Clef is fully compatible with the Jev API, runs natively on Cloudflare’s global Workers AI network, and slashes decision latency down to 38 milliseconds . Here is how Clef works, why it outperforms generative LLMs for agent control loops, and how to wire it into your production stack. 1. What Is a Decision Model? Generative vs. Bounded To understand why Clef is so fast, you have to look at the mathematical difference between generative decoding and bounded decision evaluation: Generative LLM Autoregressive Token by Token Prompt ──► Forward Pass 1 "{" ──► Pass 2 "\n" ──► Pass 3 '"approved"' ──► 2,000 - 4,000ms Decision Model Single-Pass Logit Slicing Input State + Typed Schema ──► Single Forward Pass ──► Calibrated Probability Tensor ──► 38ms In a traditional generative LLM, producing a structured JSON decision requires 30 to 100 sequential forward passes . The model generates one character at a time, calculating attention across the entire preceding token history for every byte. A decision model does not generate text. It operates in a single forward pass: 1. You feed the model an unstructured input state a security log, an email, or a customer ticket alongside typed questions with declared enum options. 2. The model evaluates the state and slices its output logits strictly over the allowed candidate choices. 3. In 38 to 120 milliseconds , the model returns typed scores, probabilities, and confidences. Zero text decoding. Zero JSON parsing errors. Zero schema hallucinations. 2. Clef vs. Jev: The Evolution of System One When TypeSafe AI originally introduced Jev, it proved the commercial viability of non-generative models. But Jev was proprietary, cloud-hosted, and restricted to text-only classification. Cloudflare took that paradigm and elevated it across three critical dimensions: 1. Open-Source and Edge-Native: Cloudflare fully open-sourced the weights for Clef under the Apache 2.0 license on Hugging Face, while deploying it across their serverless edge network on Workers AI. 2. Native Multimodal Vision: Unlike earlier text-only decision models, Clef incorporates a vision encoder. It can ingest raw screenshots, PDF invoices, and webcam streams, returning typed decisions directly from visual pixels. 3. 64k Context Window: Clef doubles Jev’s context window to 64,000 tokens, allowing agents to pass extensive API logs, codebase diffs, and customer histories into a single decision call. In benchmark evaluations on the Jev Decision Index , Cloudflare tested Clef against frontier models across 43 diverse tasks: While specialized ultra-fast models like Laya run in 5ms, they suffer catastrophic accuracy drops on complex reasoning tasks scoring 14.2 on Banking intent . Clef-flash hits the sweet spot: sub-40ms speed with top-tier enterprise accuracy. 3. The Two-Tier Agent Architecture In production agent harnesses, you should never make an architectural choice between a decision model and a reasoning LLM. You combine them into a Two-Tier Agent Control Loop : Here is how the two tiers interact: Tier 1: The Edge Decision Layer Fast Path Every incoming request, database trigger, or user message first hits Clef-flash running on Workers AI at the edge: - Sub-40ms Triage: Clef evaluates the input against strict enums: “Is this request safe? What is the user’s intent? Does it need escalation?” - Deterministic Execution: If the decision is unambiguous e.g. 98% confidence that a ticket is a billing dispute , your deterministic application code immediately executes the corresponding API route without ever touching a heavy generative model. Tier 2: The Frontier Reasoning Layer Slow Path Only when Clef returns high entropy e.g. confidence scores split evenly across options or explicitly classifies a task as “Requires Complex Synthesis” does the system invoke a heavy generative model like Claude 3.7 Sonnet or OpenAI o3 . In production testing at Cloudflare, this two-tier split diverted over 85% of traffic away from expensive generative LLMs , slashing agent pipeline latency from 4.7 seconds down to 2.2 seconds while cutting cloud compute bills by more than 70%. 4. The 3 Production Scars: Where Decision Models Break Before replacing your LLM classifiers with Clef, keep these three production lessons in mind: Scar 1: The Ambiguous Classification Cliff Decision models are forced-choice systems. If your schema provides three options Billing , Technical , Account , and a user sends a completely unrelated query like “Tell me a joke” , a naive decision model will still force a prediction into one of the three categories. - The Guardrail: Always include an explicit Other / Unmatched choice in your schema, and enforce a minimum confidence floor e.g. 0.70 . Any decision falling below that floor must automatically route to human review or a fallback handler. Scar 2: Reward Function Exploitation in RL Fine-Tuning Cloudflare introduced an automated Reinforcement Learning RL platform allowing enterprises to fine-tune Clef on their own data. If your reward function naively rewards high confidence without penalizing false positives, the model will rapidly overfit, outputting 99% confidence on edge cases it actually misunderstands. - The Guardrail: Calibrate reward functions with asymmetric loss: penalize confident wrong predictions three times more severely than humble low-confidence uncertainties. Scar 3: Cold-Start Latency Spikes on Serverless Workers When deploying open-source models on serverless GPU clusters, the first request hitting a cold edge node can suffer a 500ms model-loading penalty. - The Guardrail: Pre-warm workers or deploy Clef on dedicated Workers AI reserved capacity to guarantee predictable sub-50ms execution on critical transaction paths. 5. How to Prototype This Weekend Python Blueprint Because Clef is fully Jev-API compatible, you can query it using standard Python HTTP requests or the Workers AI SDK: python Minimal Clef-Flash Decision Model Client import requests def evaluate decision input text: str, categories: list str , api token: str - dict: url = "https://api.cloudflare.com/client/v4/accounts/{ACCOUNT ID}/ai/run/@cf/cloudflare/clef-flash" headers = {"Authorization": f"Bearer {api token}", "Content-Type": "application/json"} Declare the typed decision schema Jev-compatible payload = { "input": input text, "questions": { "name": "intent category", "type": "choice", "options": categories }, { "name": "is urgent", "type": "noul" Calibrated probability 0.0 to 1.0 } } response = requests.post url, json=payload, headers=headers return response.json "result" Example usage: ticket = "My production database has been locked for 30 minutes, please help immediately " categories = "billing", "account settings", "infrastructure outage", "unrelated" decision = evaluate decision ticket, categories, api token="CF TOKEN" print f"Category: {decision 'intent category' 'selected' } {decision 'intent category' 'confidence' :.2f} " print f"Urgency Probability: {decision 'is urgent' 'probability' :.2f}" Output: Category: infrastructure outage 0.98 , Urgency Probability: 0.99 6. The Strategic Bottom Line For software leaders and system architects, Cloudflare’s release of Clef marks a critical milestone in the maturity of artificial intelligence: The era of using massive generative chatbots for simple software decisions is coming to an end. Just as a musical clef defines the domain and structure of the notes that follow it, a decision model defines the constraints and routing of the agentic actions that follow. By deploying compact, open-source decision models at the edge: 1. You slash agent control loop latency from seconds to milliseconds. 2. You eliminate schema hallucinations and JSON parsing retry loops. 3. You reserve your expensive generative models for the few creative tasks that actually require them. The fastest, most reliable software agents of the future will not be the ones that talk the most. They will be the ones that decide the fastest. Further Reading & Resources - Cloudflare Blog: Introducing Clef Open-Source Decision Models https://blog.cloudflare.com/clef-decision-models : The official technical release and benchmark analysis across 43 evaluation suites. - Cloudflare Hugging Face Repository: Clef Weights https://huggingface.co/Cloudflare/clef : Download the open-source Apache 2.0 weights to run Clef locally or on internal GPU clusters. - TypeSafe AI: Introducing System One Models and Jev https://typesafe.ai/blog/introducing-system-one-models-and-jev : The foundational paper introducing non-generative decision models for software execution. - Jev Decision Index Benchmark https://huggingface.co/spaces/multimodalart/jev-decision-index : Open community leaderboard evaluating decision model speed, accuracy, and calibration.