The entire industry spent the last week talking about Jev.
TypeSafe AI correctly diagnosed a fundamental pathology in modern software architecture: we have been renting slow, expensive System 2 generative LLMs to make mundane System 1 decisions. You don’t need an autoregressive 70B parameter model spitting tokens to decide which queue an incident belongs in, whether a SQL query needs human approval, or which tool an agent should run next.
Jev solved the output waste by bounding the problem space to three typed primitives—Choice, Score, and Noul (boolean confidence)—and running them in a single parallel pass without token-by-token generation.
The thesis is spot on. But take a hard look at the telemetry coming out of production:
Why are we celebrating a 200ms round-trip for a discrete decision?
If you operate high-throughput distributed backends, game loops, local agent runtimes, or high-frequency order routers, 200 milliseconds is an eternity.
The moment your "System 1" engine sits behind TLS handshakes, JSON serialization, HTTP headers, and external cloud gateways, you haven't built an intuitive reflex—you’ve built a slightly cheaper remote bottleneck.
What does a real System 1 decision engine look like when it lives where your code actually executes?
When you call an external decision model like Jev over the wire, where does your time actually go?
The compute itself was fast, but the plumbing around it ate 80% of the budget.
If you are coordinating an AI agent that takes 15 discrete tool-choice steps per turn, that’s 3 full seconds of your user staring at a spinner just waiting for routing decisions.
In high-performance .NET systems, we treat allocations and boundaries as first-class constraints. If an operation doesn't mutate memory or generate arbitrary text, it should run:
This was the design motivation behind Glacier.Clavier.
Instead of treating Choice, Score, and Noul as HTTP endpoints, Clavier treats them as native, in-process evaluation primitives backed by low-overhead C# .NET 10 runtimes and unified local memory (such as AMD ROCm / RDNA unified APU memory or bare-metal local compute).
To get sub-millisecond execution, you cannot allow the GC to track intermediate state. Everything sent to and returned from the evaluation engine must be blittable:
// 8-byte blittable decision primitive - zero heap allocation
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public readonly record struct ChoiceDecision(
byte SelectedIndex,
Half Confidence,
Half MarginToSecond
);
When feeding state to the engine, instead of allocating JSON buffers, the engine operates directly over contiguous spans of memory:
ReadOnlySpan<byte> contextState = GetCurrentAgentStateBuffer();
ReadOnlySpan<ChoiceOption> candidateTools = stackalloc ChoiceOption[]
{
ToolCatalog.Grep,
ToolCatalog.FileWrite,
ToolCatalog.ExecuteSql
};
// Evaluates directly against in-memory tensor weights
ChoiceDecision decision = ClavierEngine.EvaluateChoice(contextState, candidateTools);
if (decision.Confidence > (Half)0.85f)
{
ExecuteTool(candidateTools[decision.SelectedIndex]);
}
| Metric | Cloud HTTP Decision API (Jev) | Native In-Process Engine (Glavier.Clavier) |
|---|---|---|
| P50 Latency | ~110 ms | ~140 µs |
| P99 Latency | ~450 ms | ~280 µs |
| Heap Allocations | Hundreds of KB (JSON payload/response) | 0 bytes (blittable structs) |
| Network Failure Mode | Gateway timeouts, retries, rate limits | None (deterministic execution) |
| Cost Per Decision | Pay per input million tokens | Zero incremental cost |
That is a ~1,000x difference in latency.
At 140 microseconds, a System 1 decision can be invoked inside a 60 FPS update loop, inside a live database stream, or 50 times in a single agent step without the user perceiving a single hiccup.
None of this is to say cloud decision models have no place.
If your backend is already an asynchronous serverless workflow routing tier-2 Zendesk tickets or classifying occasional incoming webhooks, an API like Jev is a massive win over a sluggish 8B parameter generative model. It eliminates hallucination, standardizes outputs, and cuts your inference bill dramatically.
But if you are building:
...then an external network hop contradicts the definition of System 1.
Reflexes must be local. They must be memory-efficient. And they shouldn't depend on an internet connection to decide which branch of your code to execute next.
What’s your threshold? If an agent requires 20 routing decisions to solve a task, would you tolerate a 4-second network overhead, or does decision-making belong in the runtime?
Check out the repo and implementation details here: github.com/ian-cowley/Glacier.Clavier