# Out-Of-Distribution Errors – What Might Be Lurking In The Long Tail?

> Source: <https://codemanship.wordpress.com/2026/09/14/out-of-distribution-errors-what-might-be-lurking-in-the-long-tail/>
> Published: 2026-09-14 06:34:33+00:00

In my previous post, I derived a formula from my Special Theory of Autonomous Agent Reliability that predicts the [reliable horizon](https://codemanship.wordpress.com/2026/09/12/out-of-distribution-checks-how-human-intelligence-stabilises-agentic-workflows/) over which a coding agent can operate before Actual Intelligence needs to step in to check for unanticipated errors.

I theorised that the reliability of a single step – a model interaction – in an agentic workflow is:

R = 1 – (1 – C)(1 – P)

Where C is the probability of the model getting it right, and P is the probability of errors being caught before they propagate and compound.

*1 – P* is doing a lot of the work here. It represents the probability of errors our automated quality checks didn’t anticipate – errors that are outside their distribution (we didn’t write a test for that, the linter doesn’t check for that, the model wasn’t trained on that, etc).

Those of us who care about reliability – and I appreciate we are rare beasts, verging on extinct in 2026 – might well ask “What’s in that long tail of out-of-distribution errors?”

These are the errors that are less likely to occur and more difficult to spot, and presumably therefore underrepresented in LLM pre-training and reinforcement learning.

Labs can use automated tests as reward signals in reinforcement learning, for example. *assertTrue(2 + 3 == 4)* will fail every time in exactly the same way, and that makes it a great feedback signal. But – without complete control over thread scheduling and/or interleaving of multithreaded operations during testing – a race condition may well give rise to a Heisenbug and a test that fails occasionally. Not such a great feedback signal. Quite a noisy signal, in fact.

No doubting models will have gorged on many, many examples of code that has concurrency bugs in it, but it would be very challenging to train models to recognise which pieces of concurrently-interacting code did the deed.

We might pin our hopes on the model “reasoning” about concurrent behaviour, but their lack of temporal reasoning (A happens before B) is well-documented. If you don’t believe me, play one at Rock-Paper-Scissors, insist it goes first every round, then ask it to explain why it keeps losing.

Other weaknesses of deep neural networks in pattern recognition – like probability collapse over extended sequences (play one at chess and you’ll see), and the difficulty in training them on long-range patterns – push other classes of bugs into the long tail of *1 – P*.

From experience, here’s a rough taxonomy of errors I’ve discovered after quality gates missed them. With each category, I’ve drawn on my experience working on high-integrity systems, plus what I’ve learned about working with LLMs and coding agents, to suggest potential mitigations, and alternative detection techniques I’ve seen work much better.

And, yes, testing in that long tail *is* starting to look [rather expensive](https://codemanship.wordpress.com/2026/08/28/the-wall-confronting-reliable-coding-agent-autonomy/), isn’t it? And there was you thinking you were way ahead with mutation testing!

| Category of Error | Why OOD? | Mitigation Strategies | Better Detection Techniques | 
|---|---|---|---|
| **Temporal & concurrency errors** | LLMs have weak temporal reasoning. Correctness depends on *ordering and interleaving* , not merely the code visible at one instant. Rare schedules create a combinatorial tail and provide poor/repeatability-challenged training signals: the same buggy code may usually appear to work. | Make ordering, ownership and synchronization explicit; structured concurrency; explicit state machines; more thinking to enumerate possible interleavings. | Race detectors; controlled/systematic concurrency testing; model checking; temporal verification. | 
| **Path & history-dependent errors** | Correctness depends on a *sequence* of states/actions rather than the current state. The number of possible paths grows combinatorially with sequence length. This resembles**probability/reliability collapse over extended sequences** : even high per-step accuracy compounds badly over long reasoning chains. | Short workflows; explicit state; constrain legal transitions; provide execution history; more thinking to trace paths explicitly. | Stateful property testing; model-based testing; sequence fuzzing; model checking. | 
| **Long-range & hidden dependencies** | Cause and effect may be separated by large semantic distance or dependencies may not be signposted at all. Training LLMs to learn very long-range relationships is difficult, and simply fitting both ends into the context doesn’t guarantee the model will associate them. Dynamic typing, reflection, DI, events and configuration make discovery harder still. | High cohesion; small blast radius; explicit dependencies/interfaces; type hints; contracts; architectural boundaries; supply call/dependency graphs; more thinking/search to reconstruct relationships. | Compilers/type checkers; static analysis; call/dependency analysis; architecture tests; integration/contract tests. | 
| **Combinatorial & boundary errors** | Familiar features can interact in combinations that are individually IID but collectively novel. Rare input combinations, numerical boundaries, configuration combinations and adversarial inputs naturally occupy the distribution’s long tail. | Constrain valid states; strong types/schemas; explicit ranges and invariants; simplify configuration; more thinking to systematically enumerate edge cases. | Property-based testing; fuzzing; combinatorial testing; boundary analysis; sanitizers; security analysis. | 
| **External-state & environment errors** | Relevant information isn’t contained in the code—or perhaps in the model’s context at all. Correctness depends on runtime configuration, filesystem, clock, timezone, OS, external services, network behaviour, permissions, deployment state, etc. No amount of reasoning can recover facts that aren’t available. | Make environmental assumptions explicit; hermetic environments; configuration schemas; expose relevant runtime state/telemetry to the agent. | Integration/environment-matrix testing; contract testing; fault injection; synthetic monitoring; production observability. | 
| **Scale, duration & emergent errors** | Behaviour changes with workload, duration or system scale. Resource leaks, nonlinear performance and multi-component interactions may only emerge after thousands/millions of operations. Training examples and feasible reasoning traces disproportionately represent much shorter executions. | Bounded resources; explicit budgets; simple interaction protocols; loose coupling/high cohesion; make scale assumptions explicit. | Load/stress/soak testing; profiling; leak detection; simulation; chaos testing; production telemetry. | 
| **Implicit semantic/invariant errors** | The missing relationship exists in *meaning* , not necessarily syntax or structure: “if X is true, Y must already have happened.” Such invariants may be unique to this system or organisation and therefore impossible to have learned during pretraining. | Make invariants explicit; executable specifications; examples/tests; assertions; domain types; documentation close to code; provide relevant domain context. | Property/invariant testing; acceptance tests; runtime assertions; monitoring; real-world feedback. |