Show HN: Spanda – Sub-microsecond LLM epistemic uncertainty in Rust

Detect LLM hallucinations and quantify uncertainty in microseconds without secondary NLI cross-encoders.

Traditional epistemic uncertainty estimation in LLMs relies on Semantic Entropy (SE) (Kuhn et al., 2023; Farquhar et al., Nature 2024). While effective, Semantic Entropy requires clustering

This introduces two severe production bottlenecks:

Quadratic Cost:$\binom{K}{2}$ forward passes per query (45 neural evaluations for$K=10$ ). 2. Serving Latency: Adds $\sim$90 ms of GPU overhead per inference call, making it unusable for high-throughput production serving.

Spanda introduces Exact-Match Normalized Entropy ( $R_{sc}$): a zero-parameter, zero-GPU metric that computes uncertainty directly over deterministic lexical clusters.

Across empirical evaluations spanning two orders of magnitude (1.5B to 120B parameters), Spanda matches or exceeds neural Semantic Entropy on structured reasoning while operating ~90,000$\times$ faster (

As model capacity increases from 1.5B to 27B parameters, internal reasoning coherence causes correct predictions to naturally converge to identical lexical sequences. On mathematical reasoning (GSM8K), exact-match AUROC scales monotonically:

At 7B+ parameters, Spanda achieves the exact same discriminative power as heavy DeBERTa-v3 NLI cross-encoders, rendering the neural clustering step redundant for reasoning.

At the 120B frontier scale on ungrounded factual recall (TriviaQA), the model exhibits Confident Mode Collapse: its parametric memory and RLHF tuning cause it to hallucinate the exact same incorrect answer identically across all inverted AUROC of 0.091 (

Critical Safety Implication: Any system using self-consistency or agreement as a proxy for truth will be systematically deceived by frontier models on ungrounded factual recall. External grounding (RAG) is mandatory in this regime.

⚠️

Empirical audit conducted across 50,000 evaluation iterations and 300 concurrent live HTTP reverse-proxy round-trips:

Given

The Normalized Shannon Entropy is: $$H_{\text{norm}} = \begin{cases} 0 & \text{if } n = 1 \ \displaystyle\frac{-\sum_{i=1}^n w_i \ln w_i}{\ln K} & \text{if } n > 1 \end{cases}$$

The combined Spanda Risk Score ( $R_{sc}$) balances entropy dispersion with modal dominance (

$R_{sc} = 0$ : Complete consensus (model is confident). #

$R_{sc} \to 1$ : Maximum epistemic divergence (model is guessing / hallucinating).

Spanda is lightweight and requires zero third-party dependencies (pure Python standard library).

pip install spnda

(Package name on PyPI is spnda; module is imported in Python as import spanda)

Or install from source:

git clone https://github.com/Adarshent/Spnda.git
cd Spnda
pip install -e .

Wrap any standard OpenAI, Groq, Ollama, or OpenAI-compatible client with transparent multi-path epistemic uncertainty quantification (

import spanda
from openai import OpenAI

client = spanda.wrap(OpenAI(), k=3, threshold=0.35, block=False)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "What is 17 * 19?"}]
)

print(response.spanda.rsc)             # 0.0000 (Unanimous consensus)
print(response.spanda.is_safe)         # True
print(response.spanda.decision)        # 'FAST_PASS_CONSISTENT'
print(response.spanda.latency_us)      # 0.7 µs!
print(response.choices[0].message.content) # Dominant consensus answer

If block=True is passed and the model hallucinates or diverges, spanda.wrap raises a SpandaUncertaintyError before invalid data reaches your users.

For production microservices and non-Python languages (TypeScript, Go, Rust, Ruby, curl), run the standalone compiled Rust gateway:

spnda serve --upstream http://localhost:11434/v1 --port 8080 --block --k 3

spnda bench --iterations 200000

spnda eval "42" "42.0" "42"

Any application in any language can simply point base_url="http://localhost:8080/v1" to receive automatic sub-microsecond epistemic verification, Prometheus /metrics, and headers:

• X-Spanda-Rsc: 0.0000
• X-Spanda-State: CONSISTENT
• X-Spanda-Decision: FAST_PASS_CONSISTENT
• X-Spanda-Latency-Us: 0.7
• X-Spanda-Attractor: false

from spanda import compute_rsc, detect_hallucination, batch_compute_rsc

samples = ["Paris", "paris.", "Paris", "Paris", "Paris"]
res = compute_rsc(samples)
print(f"R_sc Score: {res['rsc']}")             # 0.0 (High confidence)
print(f"Dominant Answer: {res['dominant_answer']}") # 'Paris'

guard = detect_hallucination(["42", "42", "24", "17", "99"], threshold=0.35)
if guard["is_uncertain"]:
    print(f"🚨 Hallucination Warning (R_sc = {guard['rsc']}). Routing to RAG / Review.")
else:
    print(f"✅ Safe output: {guard['dominant_answer']}")

batch = [
    ["Answer A", "Answer A", "Answer A"],
    ["Choice 1", "Choice 2", "Choice 3"]
]
for r in batch_compute_rsc(batch):
    print(r["rsc"], r["dominant_answer"])

For mission-critical production pipelines, Spanda provides a 2-Tier Cascaded Guardrail that combines sub-millisecond consensus filtering with context grounding and tool-call safety:

from spanda import CascadedGuardrail

guard = CascadedGuardrail(
    uncertainty_threshold=0.3,
    grounding_threshold=0.15
)

rag_context = "Documentation: The production cluster runs in us-east-1."
unanimous_hallucination = ["eu-west-3 Paris", "eu-west-3 Paris", "eu-west-3 Paris"]

receipt = guard.evaluate(unanimous_hallucination, context=rag_context)
print(receipt.decision)       # 'MODE_COLLAPSE_RISK'
print(receipt.is_safe)        # False (Unanimous agreement, but 0% grounded in source!)
print(receipt.tier_executed)  # Tier 2
print(receipt.latency_ms)     # < 0.05 ms

tool_calls = [
    {"command": "rm -rf /var/cache"},
    {"command": "rm -rf /var/log"},  # Conflict detected across parallel paths!
]
agent_receipt = guard.evaluate_tool_calls(tool_calls)
print(agent_receipt.decision) # 'TOOL_ARG_MISMATCH' (Execution blocked!)

import json
print(json.dumps(receipt.to_dict(), indent=2))

Spanda connects into modern enterprise LLM pipelines with zero external dependencies:

from spanda.integrations.langchain import SpandaStringEvaluator

evaluator = SpandaStringEvaluator(uncertainty_threshold=0.35)
result = evaluator.evaluate_strings(
    prediction=["Paris", "Paris", "Paris", "Paris"],
    context="Paris is the capital of France."
)
print(result["value"])  # 'PASS' (Score: 0.0)

from spanda.integrations.llamaindex import SpandaRAGGuardrail

guard = SpandaRAGGuardrail()
receipt = guard.validate_response(
    samples=["Result A", "Result A", "Result A"],
    context_str="Retrieved node knowledge..."
)
print(receipt.is_safe)  # True

import litellm
from spanda.integrations.litellm import SpandaLiteLLMGuardrail

litellm.callbacks = [SpandaLiteLLMGuardrail(threshold=0.35, block_mode=False)]

Run the standalone compiled Rust gateway in Docker or Kubernetes:

docker run -d -p 8080:8080 \
  -e SPANDA_UPSTREAM=https://api.openai.com/v1 \
  -e SPANDA_THRESHOLD=0.35 \
  brhmn/spnda-gateway

• GET /metrics : Standard Prometheus format for Grafana (spanda_requests_total ,spanda_evaluations_total ,spanda_mode_collapses_total ,spanda_eval_latency_avg_us ).
• GET /healthz : Kubernetes liveness probe.
• GET /readyz : Kubernetes readiness probe.
• Structured JSON logging: Every transaction emits a machine-parseable log line to stdout for Datadog / CloudWatch / Splunk.

Operational Scope: Spanda is engineered for structured reasoning, math, code, agent tool-call arguments, SQL, and canonical factual RAG extraction where 90ms GPU cross-encoders are an unacceptable bottleneck. It is not designed for open-ended, free-form creative prose (e.g., essays or poetry), where synonymous phrasing is naturally diverse and requires heavy neural NLI.

⚠️

Run the test suite:

python3 -m unittest discover tests

If you use Spanda in your research or production systems, please cite:

@article{nayak2026spanda,
  title={Spanda: Zero-Cost Lexical Entropy Matches Neural Semantic Uncertainty---Until Frontier Models Break It},
  author={Nayak, Bhupen},
  journal={arXiv preprint},
  year={2026},
  doi={10.5281/zenodo.22233648},
  url={https://doi.org/10.5281/zenodo.22233648}
}

Spanda adopts a developer-friendly dual-licensing model:

• Python SDK & Integrations (spanda) : Permissive**MIT License** . Free for all developers, commercial and open-source applications, with zero dependency friction.
• Compiled Rust Core Engine & Gateway (spnda) :Business Source License 1.1 (BSL 1.1) . Free for developers, research, and internal production infrastructure. Prohibits offering Spanda as a competing commercial third-party managed service without an enterprise license from BRHMN Labs Private Limited. Automatically converts to Apache 2.0 on January 1, 2030.