Ei-Core just said no to a retention estimate — and that refusal

Ei-Core's AI model refused to provide a retention estimate for a cohort with only 23 engagement events, citing an internal threshold of 150 events for reliable projection. The refusal, triggered by an uncertainty quantifier during safety tuning, was praised by a VP as preferable to inventing a number. The incident highlights the value of abstention mechanisms in AI tools, though the threshold requires validation against production data.

Ei-Core just said no to a retention estimate — and that refusal The model didn't hallucinate. Didn't hedge. Didn't give a confidence interval. It flat-out refused: I cannot provide a retention estimate for this cohort. Insufficient behavioral signal density n=23 events falls below the minimum threshold n=150 for reliable projection. Recommend extending observation window or enriching with external firmographic data. First reaction: panic. Client-facing tool just... declined. That's not in the spec. Then I checked the logs. The refusal trace showed the uncertainty quantifier firing at step 3 of the reasoning chain — before any generation attempt. The model had internally computed the effective sample size, compared it against the calibrated threshold we set during safety tuning, and triggered the abstention gate. From the refusal trace if effective n < MIN SAMPLE THRESHOLD: raise AbstentionError reason="insufficient signal", metadata={"effective n": effective n, "threshold": MIN SAMPLE THRESHOLD} We'd built this gate six months ago during red-teaming. Forgot it existed until it saved us. The cohort genuinely had sparse data — only 23 meaningful engagement events across 47 accounts. Any number the model would've generated would've been fiction dressed as insight. The refusal was the correct answer. Changed the demo narrative on the spot. Showed the VP the refusal, explained the threshold, walked through what data would unlock a real estimate. She thanked us for not inventing a number. Two things I'm still chewing on: 1. The threshold 150 came from a synthetic benchmark, not production drift. Need to validate it against actual forecast error curves. 2. The error message leaks internal config MIN SAMPLE THRESHOLD . Should wrap that in a user-facing code like ERR INSUFFICIENT SIGNAL and keep the numeric detail in structured logs only. Also wondering if we should surface a "what would it take to answer" suggestion engine — the trace already knows which features are missing. Could auto-generate a data-collection checklist for the client. Anyone else hit a moment where a refusal felt more like a feature than a bug? Curious how you're handling abstention UX in production. Next That GPT-4o hallucination that invented a whole asyncpg API → /en/threads/7206/ these AI tool field notes https://tanyan888.com/ , with plenty of directly applicable cases.