Design AI Confidence as a Recovery Path, Not a Percentage

A developer proposes a framework for designing AI confidence displays that prioritize actionable recovery paths over numerical percentages. The approach classifies actions by consequence level and provides evidence, uncertainty gaps, and bounded authority rather than a single confidence score. The framework is illustrated with examples from the MonkeyCode project, though it is not a study of that product's current interface.

An interface that says “92% confident” appears precise while leaving the user with the real question: what should I do differently because of that number? Confidence should change the interaction, not decorate it. Classify the proposed action before choosing a pattern: | Consequence | Example | Interface response | |---|---|---| | reversible, private | rename a local draft | show preview and undo | | reversible, shared | open a pull request | show evidence, diff, and reviewer | | costly to reverse | migrate data | require plan, validation, backup, approval | | irreversible or external | delete production data | keep deterministic guardrails; never rely on confidence | A model score cannot authorize a high-consequence action. It can help prioritize review only when calibrated for the same task distribution and clearly explained. Proposed change Update the cache TTL from 5 to 30 minutes. Evidence • requirement.md, lines 18–22 • config/cache.ts, current value Uncertainty The requirement does not specify the staging environment. Actions Edit proposal Ask a question Approve for staging Recovery Creates a reviewed pull request; does not deploy. This communicates provenance, a specific gap, bounded authority, and recovery. It remains useful without a percentage. If a score is necessary, label the event it predicts: “In validation, 81% of suggestions above this threshold required no factual correction.” Link the dataset date, sample size, task definition, threshold, and known failure groups. Never translate model probability directly into “chance this is true.” Give participants realistic tasks with low and high consequences. Vary evidence quality and missing information. Observe whether they notice unsupported claims, choose an appropriate action, understand what approval will do, and recover after a deliberately wrong suggestion. Include keyboard and screen-reader use; uncertainty cannot be encoded by color or a tooltip alone. The public MonkeyCode repository https://github.com/chaitin/MonkeyCode describes AI tasks, project requirements, managed development environments, and team collaboration. Evidence-and-recovery patterns are relevant to that product category, but this is a proposed framework, not a study of MonkeyCode's current interface. Disclosure: I contribute to the MonkeyCode project. The contextual description is public; no user-research result about MonkeyCode is claimed. Useful confidence design helps a person inspect, constrain, correct, and recover. If the number does none of those things, remove it.