We wanted to know when LLM is guessing versus when it actually knows the answer.
LLM models expose logprobs - after every word they generate, you can request the top alternative tokens and their probabilities. Low entropy means the model was certain, high means it was guessing.
I tested two models using 0.0 temperature and 3 prompts: "what is 2+2", "the opposite of hot is", "once upon a time." The results:
For the math prompt, Nano gave "4" at 87% and "2" at 13%. For the story prompt, it echoed the input back verbatim, so entropy collapsed because it was copying input, not generating. Mini generated the story with five competing candidates, none above 31%. That's honest uncertainty. Both LLM models are cheap and still popular, but Nano is miscalibrated and not recommended for agentic autonomous decision making.