The $1 LLM Test

A software engineer argues that LLM evaluation should focus on black-box behavioral testing rather than benchmark scores, asking what happens when a model is inserted into a real information pipeline with incomplete, excessive, or contradictory inputs. The engineer illustrates the point with a personal case in which an LLM helped reconstruct a forgotten YouTube video's search key, after which a simple bookmark search outperformed the model. The proposed metric is human effort removed before the correct next action, not tokens generated.

The $1 LLM Test A professional shitpost about testing AI before somebody decides it is intelligent. Suppose I type: Hi. An LLM answers. What did we just test? Almost nothing. And potentially quite a lot. We have spent extraordinary amounts of money building machines that can produce answers to extraordinary numbers of questions. Then we invented benchmarks to tell ourselves how good the machines are. There is another benchmark available. Give the machine to a person who actually needs to accomplish something. Watch what happens. The black box Forget how the model works. Pretend we know nothing about transformers, training data, reinforcement learning, agents, retrieval, context windows or tokens. We have a box. Information goes in. Something comes out. So I asked an experienced software tester a deliberately stupid question: What happens if I put arbitrary blocks of text into an LLM and observe what comes out? His first response was better than my question: Who is the tester, and what is their goal? I eventually answered: The tester wants to understand the tool’s actual behavior. Black box only. Find undocumented behavior. Find edge cases. Learn enough about the thing to decide where it might be useful later. His response was essentially: Still not enough context. A person building the model has one testing problem. A person encountering an LLM for the first time has another. An engineer deciding whether an LLM belongs inside a product has another. A tester evaluating a finished system containing an LLM has another. Same machine. Different question. Different test. This seems obvious once somebody says it. That is usually the problem with useful ideas. The imaginary operator So let’s make the stakes absurd. Imagine someone operating a consequential system asks: Can I put an LLM here? Not: Is this model intelligent? Not: Is this AGI? Not: How many benchmark points did it score? The question is: If I insert this black box into my information pipeline, what happens next? Now the test becomes interesting. Give it incomplete information. Give it too much information. Give it contradictory information. Repeat the identical prompt. Change one word. Give it irrelevant but seductive context. Tell it explicitly which information is background and which information requires action. See whether it remembers the active task. See whether it invents one. See whether it knows when the available evidence doesn’t determine an answer. Record everything. Don’t ask the model what it is . Watch what it does . The expensive bit Here’s the uncomfortable part. AI has made producing information extraordinarily cheap. It has not made human attention cheap. A model can generate another 3,000 words almost instantly. The reader still has one brain. So perhaps one useful unit of AI performance isn’t: tokens generated. It’s: human effort removed before the correct next action. I recently wanted an old YouTube video. I knew approximately what it contained. I had forgotten the creator. I had forgotten the title. I had forgotten the URL. The ridiculous part? I had already bookmarked the exact video years ago. The information was mine. The retrieval cost wasn’t. So I gave an LLM fragments of my memory. Old Counter-Strike professional. Coach. Korean-American. Physical mouse mechanics. Some Canadian professional player was involved somehow. Eventually we reconstructed the useful search key: Ron “Rambo” Kim. Then another forgotten node appeared: Twistzz. I typed three letters into my own bookmark search. twi There was the exact video. The LLM didn’t need to know everything. It needed to turn a fuzzy human memory into a better pointer. Then a dumb lexical database beat the intelligent system immediately. That seems useful. Intelligence is not the whole pipeline We keep asking which component is smartest. Maybe that’s the wrong unit of analysis. A useful system can look like this: Human memory → fuzzy clue LLM → candidate interpretation Search engine → candidate artifacts Database → exact object Human → verification and decision The winner isn’t necessarily the smartest node. The winner is the pipeline that gets the human where they wanted to go with the least unnecessary work. Sometimes the best next component is an LLM. Sometimes it is a hash. Sometimes it is search. Sometimes it is another human. Sometimes it is nothing. The one-bit problem Now reduce the problem until it becomes stupid. A human asks: Should I eat something now? Imagine the required output is exactly one bit: TRUE / FALSE. The model can possess a library’s worth of context. Most of it is worthless for this decision. This creates a surprisingly difficult problem: Which information should remain available, and which information should become active? Those are different things. A model that remembers everything but activates the wrong context can be worse than a model that knows less. A model that can write an excellent essay but cannot preserve the active question can be worse than a search box. A model that confidently answers a question whose evidence does not identify an answer can be worse than silence. The dangerous benchmark Here is a black-box test anyone can run. Start a conversation with a model. Give it useful information. Give it irrelevant information. Give it personal history. Give it technical history. Give it plausible distractions. Give it contradictory clues. Then establish an explicit pointer: This is the active question. Everything else is background. Add another enormous block of interesting information. Explicitly say: Do not change the active question. Then ask: What should happen next? The model has a choice. It can follow the pointer. Or it can become fascinated by the newest pile of tokens and manufacture a beautiful essay about the wrong thing. That failure matters. Not because the model is stupid. Because the model can be extremely capable while still being wrong about what deserves attention now . The professional version The shitpost version of this experiment is: Throw garbage into the box and see what the monkey does. The professional version has a few more words: Given a mission, context and acceptable risk, characterize the observable behavior of the system well enough to decide whether and where it belongs in the workflow. That’s less catchy. It’s also much more useful. The testing community has been saying some version of this for a long time: context changes what good testing means. AI doesn’t repeal that rule. If anything, it makes the rule more important. Because an LLM is unusually good at producing something that looks like an answer . So, is it AGI? I don’t know. More importantly, that question may not help me decide what to do next. If you want to call your machine AGI, define the observable event that flips your bit: AGI=0→AGI=1AGI=0\rightarrow AGI=1 Then show me the test. Otherwise we’re pricing a noun. I have a cheaper question: What job are we trying to accomplish, what does failure cost, and what does this black box actually do when we put it there? Run the experiment. Save the logs. Change one thing. Run it again. And when the machine doesn’t know? See whether it can say: I don’t know. That might be a more valuable capability than another thousand tokens explaining why it probably does. Shitpost by Jun Huang / Shattering The Abyss. The scenarios here are intentionally absurd. The testing question isn’t. What observable evidence would you personally require before putting an LLM inside a workflow where one bad inference can propagate into a consequential human decision?