You gave the LLM the answer. So why is it writing it out from scratch?
Send a model a support ticket and four labels, and ask which one fits. It writes back {"label": "billing"}, one token at a time. The word "billing" was in your prompt the whole time.
Extraction is the same. The invoice number is in the PDF, and the model types it back to you. Usually it types it right.
An LLM reads your prompt in one pass, then writes the answer one token at a time, each token another trip through the model. That's also why output costs more: on Anthropic's price list, Sonnet 5.5 is $2 per million input tokens and $10 per million output.
For a short label the cost isn't the real problem. The problem is that a writer can write anything: "Billing", "billing_dispute", "Sure! The label is billing." So you add a parser, a retry, a schema. JSON mode helps, but it's "a format constraint, not a correctness guarantee," as one guide puts it. A pointer works differently. It reads the input once and scores what's already there: each of your labels, or each possible span of the document. The answer can only be something from the input.
This isn't new. Pointer Networks (2015) picked elements of the input as output. SQuAD (2016) defined every answer as "a segment of text from the corresponding reading passage." BERT answered it by scoring each token as a possible start or end of the answer. Named entity recognition still returns start and end offsets today.
ScaleDown launched four small task models on October 13 and calls them Constrained Decoding Models. Their partner post with LangChain sums up the idea: "a CDM is trained for one task and only returns answers that task allows, like a label from your list or spans of the input."
Going by ScaleDown's docs:
top_label plus scores that sum to 1.0. start/ end offsets. So the idea holds for two of the four. One naming note: "constrained decoding" usually means masking a normal LLM's tokens so it can't break your schema (Outlines, XGrammar). Here it's a product name.
Speed is nice. The bigger deal is that a pointed answer comes with proof. If Extract says the invoice number sits at characters 8 to 17, you slice your own string and check. A written answer with two digits swapped looks exactly like a correct one.
Not instead of your main LLM. Around it. Small models do the routine jobs and the general model does what actually needs writing.
The loop check comes from LangChain's example: stop the run when Classify scores "looping" above 0.8. That only works if 0.8 means something on your data. ScaleDown doesn't publish calibration numbers, and Classify "always returns a winner - even if the model is uncertain," so add an other label and measure on a few hundred of your own examples.
TypeSafe's Jev, OpenAI's Decisions API and Cloudflare's Clef all answer typed questions with probabilities. That's the Classify half. None of them document pointing at spans.
Want the deeper dive? Check out the detailed blog with interactive examples here: smiit.in/writing/answer-is-in-the-input Which of your LLM calls are secretly multiple choice? Tell me in the comments.