{"slug": "don-t-classify-hallucinate", "title": "Don't classify. Hallucinate!", "summary": "OpenAI's GPT-5.4-mini can classify e-commerce queries more cheaply by hallucinating fake categories and matching them to real ones via embeddings, according to a developer who shared the technique. The method avoids sending large lists of legal values, reducing token costs and bypassing structured output limits. The approach uses MiniLM embeddings to resolve invented classifications like 'Furniture / Living Room / Tables / Coffee' into the closest real Wayfair category.", "body_md": "Using LLMs to classify products, search queries, etc is by now boring. Yet it can still be difficult to constrains the LLM’s output to the legal vocabulary of brands, colors, categories, etc your system allows.\n\nIn the Wayfair WANDS e-commerce dataset, for example, you want to classify a query like “wood coffee table” into its most appropriate category. Of which there are hundreds:\n\n```\nFurniture / Office Furniture / Desks\nFurniture / Living Room Furniture / Coffee Tables & End Tables / Coffee Tables\nFurniture / Living Room Furniture / Coffee Tables & End Tables / End & Side Tables\nDécor & Pillows / Decorative Pillows & Blankets / Throw Pillows\nFurniture / Bedroom Furniture / Dressers & Chests\n```\n\nThe classic way to implement this would be with structured outputs. You tell your provide it must constrain its outputs to a list of legal values. In Pydantic, you create a giant literal of legal output values:\n\n``` python\nfrom typing import Literal\nfrom pydantic import BaseModel, Field\n\nFullyQualifiedClassifications = Literal[\n 'Furniture / Bedroom Furniture / Beds & Headboards / Beds',\n 'Furniture / Living Room Furniture / Chairs & Seating / Accent Chairs',\n 'Rugs / Area Rugs',\n  ...\n  # times 500\n]\n\nclass QueryClassification(BaseModel):\n    \"\"\"\n    Structured representation of a search query for furniture e-commerce.\n    Inherits keywords from the base Query model and adds category and sub-category.\n    \"\"\"\n    classifications: list[FullyQualifiedClassifications] = Field(\n        description=\"A possible classification for the product.\"\n    )\n\nresponse = client.responses.parse(\n    model=\"gpt-5.4-mini\",\n    input=\"Classify the query: brown coffee table\",\n    text_format=QueryClassification,\n)\n\nprint(response.output_parsed.message)\n# Outputs: Furniture / Living Room Furniture / Coffee Tables & End Tables / Coffee Tables\n```\n\nThis works. But there’s a way to do this a lot cheaper with small / dumb models at scale. Not to mention, there’s an [upper limit you can send](https://developers.openai.com/api/docs/guides/structured-outputs)\n\nLuckily, there’s an easy pattern that makes LLM classification pretty seamless.\n\nJust ask a dumb LLM to invent plausible, fake classifications for your query:\n\n```\nhallucination_prompt = f\"\"\"\nYour task is to create novel, never seen before, furniture, home goods, or hardware classification that best fit a search query. \n\nProduct classifications might look like:\n\nFurniture / Living Room Furniture / Coffee Tables & End Tables / Coffee Tables\nDécor & Pillows / Decorative Pillows & Blankets / Throw Pillows\nFurniture / Bedroom Furniture / Dressers & Chests\nKitchen & Tabletop / Kitchen Organization / Food Storage & Canisters\nSchool Furniture and Supplies / School Furniture / School Chairs & Seating / Stackable Chairs\nBaby & Kids / Toddler & Kids Bedroom Furniture / Kids Beds\n\nHere's the query to generate classifications for:\n\nbrown coffee table\n```\n\nNow we’re not sending the list of legal classifications. We’re instead, asking the LLM to make stuff up:\n\n```\nresponse = client.responses.parse(\n    model=\"gpt-5.4-mini\",\n    input=hallucination_prompt,\n    text_format=list[str],\n)\n```\n\nIt’ll then make up some BS that doesn’t actually exist in your real taxonomy like:\n\n```\nFurniture / Living Room / Tables / Coffee\n```\n\nWell that’s not very helpful.\n\nActually it’s extremely helpful. You can now resolve that into the real vocabulary.\n\nIt’s very cheap to build an in-memory set of embeddings of the REAL classifications. As I’ve done in [this notebook](https://colab.research.google.com/drive/1ljk72SBRuqWIijuEusCnDbhG1WAfZFcC#scrollTo=RZ-hEr-CSr9T) and [this utility](https://github.com/softwaredoug/cheat-at-search/blob/main/cheat_at_search/enrich/vocabulary.py).\n\nIn the notebook, I compute a MiniLM embedding of every real Wayfair classification. I compute the embedding of the fake, hypothetical embedding from the LLM. I then dot product the fake embedding into the real ones to find the most similar. Producing:\n\n```\nFurniture / Living Room Furniture / Coffee Tables & End Tables / Coffee Tables\n```\n\nYou can give these hallucination tasks to dumb / cheap LLMs. And you don’t need to ship the schema over to the LLM every time.\n\n### Upcoming events: Vectors Week\n\nJoin me for Vectors Week, a series of events about vector retrieval, hybrid search, and building your own vector database.", "url": "https://wpnews.pro/news/don-t-classify-hallucinate", "canonical_source": "http://softwaredoug.com/blog/2026/08/10/hypothetical-classifications.html", "published_at": "2026-08-10 00:00:00+00:00", "updated_at": "2026-08-10 21:07:37.841045+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "natural-language-processing", "ai-tools"], "entities": ["OpenAI", "GPT-5.4-mini", "Wayfair", "MiniLM", "Pydantic"], "alternates": {"html": "https://wpnews.pro/news/don-t-classify-hallucinate", "markdown": "https://wpnews.pro/news/don-t-classify-hallucinate.md", "text": "https://wpnews.pro/news/don-t-classify-hallucinate.txt", "jsonld": "https://wpnews.pro/news/don-t-classify-hallucinate.jsonld"}}