{"slug": "full-text-search-still-works-it-just-doesnt-get-you-to-an-answer", "title": "Full-Text Search Still Works. It Just Doesn’t Get You to an Answer", "summary": "Manticore Search introduced Conversational Search, a feature that combines full-text, vector, and hybrid search with a language model to answer complex product queries, as demonstrated in its Manticore Apparel Shop demo built on the ConvApparel dataset of 82,524 products. The feature addresses the growing expectation for search systems to handle multi-part questions, a trend highlighted by Google's report that AI Mode surpassed one billion monthly users in May 2026.", "body_md": "Imagine a typical online shoe store.\n\nA shopper opens search and types:\n\n```\nI need black waterproof running shoes for daily runs on wet pavement. What would you recommend?\n```\n\nA few years ago, almost no one expected this from a search box. The query would have been shortened to something like:\n\n```\nblack waterproof running shoes\n```\n\nThen the shopper would open several product cards, compare descriptions, materials, intended use, and price, and make the decision alone.\n\nToday, people increasingly expect search itself to do part of that work. Not because full-text search has become worse. It still performs very well with exact names, SKUs, product codes, brands, and keywords. What has changed is what people expect to be able to ask a search system.\n\nFor example, Google reported in May 2026 that AI Mode had surpassed one billion monthly users. The company notes that people are asking longer, more complex questions that previously did not fit into conventional search. ([blog.google](https://blog.google/products-and-platforms/products/search/search-io-2026/)\n)\n\nThe same thing happens in an online store.\n\nA query such as:\n\n```\nI need black waterproof running shoes for daily runs on wet pavement.\n```\n\nonly looks like one sentence. For a search system, it contains several tasks.\n\nIt needs to extract constraints such as color and waterproofing; understand that this is about running rather than walking; account for price, size, and availability; find suitable products; and, if the user asks “which are better,” explain the differences.\n\nNo single algorithm solves all of that.\n\nFull-text search, vector search, filters, hybrid search, and a language model each solve different parts of the problem. It is far more effective to combine them than to choose between them.\n\nThat is exactly why Manticore has Conversational Search.\n\n## Word search, semantic search, and conversation are different tasks\n\nStart with a simple query:\n\n```\nNike Pegasus 41 black\n```\n\nHere, the system barely needs to interpret the user’s intent. Full-text search handles it directly.\n\nOr something even simpler:\n\n```\nSKU 123456\n```\n\nSemantic methods are not needed here.\n\nNow consider another example:\n\n```\nlight shoes for long summer walks\n```\n\nA product card may not contain the words “summer” or “long walks,” but it may include details such as “breathable material” or “lightweight construction.”\n\nThis is where vector search becomes useful.\n\nReal queries often fall between these extremes:\n\n```\nblack Gore-Tex shoes for everyday running\n```\n\nSome parameters — `black`\n\nand `Gore-Tex`\n\n— need to be preserved. `Everyday running`\n\ndescribes the user’s intent rather than an exact attribute.\n\nFor such cases, Manticore uses hybrid search, combining full-text and vector search through result ranking.\n\nBut even hybrid search returns only a list of results.\n\nAt that point, search considers its job done. The user usually does not.\n\nIt does not answer questions such as:\n\n```\nWhich of these models are better suited to rain?\n```\n\nAnd it certainly does not handle a follow-up such as:\n\n```\nWhich of those cost less than $120?\n```\n\nThat is a conversation. It needs another layer.\n\n## What we built\n\nTo test this in practice, we used [ConvApparel](https://arxiv.org/abs/2602.16938)\n, a dataset of conversations about choosing apparel. After cleanup, it contained 82,524 products: footwear, pants, tops, and outerwear. Each product has a description, category, images, and attributes. We built Manticore Apparel Shop on this data.\n\nFor example, you can type:\n\n```\nI need black waterproof running shoes for jogging\n```\n\nThe system first finds suitable products, then a language model generates an answer using them as context, while the interface shows the products themselves.\n\nTry the demo:[Manticore Apparel Shop]generates a random product and a query that should retrieve it, then demonstrates that the same query does retrieve that product through Manticore.\n\nIt is important to keep the connection between the answer and the data. If the system claims that a model is suitable for rain, the user should be able to open the product and verify the source of that claim.\n\nIn this approach, the language model does not replace search. It interprets its results.\n\n## How it works\n\nTwo main commands are used:\n\n```\nCREATE CHAT MODEL\n```\n\nand\n\n```\nCALL CHAT(...)\n```\n\nFirst, you create a Conversational Search model and set the rules it follows.\n\n```\nCREATE CHAT MODEL assistant (\n    model='openrouter:google/gemma-4-26b-a4b-it',\n    timeout=60,\n    retrieval_limit=5,\n    max_document_length=3000,\n    custom_prompt='You are a context-only shopping assistant.\n\nAnswer using only the provided context.\nDo not use outside knowledge or unsupported assumptions.\n\nRecommend only products supported by the retrieved context.\nFor every recommended product, briefly explain why it matches the request.\n\nEnd every recommendation with the corresponding\ncontext source ID in the format [ref:<id>].\n\nIf none of the retrieved products support the request,\nsay that you do not have enough information.'\n);\n```\n\n`retrieval_limit`\n\ndetermines how many documents enter the context. `max_document_length`\n\nlimits the amount of text from each document.\n\nIf there is too little context, the model will not see the right products. If there is too much, latency and query cost increase. Like a person, a language model does not become smarter just because it has been given everything to read.\n\nYou can then run a query:\n\n```\nCALL CHAT(\n    'I need black waterproof running shoes for jogging',\n    'convapparel_products',\n    'assistant',\n    'demo-session-001',\n    'embedding_vector'\n);\n```\n\nYou can then continue the conversation:\n\n```\nCALL CHAT(\n    'Which of these are better for daily use?',\n    'convapparel_products',\n    'assistant',\n    'demo-session-001',\n    'embedding_vector'\n);\n```\n\nThe system uses conversation history, so the user does not need to repeat the context.\n\n## Through the HTTP API\n\nConversational Search is also available through the JSON API:\n\n```\n{\n  \"chat\": {\n    \"query\": \"I need black waterproof running shoes for jogging\",\n    \"table\": \"convapparel_products\",\n    \"model_name\": \"assistant\",\n    \"conversation_uuid\": \"demo-session-001\",\n    \"vector_field\": \"embedding_vector\"\n  }\n}\n```\n\nThe request is sent to `/search`\n\n.\n\n## What the system returns\n\nThe response contains:\n\n`conversation_uuid`\n\n`user_query`\n\n`search_query`\n\n— the search query generated by the system`response`\n\n`sources`\n\n`search_query`\n\nis particularly important.\n\nIf the user writes:\n\n```\nWhich of these would work better in rain?\n```\n\nOn its own, this query makes no sense without context. The system therefore forms a complete search query using the conversation history.\n\nThis also simplifies debugging: you can trace the entire chain from query to answer.\n\n## How retrieval works\n\nConversational Search uses vector search over an embedding field. The flow looks like this:\n\n```\nuser question\n→ conversation history\n→ search query\n→ vector search\n→ retrieved documents\n→ language model\n→ answer and sources\n```\n\n## Where search ends and conversation begins\n\nFull-text search works well for exact queries. Vector search works with semantic ones. Hybrid search works with their combination. Conversational Search is needed when the result must be explained, compared, or refined.\n\n## Quality\n\nWe tested this on our [conversational search quality benchmark](https://github.com/manticoresoftware/conversational-search-quality-becnhmark)\n, using 200 deterministic shopping queries from ConvApparel. The benchmark evaluates the product IDs returned as sources, rather than the wording of the generated answer.\n\nIn the current run, Manticore scored **0.3650 Hit@3**, **0.4250 Hit@5**, **0.5250 Hit@10**, and **0.2790 MRR** — the best result on each of those metrics among the engines tested. The full repository includes the dataset-building rules, engine configuration, smoke tasks, and raw results.\n\n## Conclusion\n\nSearch in modern systems is not one algorithm but several layers:\n\n- full-text search\n- vector search\n- hybrid search\n- Conversational Search\n\nEach solves its own task. A language model does not replace search; it works on top of it.\n\nWithout good search, you do not get a smart assistant; you get a very talkative consultant that barely knows its own catalog.\n\nWant to test the approach in practice?Try[Manticore Apparel Shop]: choose a random product, ask a question based on it, and confirm that the same product is found and suggested in the answer.\n\nIf you want to run it locally or explore the code, check the [GitHub repository](https://github.com/manticoresoftware/demo-conversational-search)\n.", "url": "https://wpnews.pro/news/full-text-search-still-works-it-just-doesnt-get-you-to-an-answer", "canonical_source": "https://manticoresearch.com/blog/conversational-search/", "published_at": "2026-08-20 00:00:00+00:00", "updated_at": "2026-08-20 16:13:47.472350+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "natural-language-processing", "ai-products", "ai-tools"], "entities": ["Manticore Search", "Google", "ConvApparel", "Manticore Apparel Shop"], "alternates": {"html": "https://wpnews.pro/news/full-text-search-still-works-it-just-doesnt-get-you-to-an-answer", "markdown": "https://wpnews.pro/news/full-text-search-still-works-it-just-doesnt-get-you-to-an-answer.md", "text": "https://wpnews.pro/news/full-text-search-still-works-it-just-doesnt-get-you-to-an-answer.txt", "jsonld": "https://wpnews.pro/news/full-text-search-still-works-it-just-doesnt-get-you-to-an-answer.jsonld"}}