{"slug": "full-text-search-is-now-generally-available-in-pinecone-database", "title": "Full-Text Search is Now Generally Available In Pinecone Database", "summary": "Pinecone Systems announced that full-text search is now generally available in Pinecone Database, adding BM25 keyword ranking and text-match filters to the same index as dense vectors, enabling exact matches for identifiers like SKUs and error codes that embeddings miss. The feature, validated in private preview with user partners, eliminates the need for separate node-based search systems, with usage-based capacity metered in read and write units.", "body_md": "If you run search, recommendations, RAG, or agents, the queries hitting your retrieval system come in more than one kind. Semantic search was built for when you need to search by meaning. The embedding captures what the user meant, and a question finds the right data even when its wording differs from the text.\n\nIdentifiers and exact phrases are a different kind of query. SKUs, part numbers, error codes, case numbers, and order IDs are literal strings. The literal string is the query, and embeddings rarely match literal strings. In a best case scenario, an embedding model splits PROD-001 into subword tokens (if the token was present in the embedding model's training data) and places it beside every other SKU. So, a search for the part number a customer provided, PROD-001 for this example, returns PROD-002 and PROD-003 alongside the correct part. The result is confidently close when the user needs exactly correct. Nothing throws an error. The agent answers from the wrong documentation, and nothing in the response indicates that there's a problem at all.\n\nMatching exact words has usually meant running a node-based search system. These engines are built for the job, and they are good at matching exact words, but running one is its own project that your team owns: nodes to size, capacity to plan as the corpus grows, shards, memory, merge cycles to tune, and version upgrades to run. The engine is good at the job, but keeping it running requires your team's time and effort.\n\nAfter months of private preview validation with our user partners, today, we're announcing that **full-text search is generally available in Pinecone Database**. It adds BM25 keyword ranking and text-match filters to the same index as your dense vectors, so agents get the exact order number or error code embeddings miss, with no separate system to tune and maintain.\n\n## **TL;DR**\n\nIf your search, recommendation, RAG, or agent workload mixes questions with identifiers and exact phrases, Full-Text Search gives you:\n\n- **Exactly correct instead of confidently close** on the queries embeddings get wrong: SKUs, error codes, case numbers, order IDs, and exact quoted phrases.\n- **BM25 keyword ranking** across multiple text fields per index, with Lucene query syntax, fuzzy matching, and tokenization and stemming in 18 languages.\n- **Text-match filters** that restrict the results of a semantic search to the documents matching the filter. One query both retrieves based on the identifier and ranks the rest by meaning.\n- **One index and one schema** for text fields, dense vectors, and sparse vectors, queried through the Documents API, with no cluster to tune, patch, or upgrade.\n- **Usage-based capacity** by default, metered in the same read units and write units as vector indexes.[Dedicated Read Nodes](/blog/dedicated-read-nodes-ga/) are available for provisioned read capacity, and[BYOC](/product/bring-your-own-cloud/) for running inside your own cloud.\n\n## **The queries vector embeddings get wrong**\n\nSemantic search works on the queries it was designed for, where meaning is the signal. A literal string is a different test, and a vector-only index fails it the same way every time.\n\n- Lookalikes rank as well as the right record. Every SKU with the same pattern shares most of its subword tokens with PROD-001, and the ranking has no way to tell which one the customer meant.\n- The failure is silent. A vector search always returns its nearest neighbors. There is no empty result to check for and no error to catch. A wrong answer looks the same as a right one until a customer notices.\n- The alternative is a node-based lexical engine. A lexical engine handles the exact word queries well, but it comes with nodes your team sizes, tunes, and upgrades.\n\nThose failures have a business cost:\n\n- Wrong actions that look right. An agent that retrieves the lookalike ships the wrong part or applies the wrong policy, and each one erodes users' trust.\n- An ongoing cost of ownership. Someone sizes the nodes, plans capacity as the corpus grows, and runs the upgrades, and that work continues for as long as the system is in production.\n- Slower delivery. Engineers spend their time maintaining search infrastructure instead of building the product.\n\nFull-text search is built for the queries where the answer depends on a literal string, and it runs in the same index as the semantic search that handles all of your other queries.\n\n## **Where exact-match precision matters**\n\n**Identifier-heavy retrieval.** SKUs, part numbers, error codes, and case numbers are where embedding models fail most visibly, and where full-text search shines. BM25 ranks the term itself, and a text-match filter returns nothing when no document carries the identifier. So, a miss reads as a miss instead of a lookalike.\n\n**Recommendations with hard constraints.** A customer who names a specific model or a compatible part number expects every recommendation to honor it, and embeddings alone can't do that (e.g., accessories for PROD-001 land next to the ones for PROD-002). A text-match filter pins the candidate set to items whose text carries the literal term, and the dense vector query ranks within it, so similarity never overrides a constraint the user stated. This is also true for content recommendations keyed to a ticker symbol, regulation number, or a streaming video ID.\n\n**RAG over enterprise documents.** A conceptual question needs semantic search. A query for a specific clause or an error code requires the exact term, and only lexical search reliably returns it. With full-text search and semantic search in the same index, the RAG system retrieves the clause the question depends on instead of a passage that resembles it.\n\n**Agent tool use over structured data.** Agents often need to look up data using a value from an earlier step, like an order number or a status string. Full-text search matches the value itself, so every step after that builds on the right data.\n\n## **How full-text search works in Pinecone**\n\nPinecone Database indexes created with the Documents API now run on a document schema. You declare text fields, dense vector fields, and sparse vector fields in one schema, and one index holds all of them.\n\nYou keep:\n\n- The same Pinecone APIs and SDKs\n- The same index lifecycle\n- The same usage-based capacity model, metered in storage, read units, and write units\n- The same serverless architecture, with no nodes for your team to maintain\n- The same deployment options, including Dedicated Read Nodes and Bring Your Own Cloud\n\nYou add:\n\n- BM25 keyword ranking across multiple text fields per index\n- Lucene query syntax, including boolean operators and phrase queries\n- Text-match filters that restrict any search, including a semantic one, to documents matching the filter\n- Fuzzy matching with the ~ operator for typo tolerance\n- Tokenization and stemming in 18 languages, plus language-agnostic n-gram tokenization for substring- and prefix-matching\n\n## **Search that finds what you mean and exactly what you say**\n\nAn agent that retrieves a lookalike record fails quietly. It ships the wrong part or cites the wrong clause, and the error shows up later as a support ticket or a lost customer. Putting a node-based lexical engine behind those queries trades the risk for a cluster your team owns.\n\nFull-text search, now generally available, closes both gaps inside the vector database you already run. The queries that embeddings get wrong now return the right records, and semantic search keeps handling everything else. One index and one schema sit behind both, with no cluster to tune and maintain.\n\n[Create an index](https://app.pinecone.io/) and start using full-text search today, or [contact us](/contact/?contact_form_inquiry_type=Sales) if you have questions about full-text search or want help sizing an instance.\n\n### **Learn more about full-text search**\n\n- **Read the docs.** The[full-text search guide](https://docs.pinecone.io/guides/search/full-text-search) covers schemas, analyzers, query syntax, current limits, and more.\n- **Run an example.**[Searching for Birds with Pinecone Full-Text Search](/learn/searching-for-birds-pinecone-full-text-search/) walks through schemas, filters, and query syntax, and the Bird Search demo combines full-text search with multimodal vector search over a Wikipedia bird dataset.\n- **Go deeper.**[Full Text Search: Architecture and Design](/blog/full-text-search-architecture/) covers how it's built, for anyone who wants the mechanism behind the API.\n\nWas this article helpful?", "url": "https://wpnews.pro/news/full-text-search-is-now-generally-available-in-pinecone-database", "canonical_source": "https://www.pinecone.io/blog/full-text-search-generally-available/", "published_at": "2026-09-09 12:00:00+00:00", "updated_at": "2026-09-09 12:41:03.315245+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-products", "ai-tools"], "entities": ["Pinecone Systems", "Pinecone Database", "BM25", "Documents API"], "alternates": {"html": "https://wpnews.pro/news/full-text-search-is-now-generally-available-in-pinecone-database", "markdown": "https://wpnews.pro/news/full-text-search-is-now-generally-available-in-pinecone-database.md", "text": "https://wpnews.pro/news/full-text-search-is-now-generally-available-in-pinecone-database.txt", "jsonld": "https://wpnews.pro/news/full-text-search-is-now-generally-available-in-pinecone-database.jsonld"}}