{"slug": "announcing-native-bm25-ranking-in-alloydb-and-cloud-sql", "title": "Announcing Native BM25 Ranking in AlloyDB and Cloud SQL", "summary": "Google announced the preview of a native BM25 index in AlloyDB and Cloud SQL for PostgreSQL 17+, built on the open-source pg_textsearch extension created by TigerData, eliminating the need for a separate full-text search backend. The native index adds industry-standard keyword ranking with inverse document frequency, term frequency saturation, and document length normalization, and AlloyDB also delivers up to 6x and 10x faster vector search queries versus standard PostgreSQL using ScaNN and HNSW index types. The company said the unified hybrid search backend removes data duplication, ETL pipelines, and synchronization lag for retrieval-augmented generation and data agent architectures.", "body_md": "Vector search is a critical component of generative AI, retrieval-augmented generation (RAG), and data agent architectures, but sometimes vector search alone isn't enough. While vector embeddings are incredible at understanding conceptual meaning, they stumble on specific alphanumeric IDs and exact product SKU numbers. To build truly robust search and AI applications, you may need the combination of semantic vector search and traditional exact keyword full-text search — what we call hybrid search.\n\nIn search, Best Matching 25, or BM25, is a key algorithm used to estimate how relevant a document is to a given query. Until today, if you wanted BM25 ranking with AlloyDB or Cloud SQL, you needed to add an additional full-text search backend. This introduced data silos, sync lags, and operational complexity. Today, we are eliminating the friction of maintaining a separate full-text search backend altogether, with the preview of the native BM25 index in AlloyDB and Cloud SQL for PostgreSQL 17+, made possible through the open-source [`pg_textsearch` extension](https://github.com/timescale/pg_textsearch) created by TigerData.\n\nNow, with a unified hybrid search backend, you no longer need to provision, manage, or pay for separate systems to get state-of-the-art full-text retrieval. It all happens directly inside your database, where your operational data lives, delivering:\n\n**Industry-standard keyword ranking:** Powered by TigerData's `pg_textsearch`, bring lightning-fast, C-optimized BM25 scoring directly to your Postgres tables.\n\n**No complexity, total consistency:** Eliminate the data duplication, ETL pipelines, and synchronization lag that you get when you maintain multiple backends for vector and full-text retrieval.\n\n**Supercharged semantic search (AlloyDB exclusive):** Get up to 6x and 10x faster vector search queries (when compared to standard PostgreSQL) with ScaNN and HNSW index types.\n\nIf you’ve used PostgreSQL's built-in `ts_rank` for full-text search at any meaningful scale, you already know its limitations. Ranking quality degrades as your corpus grows. There’s no support for inverse document frequency, so common words carry the same weight as rare ones. There’s no term-frequency saturation, so a document that mentions \"database\" 50 times outranks one that mentions it once. \n\nBM25 is the information retrieval gold standard, providing inverse document frequency (rarer terms matter more), term frequency saturation (repetition doesn't dominate), and document length normalization. You can learn more in this [blog post](https://www.tigerdata.com/blog/pg-textsearch-bm25-full-text-search-postgres) by TigerData about how they built a BM25 search engine on PostgreSQL pages. \n\nHere’s how to get started with BM25 full-text search on both AlloyDB and Cloud SQL. Consider a sample table, cymbal_products, that contains the unique identifier uniq_id, a product_name column, a product_description column containing a text description of each product, and a generated product_embedding column. `cymbal_products` contains information on various retail products, including indoor and outdoor plants.\n\nTo use BM25, enable the pg_textsearch extension.\n\nCreate the index on the `product_description` column from the `cymbal_products` table.\n\nA BM25 full-text search query can be executed using the <@> special operator. In the snippet below, we search for ‘cherry tree’.\n\nSample output is shown below. A more negative score indicates a stronger relevance match.\n\nSetting up a hybrid search system in AlloyDB is simple. You can create both your vector and keyword indexes on the same table and merge the results seamlessly using the [hybrid search user-defined function (UDF](https://docs.cloud.google.com/alloydb/docs/ai/run-hybrid-vector-similarity-search#hybrid-search)).\n\nHere is how to create a ScaNN vector search index:\n\nAlloyDB provides an out-of-the-box hybrid search UDF that makes it very simple to run hybrid search queries. The UDF merges the ranked results from each search component into a single, unified list using the Reciprocal Rank Fusion (RRF) algorithm. This query utilizes the UDF to perform a vector search for ‘trees that grow taller than houses’ and a keyword search for ‘California’ in the product description.\n\nAs shown in the sample output below, results are ranked in descending order of their RRF scores.\n\nHere, hybrid search bridges the gap between semantic intuition and exact keyword matching. While vector embeddings excel at grasping conceptual queries, like \"trees that grow taller than houses\", traditional full-text search provides the pinpoint precision needed for strict identifiers like \"California.\" By fusing the two, AlloyDB helps ensure your application prioritizes highly specific, locally relevant results like ‘California Sycamore’ right at the top of the list.\n\nIn Cloud SQL, you can create both your vector and keyword indexes on the same table and merge the results seamlessly using Common Table Expressions (CTEs) and coalescing the RRF score, as shown below.\n\nHere is how to create an HNSW index in Cloud SQL.\n\nHere is the hybrid search query.\n\nThe resulting output is identical to the AlloyDB hybrid search results shown above.\n\nWatch how this all comes together in this demo video.\n\nWe are incredibly excited to work with TigerData and cannot wait to see how you leverage native BM25 support to build faster, smarter, and simpler AI applications. Turn on the `pg_textsearch` extension today, and experience the ultimate hybrid search engine experience with AlloyDB and Cloud SQL.\n\n**Want to get started?** **Check out”** \n\nAlloyDB resources\n\nNew to AlloyDB? Discover AlloyDB with a [30-day free trial](https://docs.cloud.google.com/alloydb/docs/free-trial-cluster)\n\n[Choose a vector index in AlloyDB AI](https://docs.cloud.google.com/alloydb/docs/ai/choose-index-strategy)\n\n[AlloyDB BM25 documentation](https://docs.cloud.google.com/alloydb/docs/ai/create-bm25-index) \n\n[AlloyDB hybrid search UDF documentation](https://docs.cloud.google.com/alloydb/docs/ai/run-hybrid-vector-similarity-search#hybrid-search)\n\nCloud SQL resources\n\n[TigerData pg_textsearch Release Page](https://www.tigerdata.com/blog/pg-textsearch-bm25-full-text-search-postgres)", "url": "https://wpnews.pro/news/announcing-native-bm25-ranking-in-alloydb-and-cloud-sql", "canonical_source": "https://cloud.google.com/blog/products/databases/native-bm25-search-in-alloydb-and-cloud-sql/", "published_at": "2026-09-18 16:30:00+00:00", "updated_at": "2026-09-18 16:56:37.290721+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-search", "large-language-models", "generative-ai", "ai-products"], "entities": ["Google", "AlloyDB", "Cloud SQL", "PostgreSQL 17", "TigerData", "pg_textsearch", "ScaNN", "HNSW"], "alternates": {"html": "https://wpnews.pro/news/announcing-native-bm25-ranking-in-alloydb-and-cloud-sql", "markdown": "https://wpnews.pro/news/announcing-native-bm25-ranking-in-alloydb-and-cloud-sql.md", "text": "https://wpnews.pro/news/announcing-native-bm25-ranking-in-alloydb-and-cloud-sql.txt", "jsonld": "https://wpnews.pro/news/announcing-native-bm25-ranking-in-alloydb-and-cloud-sql.jsonld"}}