Maximum Marginal Relevance and Elastic: Diversifying Search Results

Elasticsearch has introduced a blog post demonstrating how to implement Maximum Marginal Relevance (MMR) to diversify search results, using a fashion product catalog as an example. The MMR algorithm balances relevance and diversity by iteratively selecting items that are relevant to the query but dissimilar to already chosen results, controlled by a lambda parameter. The implementation uses vector similarity for relevance scoring and includes code examples for reranking search results.

Blog /search-labs/blog Diversifying search results with Maximum Marginal Relevance Implementing the Maximum Marginal Relevance MMR algorithm with Elasticsearch and Python. This blog includes code examples for vector search reranking. When you search for "pants" in an e-commerce catalog, do you really want to see 10 variations of the same black capris? Probably not. You'd likely prefer a diverse selection showing different styles, colors, and types of pants. This is where Maximum Marginal Relevance MMR comes in — a powerful technique for balancing relevance with diversity in search results. In this blog, we'll explore how to implement MMR with Elasticsearch to create more diverse and useful search results, using a fashion product catalog as our example. The problem: When relevance isn't enough Traditional search systems optimize for one thing: relevance. They find items that best match your query and rank them by similarity scores. This works well for many use cases, but it can lead to redundant results. Consider searching for "pants" in a fashion catalog. A pure relevance-based search might return: Black capris score: 0.682 Black capris from another brand score: 0.681 More black capris score: 0.680 Even more black capris score: 0.680 ...you get the idea While these are all highly relevant to "pants", they're not particularly helpful for a user trying to explore different options. What we need is a way to maintain relevance while promoting diversity. Enter Maximum Marginal Relevance MMR https://www.cs.cmu.edu/~jgc/publication/The Use MMR Diversity Based LTMIR 1998.pdf is an algorithm that elegantly solves this problem by balancing two competing objectives: Relevance : How well items match the query Diversity : How different items are from each other The algorithm works iteratively, selecting items that are relevant to the query but different from already selected items. This ensures that each additional result adds new information rather than redundancy. How MMR works The MMR algorithm follows a simple but effective process: Start by selecting the most relevant item highest score For each remaining item, calculate an MMR score that combines: Its relevance to the query and its dissimilarity to already selected items Select the item with the highest MMR score Repeat until you have enough results The key insight is the MMR scoring formula: MMR Score = λ × relevance - 1 - λ × max similarity to selected The λ parameter controls the trade-off, where λ = 1.0 is pure relevance no diversity and λ = 0.0: pure diversity ignore relevance . MMR doesn’t dictate how you score relevance — it just needs a number. That could come from BM-25, a learned ranker, or any custom metric you like. Because BM-25 relies on per-term statistics stored in postings lists that aren’t available on the client, we’ll use vector similarity https://www.elastic.co/search-labs/blog/introduction-to-vector-search as our relevance function for this blog. This lets us compute relevance elegantly by taking the dot product. Implementing MMR Let's see how to implement MMR for an image search system using Elasticsearch and multimodal embeddings. To showcase the effects of MMR we will be using the paramaggarwal/fashion-product-images-dataset https://www.kaggle.com/datasets/paramaggarwal/fashion-product-images-dataset dataset. In this blog post we will only focus on the retrieval and reranking, but you can find a full end-to-end example https://github.com/elastic/elasticsearch-labs/blob/main/supporting-blog-content/result-diversification/diversification.ipynb in our search-labs GitHub repository. First, we need to search for similar items using vector similarity: python def search similar images es, index name, query vector, k=10 : """Search for similar images using vector similarity""" query = { "knn": { "field": "image vector", "query vector": query vector, "k": k }, " source": "id", "image url" , "size": k } response = es.search index=index name, body=query return extract results response This gives us our initial relevance-ranked results. Now, let's apply MMR to rerank them for diversity: python def maximal marginal relevance query embedding: List float , embedding list: List List float , lambda mult: float = 0.5, value between 0.0 and 1.0 k: int = 4, - List int : query embedding arr = np.array query embedding if min k, len embedding list <= 0: return if query embedding arr.ndim == 1: query embedding arr = np.expand dims query embedding arr, axis=0 calcuate the similarity to the query for all reranking candidates similarity to query = cosine similarity query embedding arr, embedding list 0 start with the most similar item to the query most similar = int np.argmax similarity to query idxs = most similar selected = np.array embedding list most similar Iteratively select documents that maximize MMR score while len idxs < min k, len embedding list : best score = -np.inf idx to add = -1 calulate the similarity between all candidate items and all selected items similarity to selected = cosine similarity embedding list, selected look at all candidates for i, query score in enumerate similarity to query : if i in idxs: continue Find the highest similarity of this item to already selected items redundant score = max similarity to selected i Calculate MMR score equation score = lambda mult query score - 1 - lambda mult redundant score select this item if it has the highest MMR score for this run if equation score best score: best score = equation score idx to add = i append the item with the highest MMR score for this run to our list idxs.append idx to add selected = np.append selected, embedding list idx to add , axis=0 return idxs The impact: Before and after MMR Let's look at how MMR transforms search results for the query "pants": Before MMR pure relevance : Notice the repetition? We see that most products are a dark shade with a straight cut. After MMR λ=0.5 : The reranked results showcase diversity in many features such as colors, brands, target demographics, style and more. Each result adds new information, making the exploration experience more valuable. Tuning MMR for your use case The beauty of MMR lies in its tunability. By adjusting the λ parameter, you can adapt the algorithm to different scenarios: Product Discovery λ=0.3-0.5 : Emphasize diversity to help users explore options Precision Search λ=0.7-0.9 : Prioritize relevance when users know what they want Research Applications λ=0.5-0.7 : Balance for comprehensive coverage You can even make λ dynamic based on: Query type broad vs. specific User behavior browsing vs. purchasing Result set characteristics high vs. low similarity Performance considerations While MMR provides significant value, it does come with computational costs. The algorithm computes similarities between candidates and selected items. For production systems, consider limiting the reranking depth to a top k for results for the best tradeoff between impact and performance. Also, consider that retrieving the vectors will impact your performance, as it requires serialization of large amounts of data. Beyond e-commerce: Other applications While we've focused on fashion products and traditional search bars with result lists, MMR has broad applications. When using Retrieval Augmented Generation RAG applications https://www.elastic.co/search-labs/blog/retrieval-augmented-generation-rag , a request for "great vacation spots" could, without result diversification, yield results solely focused on Greek beaches. MMR, however, would diversify the output to include a range of vacation types, from relaxing beach holidays in Greece to adventurous hikes up an Icelandic volcano. Other use-cases could include: News aggregation : Show articles from different sources and perspectives Document search : Surface documents covering different aspects of a topic Recommendation : Suggest diverse movies, music, or content Academic search : Find papers from different research groups and methodologiesAnd countless more Conclusion Maximum Marginal Relevance transforms search from a pure relevance race into a balanced information retrieval system. By implementing MMR with Elasticsearch, you can deliver search results that are not just relevant but also informative and diverse. The key is finding the right balance for your use case. Start with λ=0.7 for a relevance-leaning approach, then adjust based on user feedback and behavior. Your users will appreciate seeing a variety of relevant options rather than slight variations of the same thing. Sometimes the second-best match that's different is more valuable than another perfect match that's redundant. That's the power of diversity in search.