# MAP@k vs Precision@k vs MRR: Building Capability to choose the right one for each situation > Source: > Published: 2026-11-09 08:31:04+00:00 Clarifying the confusion once and for all in evaluating RAG and many other use cases! A simple, example-driven guide to the three most common ranking metrics in search, recommenders & RAG. 🎯 Why this topic matters Say you built a search engine, a recommender, or a RAG retriever. It returns a ranked list of results, and now you need one number that says whether it is good. Accuracy won’t work here, because order matters. A good result at position 1 is worth far more than the same result at position 10. Three metrics come up again & again: Precision@k MAP@k (Mean Average Precision at k) MRR (Mean Reciprocal Rank) They look similar, but they answer different questions. Pick the wrong one & you will optimize the wrong behavior. 🍕 One analogy to rule them all: The Pizza Menu Imagine a food app shows you a list of restaurants for “best pizza near me”. Keep this table in mind. The rest of the blog fills in the details. 1️⃣ Precision@k: “How clean is my top-k?” Definition It measures the proportion of recommended items in the top-k set that are relevant to the user. ``` Precision@k = (# relevant items in top-k) / k ``` Example Top 5 results: ✅ ❌ ✅ ❌ ✅ gives 3/5 = 0.6. 👍 Strengths Very easy to compute & explain to non-technical stakeholders. Fits UIs with a fixed list size (for example, “top 5 recommendations” on a carousel). Works well when irrelevant items hurt the user experience. 👎 Weaknesses Order-blind. P@k treats all positions equally, which may not align with user behavior where higher-ranked items are more significant. A list of ✅✅✅❌❌ & a list of ❌❌✅✅✅ both score 0.6, but the first is clearly better. It ignores how many relevant items exist in total. If a query has only 2 relevant documents, P@5 can never exceed 0.4. It depends heavily on the choice of k. 2️⃣ MRR: “How fast do I find the first right answer?” Definition The reciprocal rank of a query response is the inverse of the rank of the first correct item. If the correct item is at rank k, the reciprocal rank is 1/k. For a set of queries, MRR is the average of the reciprocal ranks of results for the queries. where |Q| is the number of queries and rank_i is the position of the first relevant result for query i. ``` RR = 1 / rank_of_first_relevant (0 if none found)MRR = average of RR over all queries ``` Quick table Notice how sharply the score drops: it rewards rank 1 heavily & barely distinguishes rank 8 from rank 10. Analogy 🔑 Think of looking for your car keys. You only care when you find the first key. Once you have it, you leave. MRR measures exactly this “time to first success”. 👍 Strengths Perfect for “one right answer” tasks: Q&A, chatbots, navigational search, autocomplete, “I’m feeling lucky”. Simple & intuitive. 👎 Weaknesses It ignores everything after the first hit. A list with 1 good result at rank 1 scores the same as a list with 10 good results starting at rank 1. It’s useless when users want several good results (shopping, music, news feeds). 3️⃣ MAP@k: “Are my relevant items high up & plentiful?” MAP@k has two layers: score each query (AP@k), then average across queries (MAP@k). Let’s build it step by step. Step 1: Precision at every relevant position Walk down the ranked list. Each time you hit a relevant item, note the precision at that point: how many of the items so far were relevant. Step 2: Average them to get AP@k (per query) AP@k averages those precision values, so it rewards rankings that put relevant items near the top. Here: P@i is the precision at cutoff i. rel(i) is 1 if the item at rank i is relevant & 0 otherwise. N is a normalizing count. Its definition varies, as covered below. Example: the relevant items in the top 5 are at ranks 1, 3 & 5, & these are the only 3 relevant items (R = 3). Precision at those ranks: 1/1, 2/3, 3/5. Their sum is ≈ 2.267. With N = 3: AP@5 ≈ 2.267 / 3 ≈ 0.756. With N = k = 5, the same ranking scores ≈ 2.267 / 5 ≈ 0.453. That gap is the hidden catch below. Step 3: Average AP@k across all queries to get MAP@k MAP@k = (1/|Q|) × Σ AP@k(q), averaged over every query q. In a recommender, each user plays the role of a query. ⚠️ The hidden catch: the denominator N This trips up many people, because there is no single universal definition. Four common versions: N = R (all relevant items for the query). This is the classic information-retrieval definition. If R > k, even a perfect top-k list can’t reach 1, because the maximum is k/R. N = k (the list length). If R < k, a perfect ranking still can’t reach 1, because the maximum is R/k. N = min(k, R). This fixes both problems above: a perfect ranking always scores 1. It’s a common choice in recommender-system evaluation. N = relevant items actually found in the top k. This is the most forgiving version: it ignores relevant items you missed entirely. If nothing relevant is found, N = 0, & implementations typically set AP = 0. Also check how a library handles queries with no relevant items at all. Some skip them, others score them 0, & that choice moves MAP. 👉 Always check which variant your library or paper uses before comparing numbers. Analogy 🎣 Imagine fishing in a lake with 4 good fish (R = 4). You cast 5 times (k = 5). One lake is one query, so this gives you AP. Fish many lakes & average the results, and you get MAP. (Here min(k, R) = R = 4, so N = R gives the same numbers as the middle column.) 👍 Strengths Rewards both relevance & ranking order. Considers every relevant item, unlike MRR, which only looks at the first one. A natural fit for recommenders & multi-answer retrieval. 👎 Weaknesses Needs binary relevance (relevant or not). For graded relevance, like 0–3 stars, use nDCG. Harder to explain than Precision@k. The normalization ambiguity above, plus the choice of how to handle queries with no relevant items. 🧪 Worked example: same Precision@5, very different quality Suppose a query has R = 4 relevant documents in total. We compare 3 systems that each return 5 results. Manual calculation for System A Relevant at ranks 1, 3, 5. P@1 = 1/1 = 1.000 P@3 = 2/3 = 0.667 P@5 = 3/5 = 0.600 Sum = 2.267, so AP@5 = 2.267 / 4 = 0.567 🔍 What do we learn? Precision@5 can’t tell the three systems apart. All score 0.6. MRR separates B from A & C (B finds the first hit late), but it can’t tell A from C. MAP@5 ranks them C > A > B, which matches human intuition. This one table is the best argument for why you shouldn’t rely on a single metric. System A is the same ranking as the MAP@k example (relevant at ranks 1, 3 & 5). It scored 0.756 there because we assumed only 3 relevant items existed. Here R = 4, so one relevant item was never retrieved, & that pulls AP@5 down to 0.567. Same ranking, four denominators One ranking can score anywhere from 0.453 to 0.756 depending on the variant. This is the “hidden catch” from the MAP@k section, in code. Computing across many queries (the “Mean” part) 💡 Pro tip: Relevant documents that are never retrieved count as zero precision. Dividing by R (or min(k, R)) is exactly what applies that penalty, which is the classic IR convention. The “found” variant divides only by the hits you retrieved, so missed documents cost you nothing. That’s why it’s the most forgiving variant. ⚔️ Head-to-head comparison 🧭 Which one should you choose? Use this decision guide: “Users need exactly one right answer” (FAQ bot, Q&A, RAG with a single gold passage) → MRR “Users see a fixed grid or carousel of N items” → Precision@k (pair it with Recall@k) “Relevance has grades (highly, somewhat, not relevant)” → consider nDCG@k instead (Another blog on this coming soon) “I’m not sure” → report all three. Each one catches a different failure. 🚫 Common mistakes to avoid! Comparing MAP@k numbers across papers or libraries without checking the denominator variant. Picking k arbitrarily. Choose k to match what the user actually sees (for example, 10 results per page or 5 recommendations). Using MRR for multi-answer tasks. It hides poor results after the first hit. Relying only on Precision@k. It can’t see ordering problems, as our A/B/C example showed. Forgetting queries with zero relevant items. Decide in advance whether to skip them or score them 0, & stay consistent. The code above scores them 0. Averaging over too few queries. Ranking metrics are noisy on small test sets, so use enough queries to get stable results. Not defining a cutoff for MRR. If you use MRR@k, return 0 when the first relevant item is beyond rank k. 🛠 Real-world use cases RAG pipelines: Recall@k for “did the needed chunk get retrieved at all?”, MRR for “did it show up early?”, MAP@k when the answer needs several chunks and you want all of them near the top (especially before a reranker or token budget trims the list), & Precision@k to measure noise in the context window. E-commerce search: MAP@k to make sure many good products rank near the top. Switch to nDCG when relevance is graded (exact match vs substitute). Recommender systems: MAP@k & Precision@k on held-out user interactions. Chatbots / FAQ retrieval: MRR, because there is typically one correct answer. Bug localization & code search: MAP & MRR are the usual pair. MRR shows how soon the first relevant file or function appears, while MAP rewards ranking all the relevant ones high. ⭐️ Final takeaway Precision@k → “How many of my top-k are good?” Simple, but blind to order. MRR → “How soon do I see the first good result?” Ideal for single-answer tasks, but ignores everything after the first hit. MAP@k → “Are all my good results ranked high?” The richest of the three for multi-answer ranking, but you must check the normalization variant (R, k, min(k, R), or hits in top-k). Three systems can share the same Precision@5 & still differ massively in real quality, so never trust one metric alone. Match the metric to the user behavior: one answer → MRR, fixed list → Precision@k, many ranked answers → MAP@k. For graded relevance, step up to nDCG. Always report your k, your relevance definition, & your formula variant so your results are reproducible. ✨ Where to go from here! You have come this far, which really means a lot. The further steps are as follows: 🎊 Appreciate yourself for deciding to read this blog, which has exponentially enhanced your knowledge. 🎉 Follow me on Medium (It’s free) & opt in for email notifications to stay updated with my latest articles on multi-tech (Complete AIOPS). 🎯 Follow me on LinkedIn (It’s free) to get the latest tech dose & tips & tricks.