Entry 07 ended on a loose thread: a document split into 17 chunks crowds out equally relevant single-entry documents, just by having more shots at ranking. The usual fix is a per-document cap, so I flagged it and moved on. Now that the collection holds 741 real chunks instead of 20, it was time to find out whether the problem was real at this size, and whether the fix worked.
Measured first, before touching anything. Same question as Entries 06 and 15, ten results, no cap:
q = ollama.embeddings(model="nomic-embed-text", prompt="how do I check pod status with oc")
results = collection.query(query_embeddings=[q["embedding"]], n_results=10)
231.1 08-mcp-server-claude-code_chunk2
320.0 15-sse-reconnect-distance-instability_chunk1
325.4 mcp-server-architecture-platform-engineering-kubernetes_chunk12
328.6 13-hand-rolled-agent-loop_chunk2
334.6 mcp-server-architecture-platform-engineering-kubernetes_chunk19
344.5 13-hand-rolled-agent-loop_chunk4
347.5 13-hand-rolled-agent-loop_chunk3
350.6 mcp-server-architecture-platform-engineering-kubernetes_chunk16
353.3 mcp-server-architecture-platform-engineering-kubernetes_chunk13
357.4 08-mcp-server-claude-code_chunk1
Four documents fill all ten slots: four chunks from the MCP architecture article, three from Entry 13, two from Entry 08, one from Entry 15. That looks like the bias I expected, until you check which documents are missing. The three longest articles in the collection (the retry-logic piece and the Terraform state piece at 38 chunks each, the zero-downtime deploy piece at 37) don't appear at all. The winners are the topically closest documents, not the longest. Several chunks of one on-topic article can all be legitimately relevant, so some of this clustering isn't bias.
The top hit also looked too good. 231.1 is 89 points ahead of the runner-up, and when I printed the chunk, it was Entry 08's own quote of Claude Code's answer, including the phrase "checking pod status with oc". The index was retrieving the write-up of this exact question. That's a side effect of embedding the series' own entries into the store, and it fits the lexical-overlap suspicion from Entry 13: near-verbatim wording scores far better than anything merely related.
Then the cap. Fetch forty results, keep only the best chunk per source document, take the top ten:
raw = collection.query(query_embeddings=[q["embedding"]], n_results=40)
seen, kept = set(), []
for doc_id, dist, meta in zip(raw["ids"][0], raw["distances"][0], raw["metadatas"][0]):
if meta["source"] not in seen:
seen.add(meta["source"])
kept.append((dist, doc_id))
if len(kept) == 10:
break
The top four were unchanged, since they were already four different documents. Slots five through ten changed completely:
| Uncapped, slots 5-10 | Capped, slots 5-10 | |
|---|---|---|
| What fills them | More chunks of the MCP architecture article, Entry 13 and Entry 08 | One chunk each from the chunking entry, secrets management, RAG runbooks, provider swapping, zero-downtime deploys, prompt versioning |
| Distance range | 334.6 to 357.4 | 360.8 to 381.5 |
The cap removed extra chunks from articles that really are about Kubernetes diagnostics over MCP and replaced them with chunks from documents that look unrelated, judging by title and by distance. At this scale it traded relevant results for variety.
I had also been treating one result as the real test. Entry 02 was the top hit in Entries 06, 07, 13 and 15, and it vanished from the top ten here, capped or not, so I assumed the cap should bring it back. It didn't. Ranking its chunks across the whole collection shows where it went:
52 415.9 02-oc-cli-mentor-system-prompt_chunk1
73 423.6 02-oc-cli-mentor-system-prompt_chunk2
77 424.1 02-oc-cli-mentor-system-prompt_chunk0
137 441.4 02-oc-cli-mentor-system-prompt_chunk3
683 555.5 02-oc-cli-mentor-system-prompt_chunk4
Its best chunk scores 415.9, lower (better) than the 437.7 it scored when it ranked first among 45 entries. The document didn't get worse. Fifty other chunks now sit ahead of it, in a crowded middle where roughly forty chunks fall inside a band about 58 points wide. Entry 15 found that rewording one query moved a distance by 54 points, so that whole band could reshuffle under a different phrasing. I haven't tested that, but the numbers point that way.
The more important realization is that Entry 02 was never a correct answer. It's about system prompts, not pod-status commands, and Claude Code said so back in Entry 08. This query probably has no right answer anywhere in the collection, which means I was judging a ranking with nothing to check it against. Whether the cap "worked" isn't a question I can answer from this data.
So the finding isn't about the cap. The cap changes results, and at this size it costs relevant ones, but I can't call that good or bad without a question where I already know which document should win. The next entry builds that: a small set of questions, each with a known right document, checked by hand. Until then, treat every ranking in this series as a description of what the index returns, not evidence of what it should return.