Visually validating vector data for RAG

A 6 November 2026 snapshot of a personal website's RAG search index — 85 pages split into 1,128 text chunks, each embedded in 512 dimensions — showed that 81.4% of the ten closest neighbors for each chunk came from the same page, the author reported. Comparing PCA, t-SNE and UMAP projections against exact cosine neighbors in the original 512-dimensional space, the author found the three settings retained 13.1%, 75.1% and 64.5% of nearest-neighbor fidelity respectively, and noted the strongest cross-page pair reached cosine similarity 0.944 because of repeated article listings rather than independently written passages. The author concluded that projection maps help choose passages to inspect but cannot measure retrieval quality, which still requires questions with reviewed answers.

Visually validating vector data for RAG · 4 min read I wanted to see what the vectors behind my website’s search actually looked like. On 6 November 2026, I took a snapshot of its published index: 85 pages, split into 1,128 text chunks, each represented by a 512-dimensional embedding. Plotting them gave me a way to choose which passages to inspect. Retrieval-augmented generation RAG retrieves document passages before passing them to a language model. An embedding represents text as numbers used for similarity search. This exercise looks at that retrieval data, using my website as a small, inspectable example. It complements the broader checks in Validating a vector database built for RAG https://pournasserian.com/writing/validating-a-vector-database-for-rag . In brief 1. A projection map helps me find groups and isolated passages to inspect, but moving 512 dimensions into two changes their relationships. 2. Exact neighbors and source text help explain what a visible group represents. In this index, 81.4% of the ten closest neighbors come from the same page. 3. Visual inspection can reveal repeated content and broken chunk boundaries. Questions with reviewed answers are still needed to measure retrieval quality. Use a map to choose what to inspect Different projections emphasize different relationships, so I compare views before interpreting a cluster. Principal component analysis PCA https://scikit-learn.org/stable/modules/decomposition.html pca gives a linear baseline. t-distributed stochastic neighbor embedding t-SNE https://scikit-learn.org/stable/modules/manifold.html t-sne emphasizes local neighborhoods. Uniform Manifold Approximation and Projection UMAP https://umap-learn.readthedocs.io/en/latest/parameters.html builds its view from a neighborhood graph. For each map, I compared every point’s ten closest neighbors with its ten closest vectors in the original space. The displayed PCA, t-SNE and UMAP settings retained 13.1%, 75.1% and 64.5% respectively. These percentages measure the map’s fidelity, not search accuracy. They describe these settings on this corpus, rather than a general ranking of the methods. Open the scalable map https://pournasserian.com/writing/visually-validating-rag-vectors/neighborhood-map.svg or compare four projections of the same data https://pournasserian.com/writing/visually-validating-rag-vectors/core-comparison.svg . The islands give me places to investigate. Their spacing and apparent size are not reliable semantic measurements, as How to Use t-SNE Effectively https://distill.pub/2016/misread-tsne/ illustrates. An isolated point might be a distinctive passage, an index page or an extraction problem. I need its text to tell those apart. Check the neighbors in the original space I confirm a map’s suggestion by looking up exact cosine neighbors in all 512 dimensions. For one passage about checking an access token’s audience, I excluded its own page and kept the closest chunk from each other page. That makes the comparison more useful than repeatedly returning the same article. Open the scalable neighbor comparison https://pournasserian.com/writing/visually-validating-rag-vectors/exact-neighbors.svg . The strongest cross-page pair elsewhere in the index had cosine similarity 0.944. Reading it showed repeated article listings on the Writing index and the MCP series page. The similarity was understandable; it was not evidence of two independently written passages reaching the same conclusion. This is why I inspect the text before treating a group as a topic or a pair as unwanted duplication. Look at the chunks behind the picture Simple distributions help explain why the map takes its shape. Longer articles contribute more points, and passages from one page often share a subject. The embedding input also includes the page title and heading path. Open the scalable diagnostics https://pournasserian.com/writing/visually-validating-rag-vectors/data-diagnostics.svg . This snapshot has five chunks below 200 characters and 24 at the 1,500-character cap. Inspecting adjacent chunks on the FluentCMS page found a URL split after https://docs.strapi.io/cms/fea , with tures/admin-tokens beginning the next chunk. That is a concrete boundary defect. Its effect on retrieval still needs testing. Method and caveats The snapshot was built at 4:49 p.m. Toronto time on 6 November 2026, using openai/text-embedding-3-small at 512 dimensions. I normalized each vector to unit length, kept the metadata aligned with its row and checked that all 85 sitemap pages were present. There were no empty chunks, zero or non-finite vectors, or exact duplicate chunk bodies. The t-SNE example uses cosine distance, perplexity 30 and seed 42; UMAP uses cosine distance, 30 neighbors, minimum distance 0.1 and the same seed. I also tried other settings. The figure pack and method notes https://pournasserian.com/writing/visually-validating-rag-vectors/figures.zip include all eight projection examples, a similarity heatmap, a hierarchy, the diagnostics and recorded parameters. The plots were calculated from existing vectors; no new embeddings were generated for this analysis. This is an inspection of a website retrieval index, not an evaluation of a complete RAG answering system. The site combines vector and keyword results. Before claiming retrieval quality, I would test representative questions against reviewed answer-bearing passages and compare the actual ranked results. The visual checks tell me where to look and what to test next.