{"slug": "day-15-visualizing-embeddings-with-t-sne-and-pca", "title": "Day 15: Visualizing Embeddings with t-SNE and PCA", "summary": "A developer demonstrated how to visualize word embeddings by reducing high-dimensional vectors to two dimensions using PCA and t-SNE. The walkthrough explains that PCA captures broad linear trends while t-SNE preserves local neighborhoods, and includes a Python example shrinking five 6-dimensional word vectors down to 2D with scikit-learn and matplotlib. The author notes such visualizations help spot patterns, diagnose biases, and debug models.", "body_md": "**Previously, on Day 14:** Explained how word embeddings represent words as vectors, the limitations of classic approaches like one-hot encoding, and how FastText uses subword units to create robust embeddings that handle rare, new, or misspelled words.\n\nWord embeddings are dense vectors that represent words as points in space. Imagine each word having a \"position\" in a space with maybe 100 or 300 directions—far beyond our normal three. Each number in the vector says how much the word lines up with one of these directions.\n\nWords with similar meanings often end up close together in this space. For example, you might find \"cat\" and \"dog\" in roughly the same region, but far away from \"car.\" By plotting these positions, you get a direct look at what the embedding has learned—what it thinks is similar, what feels distant, and which words cluster together.\n\nVisualizing embeddings isn't just for curiosity. It can help spot patterns, catch mistakes, diagnose biases, and debug your model.\n\nMost people are comfortable thinking in two or three dimensions. Word embeddings use dozens or hundreds. If you try to plot 100 axes, you won’t see anything useful.\n\nTo make these embeddings visible, we use dimensionality reduction. This means taking high-dimensional data and squeezing it down to two or three dimensions in a way that tries to keep the important relationships between points.\n\nPrincipal Component Analysis (PCA) is a classic way to reduce dimensions. It helps find the axes (directions) along which your data varies most.\n\nPicture a swarm of points floating in 3D. You want to shine a light so that this cloud casts the biggest, flattest shadow onto a wall. PCA picks the best direction for that light—finding the axes with the most spread. In higher dimensions, it does the same, but finds the \"shadow\" in 2D or 3D that keeps as much of the structure as possible.\n\nFor embeddings, PCA projects all those dimensions down to just 2 or 3. It’s a linear technique, so it captures big movements or trends, but misses subtle, nonlinear patterns.\n\nt-Distributed Stochastic Neighbor Embedding (t-SNE) is another way to reduce dimensions. While PCA focuses on big-picture spread, t-SNE tries to make sure points that were close together in the original space stay close together in 2D.\n\nImagine t-SNE as reshuffling the points so that neighborhoods stay strong. If \"cat\" and \"dog\" were tight friends in the embedding, they’ll show up beside each other in the plot. t-SNE does this by matching up how similar pairs of points are in the high-dimensional space versus the lower-dimensional plot.\n\nThe tradeoff: t-SNE makes local groups clear, but you can’t trust the exact distances between distant groups. The pattern inside a cluster is meaningful; the gap between two clusters often isn’t.\n\nMany people use both—PCA for the overview, t-SNE for finding groups inside.\n\nLet's use a tiny, hardcoded example. Real-world code would load hundreds or thousands of real vectors (say, from Word2Vec or GloVe), but it's easier to see what's happening with something simple.\n\nHere’s code to shrink five 6-dimensional word embeddings down to 2D using both PCA and t-SNE, then plot and label the results:\n\n``` python\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom sklearn.decomposition import PCA\nfrom sklearn.manifold import TSNE\n\n# Hardcoded 6-dimensional \"embeddings\" for 5 words\nwords = [\"cat\", \"dog\", \"car\", \"bus\", \"apple\"]\nembeddings = np.array([\n    [0.4, 0.2, 0.5, 0.1, 0.4, 0.2],  # cat\n    [0.5, 0.1, 0.4, 0.2, 0.3, 0.2],  # dog\n    [0.1, 0.6, 0.2, 0.8, 0.2, 0.3],  # car\n    [0.2, 0.7, 0.1, 0.9, 0.1, 0.3],  # bus\n    [0.8, 0.2, 0.9, 0.1, 0.7, 0.2],  # apple\n])\n\n# PCA to 2D\npca = PCA(n_components=2)\nreduced_pca = pca.fit_transform(embeddings)\n\n# t-SNE to 2D (small perplexity for small dataset)\ntsne = TSNE(n_components=2, random_state=42, perplexity=2)\nreduced_tsne = tsne.fit_transform(embeddings)\n\n# Plotting\ndef plot_embeddings(X, title):\n    plt.figure(figsize=(5, 4))\n    for i, word in enumerate(words):\n        x, y = X[i]\n        plt.scatter(x, y)\n        plt.text(x+0.01, y+0.01, word, fontsize=12)\n    plt.title(title)\n    plt.axis('off')\n    plt.show()\n\nplot_embeddings(reduced_pca, \"PCA Visualization\")\nplot_embeddings(reduced_tsne, \"t-SNE Visualization\")\n```\n\nThis plots each word, showing how PCA and t-SNE arrange them. Even with just five points, you’ll see \"cat\" and \"dog\" land near each other, and \"car\" and \"bus\" group up. \"Apple,\" less related, stays apart. The two methods usually agree on clusters, but t-SNE's groupings look sharper.\n\nIf you use a larger dataset, t-SNE’s patterns will move around a bit between runs (unless you fix the random seed). For deeper experiments, try swapping in real pretrained embeddings—libraries like Gensim make this easy.\n\nVisualizations like this are a first step to understanding your model’s internal map of words. For more precision, try probing embeddings with arithmetic or nearest-neighbor searches. But as a tool for sanity checks, bias hunting, or demoing what word embeddings capture, PCA and t-SNE are invaluable.\n\nPick five words—some related, some unrelated (e.g., 'king', 'queen', 'man', 'woman', 'apple'). Use the provided code to plot their embeddings with both PCA and t-SNE. Examine the plots and write one sentence describing what you notice about how the words are grouped or separated.\n\n**Coming up on Day 16:** Cosine Similarity and Semantic Search Basics", "url": "https://wpnews.pro/news/day-15-visualizing-embeddings-with-t-sne-and-pca", "canonical_source": "https://dev.to/priyeshdave6/day-15-visualizing-embeddings-with-t-sne-and-pca-21b8", "published_at": "2026-09-10 09:05:35+00:00", "updated_at": "2026-09-10 09:23:04.273838+00:00", "lang": "en", "topics": ["machine-learning", "natural-language-processing", "artificial-intelligence", "developer-tools"], "entities": ["PCA", "t-SNE", "FastText", "Word2Vec", "GloVe", "scikit-learn", "matplotlib", "NumPy"], "alternates": {"html": "https://wpnews.pro/news/day-15-visualizing-embeddings-with-t-sne-and-pca", "markdown": "https://wpnews.pro/news/day-15-visualizing-embeddings-with-t-sne-and-pca.md", "text": "https://wpnews.pro/news/day-15-visualizing-embeddings-with-t-sne-and-pca.txt", "jsonld": "https://wpnews.pro/news/day-15-visualizing-embeddings-with-t-sne-and-pca.jsonld"}}