{"slug": "still-confused-about-embeddings-start-here", "title": "Still Confused About Embeddings? Start Here.", "summary": "Embeddings are fixed-length vectors of floating-point numbers that encode semantic meaning, enabling similar sentences to be mathematically close and powering RAG, semantic search, and recommendation systems. The author explains that embedding models are trained via contrastive learning on positive and negative sentence pairs, and distinguishes dense embeddings (capturing meaning and synonyms) from sparse embeddings (excelling at exact keyword matching).", "body_md": "Give me 10 minutes. I’ll make sure you understand every bit of embeddings you need — no fluff, no jargon, just the real thing. And yes, we’re building something.\n\nIf you’ve ever read about RAG, vector databases, or semantic search, you’ve definitely come across the word embeddings.\n\nI did too.\n\nEveryone kept saying “just create embeddings and store them in a vector database.”\n\nBut nobody really explained what those numbers actually were.\n\nSo instead of reading another article, I built the simplest possible Node.js app to see them myself.\n\nThat’s when everything finally clicked.\n\nBy the end of this article you’ll understand why every piece of text becomes exactly one vector, why similar sentences end up close together, why RAG completely depends on embeddings, why chunking matters — and how to generate embeddings locally with Ollama.\n\nAn embedding is a way of representing meaning as numbers. Specifically, a fixed-length list of floating point numbers — a vector — where the position of that vector in high-dimensional space encodes semantic meaning.\n\nBut wait — what exactly is a vector?\n\nThink of a vector as a list of numbers. That’s it.\n\nInstead of storing the sentence itself, the model converts it into hundreds (or thousands) of numbers that represent its meaning. The individual numbers don’t matter. The relationship between all of them does.\n\nThe key property that makes this useful:\n\n```\n\"I love coffee\"   → [0.21, 0.84, 0.12, 0.67, ...]\"I enjoy coffee\"  → [0.22, 0.83, 0.14, 0.65, ...]  ← very close\"quantum physics\" → [0.91, 0.03, 0.78, 0.22, ...]  ← far away\n```\n\nSimilar meaning = vectors point in similar directions.\n\nThis is what powers RAG, semantic search, recommendation engines, and duplicate detection. Not magic — just math.\n\nThis is the part most people skip. But it’s what makes everything else make sense.\n\nEmbedding models are trained using **contrastive learning**. The idea:\n\n● Show the model pairs of sentences that mean the same thing → train it to produce similar vectors\n\n● Show the model pairs that mean different things → train it to produce distant vectors\n\nThe training data looks like this:\n\n**Positive pairs (should be close):**\n\n```\n\"What is your return policy?\" ↔ \"How do I get a refund?\"\"I have a headache\"           ↔ \"My head hurts\"\n```\n\n**Negative pairs (should be far apart):**\n\n```\n\"What is your return policy?\" ↔ \"What time does the store open?\"\"I have a headache\"           ↔ \"The stock market crashed\"\n```\n\nOver millions of these pairs, the model learns to compress meaning into vector space — semantically similar sentences end up near each other regardless of exact wording.\n\nOver time, the model isn’t memorizing sentences. It’s learning how language is used in the real world, so similar concepts naturally end up close together in vector space.\n\nThis one trips a lot of people up. There are actually two types of embeddings.\n\nImagine searching for *“cheap cars”*.\n\n**Dense embeddings** understand that *“affordable automobiles”* probably means the same thing — even though not a single word matches. They capture meaning.\n\n```\n\"machine learning\" → [0.21, 0.84, 0.12, 0.67, 0.43, ...]                      ↑ all 768 values are non-zero\n```\n\n**Sparse embeddings** don’t make that connection. But if someone searches for the exact phrase *“cheap cars”* — they’re excellent at that.\n\nThat’s the core difference. And it’s why both exist.\n\n```\n\"machine learning\" → [0, 0, 0, 0.9, 0, 0, 0, 0.7, 0, 0, ...]                      ↑ only 2 dimensions lit up, rest are zero\n```\n\nDense embeddings are excellent at capturing meaning and handling synonyms, making them best for semantic search and RAG — but poor at exact keyword matching. Sparse embeddings are the opposite: great for exact keyword matching and filtering, but they miss semantic similarity entirely, so “car” and “automobile” would score low together.\n\nThis is exactly why **hybrid search** exists — combine both to get the best of both worlds.\n\nIn our app today we use nomic-embed-text — that's a dense embedding model.\n\nThe simplest possible app. Two files. One endpoint.\n\n```\nPOST /embedBody: { \"text\": \"I love coffee\" }\n```\n\nResponse:\n\n```\n{  \"input\": \"I love coffee\",  \"dimensions\": 768,  \"vector\": [-0.16056, 0.55614, -3.49431, 0.26294, ...]}\n```\n\nI used **Ollama** with the nomic-embed-text model. Free, local, no API key needed.\n\n**embeddingService.js**\n\n``` js\nconst MODEL = \"nomic-embed-text\"async function generateEmbedding(text) {    const response = await fetch('http://localhost:11434/api/embeddings', {        method: 'POST',        headers: { 'Content-Type': 'application/json' },        body: JSON.stringify({ model: MODEL, prompt: text })    })    const json = await response.json()    return json.embedding}module.exports = { generateEmbedding }\n```\n\n**server.js**\n\n``` js\nconst express = require('express')const embeddingService = require('./embeddingService')const PORT = 3000const app = express()app.use(express.json())app.post('/embed', async (req, res) => {    const text = req.body.text    if (!text) {        return res.status(400).send('text is required')    }    const vector = await embeddingService.generateEmbedding(text)    return res.json({        input: text,        dimensions: vector.length,        vector: vector    })})app.listen(PORT, () => {    console.log(`server listening on port ${PORT}`)})\n```\n\nThat’s literally the entire app.\n\nNo vector database. No LangChain. No RAG.\n\nI wanted to remove every layer until all I could see was text becoming numbers.\n\nThis is the part that completely changed how I think about embeddings.\n\nNo matter how much text you send — one word or five hundred words — the embedding model always returns exactly **768 numbers**.\n\nSend *“Hi”* → 768 numbers. Send a full paragraph → 768 numbers. Send a single character → 768 numbers.\n\nAlways 768. Always.\n\nHere’s what’s happening inside:\n\n```\nInput text    ↓Tokenizer splits it into tokens[\"I\", \"love\", \"coffee\"]    ↓Each token gets its own temporary vector[0.21, 0.84, ...] ← \"I\"[0.54, 0.12, ...] ← \"love\"[0.87, 0.33, ...] ← \"coffee\"    ↓Transformer layers process ALL tokens together(tokens attend to each other — \"love\" influences \"coffee\" etc.)    ↓Pooling — all token vectors collapsed into ONE vector(mean pooling — average all token vectors)    ↓Final output: one single vector of 768 numbersrepresenting the meaning of the WHOLE input\n```\n\n**Pooling** is the step that makes it click. Imagine asking five friends to summarise the same movie. Each friend notices different details. Pooling combines all those opinions into one overall summary. That’s essentially what the model is doing with token vectors — averaging all of them into one final vector.\n\nThis is also why chunking matters in RAG — a 5000-word document embedded as one vector loses too much detail. Break it into smaller chunks so each vector captures focused meaning.\n\n768 is specific to nomic-embed-text. Different models produce different sized vectors:\n\nThe model you choose determines how many dimensions you get. nomic-embed-text from Nomic runs locally via Ollama and gives you 768 dimensions — which is what we used today. OpenAI's text-embedding-3-small gives 1536 dimensions and is great for cost-effective production use, while text-embedding-3-large goes up to 3072 for higher accuracy. If you're building with Claude, Anthropic's voyage-3 model gives 1024 dimensions. And if you need something lightweight and fast, all-MiniLM-L6-v2 from Sentence Transformers gives 384 dimensions and runs locally for free.\n\nMore dimensions = more expressive = better accuracy. But also more storage and slower search.\n\nOpenAI’s text-embedding-3 models support **dimension reduction** — you can request fewer dimensions and trade accuracy for speed and cost:\n\n``` js\nconst response = await openai.embeddings.create({  model: \"text-embedding-3-large\",  input: \"your text here\",  dimensions: 512  // reduced from 3072 — faster, cheaper, slightly less accurate})\n```\n\nWorth being clear on what they’re NOT:\n\n● They do not generate text — they only produce vectors\n\n● They are not LLMs — separate model, separate API call, separate cost\n\n● They are not multilingual by default — check if your model supports multiple languages\n\n● They can go stale — if you switch embedding models, you must re-embed your entire vector store\n\nThat last point is a production gotcha. If you upgrade from text-embedding-3-small to text-embedding-3-large, you must re-embed every single document. The new model produces vectors in a different space — old and new vectors are incompatible and cannot be compared.\n\nThese are the questions every beginner Googles after reading about embeddings:\n\n❌ **Longer text means more dimensions.** No. A longer input creates more tokens, not a bigger embedding. The output is always the same fixed size.\n\n❌ **Every number has its own meaning.** No. Meaning is spread across all dimensions together — no single number represents a concept.\n\n❌ **Embeddings understand everything.** No. They only represent patterns learned during training. They can’t reason or infer beyond that.\n\n❌ **More dimensions always mean a better model.** Not necessarily. Higher dimensions often improve quality, but training data and model architecture matter just as much.\n\nThe honest answer: you can’t read them individually. No single number means anything on its own. The meaning is distributed across all 768 dimensions together.\n\nWhat you can do is **compare** two vectors. If two pieces of text have similar meaning, their vectors will be close to each other in 768-dimensional space. This is called **cosine similarity** — a score between -1 and 1:\n\n● Close to 1 = very similar meaning\n\n● Close to 0 = unrelated\n\n● Close to -1 = opposite meaning\n\nThat comparison is the foundation of everything — RAG retrieval, semantic search, recommendation engines. All of it is just: embed text, compare vectors, find the closest ones.\n\nUntil now we’ve only been converting text into vectors.\n\nRAG simply takes that idea one step further.\n\nEvery document chunk becomes a vector. Every user question becomes another vector. Retrieval is nothing more than finding the closest ones.\n\n```\nChunk 1: \"Refunds are processed in 5-7 days...\"  → [0.21, 0.84, ...] (768 numbers)Chunk 2: \"Returns must be initiated within 30...\"  → [0.54, 0.12, ...] (768 numbers)Chunk 3: \"Contact support at help@company.com...\"  → [0.87, 0.33, ...] (768 numbers)\nUser query: \"How long does a refund take?\"         → [0.22, 0.83, ...] (768 numbers)                                                              ↑                                          closest to Chunk 1 — correct retrieval\n```\n\nWithout embeddings, RAG doesn’t exist.\n\nBefore today, embeddings felt like one of those AI buzzwords everyone used but nobody explained.\n\nAfter building this tiny app, I realized they’re surprisingly simple.\n\nText goes in. Numbers come out. Compare those numbers.\n\nThat’s the foundation behind semantic search, recommendation systems, vector databases, and RAG.\n\nSometimes the fastest way to understand AI isn’t reading another article. It’s building the smallest possible project and seeing what actually happens.\n\nFull code on GitHub: [embedding-app](https://github.com/PriyankaMali-13/AI/tree/master/embedding-app)\n\n*I’m Priyanka — Senior Software Engineer, AI learner, and someone who just watched three words turn into 768 numbers and found it oddly satisfying. I write about AI every day on this series. Follow along for Day 18.*\n\n*Technical details in this post are based on my own implementation and publicly available documentation. Results may vary across embedding models and versions.*\n\n[Still Confused About Embeddings? Start Here.](https://pub.towardsai.net/still-confused-about-embeddings-start-here-9828a2143b08) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/still-confused-about-embeddings-start-here", "canonical_source": "https://pub.towardsai.net/still-confused-about-embeddings-start-here-9828a2143b08?source=rss----98111c9905da---4", "published_at": "2026-07-20 22:01:01+00:00", "updated_at": "2026-07-20 22:30:01.385626+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "natural-language-processing", "ai-tools", "developer-tools"], "entities": ["Ollama", "Node.js", "nomic-embed-text"], "alternates": {"html": "https://wpnews.pro/news/still-confused-about-embeddings-start-here", "markdown": "https://wpnews.pro/news/still-confused-about-embeddings-start-here.md", "text": "https://wpnews.pro/news/still-confused-about-embeddings-start-here.txt", "jsonld": "https://wpnews.pro/news/still-confused-about-embeddings-start-here.jsonld"}}