# Still Confused About Embeddings? Start Here. > Source: > Published: 2026-07-20 22:01:01+00:00 Give me 10 minutes. I’ll make sure you understand every bit of embeddings you need — no fluff, no jargon, just the real thing. And yes, we’re building something. If you’ve ever read about RAG, vector databases, or semantic search, you’ve definitely come across the word embeddings. I did too. Everyone kept saying “just create embeddings and store them in a vector database.” But nobody really explained what those numbers actually were. So instead of reading another article, I built the simplest possible Node.js app to see them myself. That’s when everything finally clicked. By the end of this article you’ll understand why every piece of text becomes exactly one vector, why similar sentences end up close together, why RAG completely depends on embeddings, why chunking matters — and how to generate embeddings locally with Ollama. An embedding is a way of representing meaning as numbers. Specifically, a fixed-length list of floating point numbers — a vector — where the position of that vector in high-dimensional space encodes semantic meaning. But wait — what exactly is a vector? Think of a vector as a list of numbers. That’s it. Instead of storing the sentence itself, the model converts it into hundreds (or thousands) of numbers that represent its meaning. The individual numbers don’t matter. The relationship between all of them does. The key property that makes this useful: ``` "I love coffee" → [0.21, 0.84, 0.12, 0.67, ...]"I enjoy coffee" → [0.22, 0.83, 0.14, 0.65, ...] ← very close"quantum physics" → [0.91, 0.03, 0.78, 0.22, ...] ← far away ``` Similar meaning = vectors point in similar directions. This is what powers RAG, semantic search, recommendation engines, and duplicate detection. Not magic — just math. This is the part most people skip. But it’s what makes everything else make sense. Embedding models are trained using **contrastive learning**. The idea: ● Show the model pairs of sentences that mean the same thing → train it to produce similar vectors ● Show the model pairs that mean different things → train it to produce distant vectors The training data looks like this: **Positive pairs (should be close):** ``` "What is your return policy?" ↔ "How do I get a refund?""I have a headache" ↔ "My head hurts" ``` **Negative pairs (should be far apart):** ``` "What is your return policy?" ↔ "What time does the store open?""I have a headache" ↔ "The stock market crashed" ``` Over millions of these pairs, the model learns to compress meaning into vector space — semantically similar sentences end up near each other regardless of exact wording. Over time, the model isn’t memorizing sentences. It’s learning how language is used in the real world, so similar concepts naturally end up close together in vector space. This one trips a lot of people up. There are actually two types of embeddings. Imagine searching for *“cheap cars”*. **Dense embeddings** understand that *“affordable automobiles”* probably means the same thing — even though not a single word matches. They capture meaning. ``` "machine learning" → [0.21, 0.84, 0.12, 0.67, 0.43, ...] ↑ all 768 values are non-zero ``` **Sparse embeddings** don’t make that connection. But if someone searches for the exact phrase *“cheap cars”* — they’re excellent at that. That’s the core difference. And it’s why both exist. ``` "machine learning" → [0, 0, 0, 0.9, 0, 0, 0, 0.7, 0, 0, ...] ↑ only 2 dimensions lit up, rest are zero ``` Dense embeddings are excellent at capturing meaning and handling synonyms, making them best for semantic search and RAG — but poor at exact keyword matching. Sparse embeddings are the opposite: great for exact keyword matching and filtering, but they miss semantic similarity entirely, so “car” and “automobile” would score low together. This is exactly why **hybrid search** exists — combine both to get the best of both worlds. In our app today we use nomic-embed-text — that's a dense embedding model. The simplest possible app. Two files. One endpoint. ``` POST /embedBody: { "text": "I love coffee" } ``` Response: ``` { "input": "I love coffee", "dimensions": 768, "vector": [-0.16056, 0.55614, -3.49431, 0.26294, ...]} ``` I used **Ollama** with the nomic-embed-text model. Free, local, no API key needed. **embeddingService.js** ``` js const MODEL = "nomic-embed-text"async function generateEmbedding(text) { const response = await fetch('http://localhost:11434/api/embeddings', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ model: MODEL, prompt: text }) }) const json = await response.json() return json.embedding}module.exports = { generateEmbedding } ``` **server.js** ``` js const express = require('express')const embeddingService = require('./embeddingService')const PORT = 3000const app = express()app.use(express.json())app.post('/embed', async (req, res) => { const text = req.body.text if (!text) { return res.status(400).send('text is required') } const vector = await embeddingService.generateEmbedding(text) return res.json({ input: text, dimensions: vector.length, vector: vector })})app.listen(PORT, () => { console.log(`server listening on port ${PORT}`)}) ``` That’s literally the entire app. No vector database. No LangChain. No RAG. I wanted to remove every layer until all I could see was text becoming numbers. This is the part that completely changed how I think about embeddings. No matter how much text you send — one word or five hundred words — the embedding model always returns exactly **768 numbers**. Send *“Hi”* → 768 numbers. Send a full paragraph → 768 numbers. Send a single character → 768 numbers. Always 768. Always. Here’s what’s happening inside: ``` Input text ↓Tokenizer splits it into tokens["I", "love", "coffee"] ↓Each token gets its own temporary vector[0.21, 0.84, ...] ← "I"[0.54, 0.12, ...] ← "love"[0.87, 0.33, ...] ← "coffee" ↓Transformer layers process ALL tokens together(tokens attend to each other — "love" influences "coffee" etc.) ↓Pooling — all token vectors collapsed into ONE vector(mean pooling — average all token vectors) ↓Final output: one single vector of 768 numbersrepresenting the meaning of the WHOLE input ``` **Pooling** is the step that makes it click. Imagine asking five friends to summarise the same movie. Each friend notices different details. Pooling combines all those opinions into one overall summary. That’s essentially what the model is doing with token vectors — averaging all of them into one final vector. This is also why chunking matters in RAG — a 5000-word document embedded as one vector loses too much detail. Break it into smaller chunks so each vector captures focused meaning. 768 is specific to nomic-embed-text. Different models produce different sized vectors: The model you choose determines how many dimensions you get. nomic-embed-text from Nomic runs locally via Ollama and gives you 768 dimensions — which is what we used today. OpenAI's text-embedding-3-small gives 1536 dimensions and is great for cost-effective production use, while text-embedding-3-large goes up to 3072 for higher accuracy. If you're building with Claude, Anthropic's voyage-3 model gives 1024 dimensions. And if you need something lightweight and fast, all-MiniLM-L6-v2 from Sentence Transformers gives 384 dimensions and runs locally for free. More dimensions = more expressive = better accuracy. But also more storage and slower search. OpenAI’s text-embedding-3 models support **dimension reduction** — you can request fewer dimensions and trade accuracy for speed and cost: ``` js const response = await openai.embeddings.create({ model: "text-embedding-3-large", input: "your text here", dimensions: 512 // reduced from 3072 — faster, cheaper, slightly less accurate}) ``` Worth being clear on what they’re NOT: ● They do not generate text — they only produce vectors ● They are not LLMs — separate model, separate API call, separate cost ● They are not multilingual by default — check if your model supports multiple languages ● They can go stale — if you switch embedding models, you must re-embed your entire vector store That last point is a production gotcha. If you upgrade from text-embedding-3-small to text-embedding-3-large, you must re-embed every single document. The new model produces vectors in a different space — old and new vectors are incompatible and cannot be compared. These are the questions every beginner Googles after reading about embeddings: ❌ **Longer text means more dimensions.** No. A longer input creates more tokens, not a bigger embedding. The output is always the same fixed size. ❌ **Every number has its own meaning.** No. Meaning is spread across all dimensions together — no single number represents a concept. ❌ **Embeddings understand everything.** No. They only represent patterns learned during training. They can’t reason or infer beyond that. ❌ **More dimensions always mean a better model.** Not necessarily. Higher dimensions often improve quality, but training data and model architecture matter just as much. The honest answer: you can’t read them individually. No single number means anything on its own. The meaning is distributed across all 768 dimensions together. What you can do is **compare** two vectors. If two pieces of text have similar meaning, their vectors will be close to each other in 768-dimensional space. This is called **cosine similarity** — a score between -1 and 1: ● Close to 1 = very similar meaning ● Close to 0 = unrelated ● Close to -1 = opposite meaning That comparison is the foundation of everything — RAG retrieval, semantic search, recommendation engines. All of it is just: embed text, compare vectors, find the closest ones. Until now we’ve only been converting text into vectors. RAG simply takes that idea one step further. Every document chunk becomes a vector. Every user question becomes another vector. Retrieval is nothing more than finding the closest ones. ``` Chunk 1: "Refunds are processed in 5-7 days..." → [0.21, 0.84, ...] (768 numbers)Chunk 2: "Returns must be initiated within 30..." → [0.54, 0.12, ...] (768 numbers)Chunk 3: "Contact support at help@company.com..." → [0.87, 0.33, ...] (768 numbers) User query: "How long does a refund take?" → [0.22, 0.83, ...] (768 numbers) ↑ closest to Chunk 1 — correct retrieval ``` Without embeddings, RAG doesn’t exist. Before today, embeddings felt like one of those AI buzzwords everyone used but nobody explained. After building this tiny app, I realized they’re surprisingly simple. Text goes in. Numbers come out. Compare those numbers. That’s the foundation behind semantic search, recommendation systems, vector databases, and RAG. Sometimes the fastest way to understand AI isn’t reading another article. It’s building the smallest possible project and seeing what actually happens. Full code on GitHub: [embedding-app](https://github.com/PriyankaMali-13/AI/tree/master/embedding-app) *I’m Priyanka — Senior Software Engineer, AI learner, and someone who just watched three words turn into 768 numbers and found it oddly satisfying. I write about AI every day on this series. Follow along for Day 18.* *Technical details in this post are based on my own implementation and publicly available documentation. Results may vary across embedding models and versions.* [Still Confused About Embeddings? Start Here.](https://pub.towardsai.net/still-confused-about-embeddings-start-here-9828a2143b08) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.