{"slug": "add-ai-search-to-existing-application", "title": "Add AI search to existing application", "summary": "A developer demonstrated how to add semantic search to an existing Next.js todo app by storing OpenAI text-embedding-3-small vectors in a Postgres column via the pgvector extension, then ranking rows by distance in an order by clause. The approach replaces naive substring matching, which fails on queries like \"groceries\" against todos such as \"Purchase some apples,\" with embedding-based similarity search. The full feature amounts to a vector column, an embedding call on row creation, and an embedding of the search term at query time.", "body_md": "*How to add semantic search to an existing app using an embedding model and pgvector*\n\nAdding the most basic form of \"AI search\" to an existing app is three changes:\n\n- Add a\n**vector column** to the table you want to search.- When a row is created, send its text to an\n**embedding model** and store the numbers it returns in that column.- On search, embed the\n**search term the same way** and ask the database which rows are closest.\n\nI have a [super simple todo app](https://github.com/codegino/todo-list-with-ai/tree/starting-point) written in Next.js connected to a Postgres database.\n\nAside from the usual CRUD operations, this simple todo app has search. It was the naive one everybody writes first: lowercase the query, lowercase the title, check `includes`. Type *\"apple\"* and you get *\"Purchase some apples\"*; type *\"laundry\"* and you get *\"Laundry day\"*. It looks like it works, as long as you already know the words in the title.\n\nNow search that same list for *\"groceries\"*. Nothing, even though *\"Purchase some apples\"* and *\"Buy bread and eggs\"* are sitting right there. Same for *\"cleaning\"* against *\"Laundry day\"* and *\"Do the dishes\"*. The search is not looking at what the todos *mean*; it is looking at which letters they contain, and `groceries` is not a substring of anything.\n\nThat is the gap people reach for \"AI\" to fill. The surprise is how little is involved. The part that does the matching is not a model at all. It is arithmetic in your database.\n\nIf you would rather try the app than read about it, both versions are on GitHub:\n\nTo run either branch on your machine you only need two things: a Postgres database with `pgvector` support and an OpenAI API key. Put them in `.env` as `DATABASE_URL` and `OPENAI_API_KEY`, install the dependencies, and start the app.\n\nThe [diff between the two branches](https://github.com/codegino/todo-list-with-ai/compare/starting-point...with-ai-search) is, genuinely, the entire feature.\n\nForget training, weights, and prompts for a minute.\n\nAn **embedding model** is a function. Text goes in, a fixed-length list of numbers comes out:\n\n``` php\n\"Purchase some apples\"  ->  [0.021, -0.043, 0.118, ... ]   (1536 numbers)\n\"groceries\"             ->  [0.019, -0.038, 0.121, ... ]   (1536 numbers)\n```\n\nThink of those numbers as a location on a map. On a real map, two places with similar coordinates are close to each other. Same idea here, except this map has 1536 directions instead of two. You cannot picture that, and you do not need to. Only the rule matters: **text that means similar things ends up close together.**\n\n`1536` is not a universal number. It is just the output width of the model I picked. Other models give you 768, 1024, 3072, and some let you ask for a shorter output. Whatever you pick becomes part of your schema, so treat it as a decision and not a constant.\n\nSo \"Purchase some apples\" sits near \"groceries\" and far from \"renew passport\". Nobody programmed that. The model was trained on a very large amount of text, and that placement is the leftover shape of the language it read.\n\nHere is the part worth internalizing:\n\nOnce the text is numbers, matching is just measuring a distance. Your database does that. The AI ended at the point where you got the numbers back.\n\nThat is it. That is the whole trick. Everything below is plumbing.\n\nPostgres cannot store a list of 1536 floats usefully on its own, so we use [`pgvector`](https://github.com/pgvector/pgvector), an extension that adds a `vector` type and, crucially, distance operators that work in `order by`.\n\n```\n-- migrations/002_embeddings.sql\n\n-- pgvector is not part of stock Postgres. On Supabase it is available\n-- but not enabled until you ask for it.\ncreate extension if not exists vector;\n\n-- 1536 is the native output width of OpenAI's text-embedding-3-small.\nalter table todos add column if not exists embedding vector(1536);\n```\n\nPostgres is not the only place you can keep vectors, it just happens to be where my app already lived. I use Supabase, which is free and ships `pgvector` out of the box. If you run Postgres yourself, note that `docker run postgres:17` does not include the extension; use `pgvector/pgvector:pg17` instead. [Where to store embeddings](https://note.carlogino.com/ai/where-to-store-embeddings) compares the other options in my notes.\n\nSetting that up is not really part of this post, so it lives in my notes instead:\n\nEither way you end up with a `DATABASE_URL` and a database that understands `vector`. The rest of this post does not care which one you picked.\n\nTwo reasons, and both come up in any real app:\n\nThe entire \"AI dependency\" is one HTTP POST. No SDK required.\n\n``` js\n// src/lib/embeddings.ts\nconst ENDPOINT = 'https://api.openai.com/v1/embeddings';\n\nexport const EMBEDDING_MODEL = 'text-embedding-3-small';\n\nexport async function embed(text: string): Promise<number[]> {\n  const response = await fetch(ENDPOINT, {\n    method: 'POST',\n    headers: {\n      'Content-Type': 'application/json',\n      Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,\n    },\n    body: JSON.stringify({model: EMBEDDING_MODEL, input: text}),\n    // A hung provider must not pin a request open forever.\n    signal: AbortSignal.timeout(10_000),\n  });\n\n  if (!response.ok) {\n    throw new Error(`OpenAI embeddings failed: ${response.status}`);\n  }\n\n  const payload = (await response.json()) as {\n    data: {index: number; embedding: number[]}[];\n  };\n\n  return payload.data[0].embedding;\n}\n```\n\nThat is the AI in \"AI search\". A string in, an array of numbers out.\n\n`OPENAI_API_KEY` is the only credential involved. [Getting one is a five minute detour](https://note.carlogino.com/openai/get-an-openai-api-key-for-embeddings), and the API platform is billed separately from ChatGPT Plus, which trips up most people the first time.\n\nYou do not have to use OpenAI. Any embedding model works, as long as it takes text and gives back numbers. Google, Voyage, Cohere, or a model running locally through Ollama all fit in the same function. [The alternatives and what changes when you switch](https://note.carlogino.com/ai/alternatives-to-openai-embeddings) is its own note. The two things that move are the dimension count in your schema and the fact that every vector you have already stored becomes stale.\n\nOne small detail: `pgvector` accepts a vector written as a plain JSON array, so sending one from your code is just the array turned into a string, with an explicit `::vector` cast in the query. In the snippets below that is the `toVector` helper.\n\nThe important decision here is *where* the call goes. Creating a todo is the core feature; embedding it is not. If you `await` OpenAI before inserting the row, an OpenAI outage takes down todo creation.\n\nSo the row is inserted and returned first, and the embedding is written after the response has already gone out. Next.js gives you [`after()`](https://nextjs.org/docs/app/api-reference/functions/after) for exactly this:\n\n``` js\n// src/app/api/todos/route.ts\nimport {NextResponse, after} from 'next/server';\n\nimport {sql} from '@/db';\nimport {embed, toVector} from '@/lib/embeddings';\n\nexport async function POST(request: Request) {\n  const {title} = createTodoSchema.parse(await request.json());\n\n  const [created] = await sql`\n    insert into todos (title)\n    values (${title})\n    returning id, title, completed, created_at\n  `;\n\n  after(async () => {\n    try {\n      const embedding = await embed(created.title);\n\n      await sql`\n        update todos\n        set embedding = ${toVector(embedding)}::vector\n        where id = ${created.id}\n      `;\n    } catch (error) {\n      console.error(`Failed to embed todo ${created.id}`, error);\n    }\n  });\n\n  return NextResponse.json(created, {status: 201});\n}\n```\n\n`after()` is best-effort, not a queue. No retries, and if the process dies mid-callback the work is lost. That is fine for a demo and not fine for production. See the caveats at the end.\n\nThis is the symmetry that makes the whole thing work, and it is the one sentence I would want a reader to keep:\n\n**The search term goes through the exact same embedding call as the stored text.** Then you ask the database which stored vectors are nearest to that one.\n\n``` js\n// src/app/api/todos/search/route.ts\nimport {NextResponse} from 'next/server';\n\nimport {sql} from '@/db';\nimport {embed, toVector} from '@/lib/embeddings';\n\nconst MAX_DISTANCE = 0.6;\nconst MAX_RESULTS = 20;\n\nexport async function GET(request: Request) {\n  const q = new URL(request.url).searchParams.get('q')?.trim();\n\n  const queryVector = toVector(await embed(q));\n\n  const rows = await sql`\n    select\n      id, title, completed, created_at,\n      1 - (embedding <=> ${queryVector}::vector) as similarity\n    from todos\n    where embedding is not null\n      and (embedding <=> ${queryVector}::vector) < ${MAX_DISTANCE}\n    order by embedding <=> ${queryVector}::vector\n    limit ${MAX_RESULTS}\n  `;\n\n  return NextResponse.json(rows);\n}\n```\n\n`<=>` is pgvector's **cosine distance** operator. It answers one question about two lists of numbers: how far apart do they point?\n\n`0`: same direction, effectively the same meaning.` 1`: unrelated.` 2`: opposite.\nSo `order by embedding <=> $query` is literally \"closest first\", and `1 - distance` gives you a similarity between 0 and 1 that is friendlier to show and to reason about.\n\nNotice what is *not* in that query: no model, no prompt, no API call. By the time Postgres is involved, the AI part is over. This is ordinary maths over a column, and it is why the feature is fast and cheap to run.\n\nNearest-neighbour search has no concept of \"no results\". Ask it for the top 20 and it hands you 20 rows, however unrelated, confidently ranked. Without `MAX_DISTANCE`, searching for `asdfgh` returns your entire todo list.\n\n`0.6` is a magic number picked by eye. It depends on your data, since short todo titles behave nothing like paragraphs of prose. That is why the endpoint returns `similarity` on every result: run a few searches with `curl`, see where the useful results stop, and move the number.\n\nRemember the two searches that returned nothing at the start of the post? Here they are again, on the same todo list, with the old search and the new one side by side:\n\n``` php\n# keyword: does the title contain these letters?\n\"groceries\"  ->  (nothing)\n\"cleaning\"   ->  (nothing)\n\n# AI: which titles mean something close to this? (1 = identical meaning)\n\"groceries\"  ->  Purchase some apples    0.71\n                 Buy bread and eggs      0.68\n\n\"cleaning\"   ->  Do the dishes           0.66\n                 Laundry day             0.64\n```\n\nThe letters still do not match. `groceries` is nowhere in \"Purchase some apples\". But the two sit close together on that map, so the row comes back anyway, with a number telling you how close:\n\nNotice that I did not replace the old search. I added a button next to the search box that toggles \"AI\" search on and off, so you can run the same query both ways and see the difference. The toggle is the feature, and that is not just for the demo.\n\nVector search is bad at exact terms. Ticket IDs, product codes, names, acronyms, anything rare. Search for `TODO-1234` and it will happily return four todos that feel vaguely related and none that match. Substring search gets that right every time.\n\nSo the two are not rivals. Keyword wins on exact hits, vectors win on meaning, and keeping both is the honest setup. The usual next step is to stop making the user choose: run both and merge the results. That is called hybrid search.\n\nOne small detail: both modes only search when you submit, not as you type. Use whatever strategy you like here. I just did not want to fire an embedding call on every keystroke.\n\nThe search query skips rows where `embedding is null`, so a todo without an embedding is invisible to AI search. Two things put rows in that state:\n\nBoth are fixed the same way: select the rows with a null embedding, embed them, write the vectors back. Here it is as an API route, using an `embedMany` variant of the earlier `embed` function that sends an array in one request:\n\n``` js\n// src/app/api/todos/backfill/route.ts\nimport {NextResponse} from 'next/server';\n\nimport {sql} from '@/db';\nimport {embedMany, toVector} from '@/lib/embeddings';\n\nexport async function POST() {\n  const pending = await sql`\n    select id, title from todos\n    where embedding is null\n    order by created_at\n    limit 100\n  `;\n\n  // The API takes an array, so 100 todos is one round trip, not 100.\n  const embeddings = await embedMany(pending.map(todo => todo.title));\n\n  for (const [index, todo] of pending.entries()) {\n    await sql`\n      update todos\n      set embedding = ${toVector(embeddings[index])}::vector\n      where id = ${todo.id}\n    `;\n  }\n\n  return NextResponse.json({embedded: pending.length});\n}\ncurl -X POST localhost:3000/api/todos/backfill\n# {\"embedded\":7}\n```\n\nThe route is just the easiest trigger. A one-off script, a cron job, a queue worker, or a button in your admin page do the same job. The `limit` keeps one run bounded, so call it until it returns `0`.\n\n**An embedding is derived data.** It is a function of the text *and* the model that produced it.\n\nSwitch to `text-embedding-3-large`, shorten the output to 512 dimensions, or move to another provider, and every stored vector becomes stale. Nothing errors, because the old vectors still look like valid numbers. They are simply no longer comparable to the vectors your new queries produce, so search quietly gets worse.\n\nSo write the backfill as something you can run again, not as a one-time migration.\n\nBackfilling is the first time you send a lot of text to the model at once, so this is the right moment to talk about the bill.\n\nYou are billed per **input token**, and only on the way in. There is no output cost, because the output is a vector and not text.\n\nFor an app this size the numbers are barely real. A todo title is about 8 tokens, so embedding 10,000 of them is around 80,000 tokens, which is under a fifth of a cent on `text-embedding-3-small`. Searches are even smaller.\n\nSo the thing to watch is not the price per token. It is re-embedding text that did not change, embedding on every keystroke, and hitting the tokens-per-minute rate limit in the middle of a backfill. [How to count tokens and estimate the bill](https://note.carlogino.com/ai/counting-tokens-and-cost-for-embeddings) is in my notes, including the input length cap you will hit the moment your text is longer than a todo title.\n\nThis is a demo app, so here is what I skipped:\n\n`0.6` is tuned to my data, not yours.`where user_id = ...`, filtered vector search gets tricky, because the index finds the nearest rows globally and your filter then throws most of them away.\nThe demo app went from \"search only finds words you already typed correctly\" to \"search finds what you meant\" with one migration, one embedding call on insert, and one `order by distance` query. No new service, no separate search engine, no rewrite. The database you already have does the hard part.\n\nThe piece worth keeping in your head is that the model is doing one job: turning text into a position on a map. Everything else in this post is plumbing around that. Once the numbers are in a column, `groceries` finding *\"Purchase some apples\"* is just arithmetic.", "url": "https://wpnews.pro/news/add-ai-search-to-existing-application", "canonical_source": "https://dev.to/codegino/add-ai-search-to-existing-application-225f", "published_at": "2026-09-13 07:19:07+00:00", "updated_at": "2026-09-13 07:26:24.860077+00:00", "lang": "en", "topics": ["ai-tools", "ai-products", "developer-tools", "natural-language-processing", "ai-infrastructure"], "entities": ["Postgres", "pgvector", "OpenAI", "Next.js", "Supabase", "text-embedding-3-small", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/add-ai-search-to-existing-application", "markdown": "https://wpnews.pro/news/add-ai-search-to-existing-application.md", "text": "https://wpnews.pro/news/add-ai-search-to-existing-application.txt", "jsonld": "https://wpnews.pro/news/add-ai-search-to-existing-application.jsonld"}}