Semantic Search Without an LLM. Build Blazing-Fast Semantic Search for \$0

A developer published a tutorial for building a zero-cost semantic search system that avoids large language models entirely, combining Hugging Face's Inference API for 384-dimension embeddings with an in-memory Qdrant vector cluster and a FastAPI backend. The writeup argues that generative AI search is slow and costly, citing claimed research that over 80% of programmers rely on generic AI assistants and that 80% of average product functionality ties back to basic search. The stack exposes /upload and /search endpoints with optional category filtering and reports sub-millisecond query latency.

At the moment, we are in the period of "AI search" which is the use of effective and slow Generative LLMs to answer basic queries. But we are approaching the "generative AI fatigue" turning point. According to the global ecosystem research, there are more than 80% of programmers addicted to generic AI assistants. Constant cravings for confirmation led to the sharp decline of the trust in generative effectiveness. Even major corporations have limited the use of generative technology. This is evident in the internal policies of the companies like Amazon which make their engineers avoid using any third-party sources. The explanation of this situation can be traced back to the Pareto principle in Product Design. The research conducted by Pendo has demonstrated that 80% of the average product functionality is connected to the basic search process. User Text Data ── Hugging Face Inference API ── Dense Vectors 384 Dimensions │ ▼ GitHub Pages Frontend <── Sub-Millisecond Search <── Qdrant In-Memory Cluster app.py This FastAPI script sets up a temporary Qdrant database and monitors incoming payloads actively. It retrieves access tokens from the environment and provides the routes needed for the bulk upload of documents /upload and for searching for vector matches /search . python import os import json import requests from fastapi import FastAPI, UploadFile, File from fastapi.middleware.cors import CORSMiddleware from qdrant client import QdrantClient, models app = FastAPI Secure Cross-Origin Resource Sharing to accept external web traffic smoothly app.add middleware CORSMiddleware, allow origins= " " , allow credentials=True, allow methods= " " , allow headers= " " , Safely loading the Hugging Face Token from environment variables HF TOKEN = os.getenv "HF TOKEN" API URL = "https://huggingface.co" headers = {"Authorization": f"Bearer {HF TOKEN}"} def get embeddings texts: list : response = requests.post API URL, headers=headers, json={"inputs": texts} return response.json Initializing an un-mapped, highly isolated Qdrant vector collection inside system RAM client = QdrantClient location=":memory:" COLLECTION NAME = "enterprise docs" client.create collection collection name=COLLECTION NAME, vectors config=models.VectorParams size=384, distance=models.Distance.COSINE @app.post "/upload" async def upload data file: UploadFile = File ... : try: contents = await file.read data = json.loads contents.decode "utf-8" documents = item "text" for item in data metadata payload = {"category": item.get "category", "general" , "document": item "text" } for item in data generated ids = list range 1, len documents + 1 vector lists = get embeddings documents if isinstance vector lists, dict and "error" in vector lists: return {"status": "error", "message": f"HF API Error: {vector lists 'error' }"} client.upload collection collection name=COLLECTION NAME, vectors=vector lists, payload=metadata payload, ids=generated ids return {"status": "success", "message": f"Successfully indexed {len documents } chunks "} except Exception as e: return {"status": "error", "message": str e } @app.get "/search" async def search data query: str, category: str = "All Categories" : query filter = None if category and category = "All Categories": query filter = models.Filter must= models.FieldCondition key="category", match=models.MatchValue value=category try: query vector = get embeddings query search result = client.query points collection name=COLLECTION NAME, query=query vector, query filter=query filter, limit=3 .points results = for hit in search result: results.append { "score": round hit.score, 4 , "category": hit.payload.get "category", "general" , "text": hit.payload.get "document", "" } return {"status": "success", "results": results} except Exception as e: return {"status": "error", "message": str e } @app.get "/" def home : return {"message": "Zero-LLM Search API is Live "} requirements.txt To launch this service, keep your dependencies as lean as possible: fastapi uvicorn python-multipart qdrant-client requests knowledge base.json To check the vector grouping capability, you can load this structured data template snippet directly into the interface: { "text": "To secure your API endpoints, always configure JWT authentication middleware to intercept incoming requests and validate the token signature.", "category": "security" }, { "text": "OAuth2 authorization code flows with PKCE are highly recommended for single-page applications and mobile apps to securely exchange tokens.", "category": "security" }, { "text": "For low-latency API verification, store client API keys in a Redis caching backend with a 5-minute TTL to drastically reduce primary database strain.", "category": "devops" }, { "text": "When deploying code, configure an automated CI/CD pipeline inside GitHub Actions to trigger unit testing with the Jest framework on every pull request.", "category": "testing" } When using the application structure in this manner, questions like "how do I safeguard my routing configurations?" can be achieved with just a few clicks. The solution interprets context and meaning but makes use of the AI algorithms in a reasonable way. The application functions on public cloud services, and thus the provider puts the Python engine to sleep mode when no activity happens within a period of time. When users try to send a request after using the application for some period, it takes the server around 30-60 seconds to wake up. Drop a star on the repo if you found this template helpful, and let me know in the comments how you plan to use serverless semantic search in your own apps