I removed the vector database from my AI agent stack

Moss, a sub-10 ms semantic search runtime, removes the need for a vector database in AI agent stacks by embedding search and embedding inside the application process, eliminating network hops. Benchmarks show Moss achieves 3.3 ms mean latency on 100,000 documents, compared to 358-596 ms for ChromaDB, Pinecone, and Qdrant. The runtime supports hybrid retrieval, built-in embeddings, and runs in the browser via WebAssembly.

Moss is a sub-10 ms semantic search runtime built for Conversational AI agents. Hybrid retrieval semantic + Keyword Search , built-in embeddings, metadata filtering, and a WebAssembly build that runs in the browser - all from a single SDK that embeds in your application. No network hop on the hot path. No clusters to tune. Point the SDK at Moss Cloud, load your index, and query it in under 10 ms . Python, TypeScript, Elixir, and C. Before you start: sign up at moss.dev https://moss.dev for project id and project key - free tier available. The snippets below need Python 3.10+ or Node.js 20+. pip install moss python from moss import MossClient, QueryOptions client = MossClient "your project id", "your project key" Create an index and add documents await client.create index "support-docs", {"id": "1", "text": "Refunds are processed within 3-5 business days."}, {"id": "2", "text": "You can track your order on the dashboard."}, {"id": "3", "text": "We offer 24/7 live chat support."}, Load and query — results in <10 ms await client.load index "support-docs" results = await client.query "support-docs", "how long do refunds take?", QueryOptions top k=3 for doc in results.docs: print f" {doc.score:.3f} {doc.text}" Returned in {results.time taken ms}ms npm install @moss-dev/moss js import { MossClient } from "@moss-dev/moss"; const client = new MossClient "your project id", "your project key" ; // Create an index and add documents await client.createIndex "support-docs", { id: "1", text: "Refunds are processed within 3-5 business days." }, { id: "2", text: "You can track your order on the dashboard." }, { id: "3", text: "We offer 24/7 live chat support." }, ; // Load and query — results in <10 ms await client.loadIndex "support-docs" ; const results = await client.query "support-docs", "how long do refunds take?", { topK: 3 } ; results.docs.forEach doc = { console.log ${doc.score.toFixed 3 } ${doc.text} ; // Returned in ${results.timeTakenInMs}ms } ; Most retrieval stacks call out to a remote vector database. The round trip alone runs 200–500 ms - enough to break a real-time conversation. Moss runs search and embedding inside your process. There's no network hop on the hot path, so query latency lands in the single digits - fast enough that retrieval disappears from the latency budget. If you're building a voice bot, a copilot, or any agent that talks to humans, that's the difference between a tool that feels alive and one that feels laggy. End-to-end query latency embedding + search on 100,000 documents, 750 measured queries, top k=5. Tested with Macbook pro M4 Pro, 24GB . | System | P50 | P95 | P99 | Mean | |---|---|---|---|---| Moss | 3.1 ms | 4.3 ms | 5.4 ms | 3.3 ms | | Pinecone | 432.6 ms | 732.1 ms | 934.2 ms | 485.8 ms | | Qdrant | 597.6 ms | 682.0 ms | 771.4 ms | 596.5 ms | | ChromaDB | 351.8 ms | 423.5 ms | 538.5 ms | 358.0 ms | Moss includes embedding in the measurement — competitors use an external embedding service modal https://modal.com/docs/examples/text embeddings inference . Pinecone and Qdrant use cloud search. Moss isn't a database It's a search runtime . You don't manage clusters, tune HNSW parameters, or worry about sharding. You index documents, load them into the runtime, and query. That's it. Sub-10 ms semantic search - single-digit-ms p99 in our benchmarks benchmarks Hybrid search - semantic + keyword in a single query Built-in embedding models - no OpenAI key required or bring your own Metadata filtering - $eq , $and , $in , $near operators Runs in the browser too - separate WebAssembly SDK for client-side semantic search with no server @moss-dev/moss-web Database connectors - ingest directly from SQLite, MongoDB, MySQL, and Supabase packages/moss-data-connector/ CLI - manage indexes and query from the terminal packages/moss-cli/ SDKs - Python 3.10+ , TypeScript / Node.js 20+ , Elixir, and C libmoss Framework integrations - LangChain, DSPy, LlamaIndex, Pipecat, LiveKit, Vapi, ElevenLabs, Strands Agents This repo contains working examples you can copy straight into your project: examples/ ├── python/ Python SDK samples │ ├── load and query sample.py │ ├── comprehensive sample.py │ ├── custom embedding sample.py │ └── metadata filtering.py ├── python-classification/ Classification example ├── javascript/ TypeScript SDK samples │ ├── load and query sample.ts │ ├── comprehensive sample.ts │ └── custom embedding sample.ts ├── javascript-web/ Browser / WASM SDK samples ├── c/ C SDK samples libmoss ├── go/ Go SDK samples ├── voice-agents/ End-to-end voice agents ambient + multi-agent │ ├── airline-pnr/ Ambient retrieval; per-PNR Moss indexes, swap mid-call │ └── mortgage-lending/ Multi-agent flow with shared session state └── cookbook/ Framework integrations ├── langchain/ LangChain retriever ├── dspy/ DSPy module ├── crewai/ CrewAI integration ├── haystack/ Haystack retriever ├── autogen/ AutoGen integration ├── mastra/ Mastra retriever ├── pydantic-ai/ Pydantic AI integration └── daytona/ Daytona sandbox example apps/ ├── next-js/ Next.js semantic search UI ├── pipecat-moss/ Pipecat voice agent with Moss retrieval ├── vapi-moss/ Vapi voice agent with Moss retrieval ├── elevenlabs-moss/ ElevenLabs voice agent with Moss retrieval ├── livekit-moss-vercel/ LiveKit voice agent on Vercel ├── agora-moss/ Agora Conversational AI MCP server with Moss retrieval ├── moss-llamaindex/ LlamaIndex RAG backend + frontend ├── moss-bun/ Bun runtime example └── docker/ Dockerized examples ECS/K8s pattern moss-live-labs/ Experimental zone: prototypes and community demos ├── python/ Minimal Python quickstart + advanced query example ├── typescript/ Minimal TypeScript quickstart + advanced query example ├── examples/ Larger experiments image search, voice agents │ ├── voice-agent/ LiveKit + Moss voice assistant │ ├── advanced-voice-agent/ Persona impersonator built on a PDF knowledge base │ └── image-search/ FastAPI + React image search over COCO └── community-demos/ Community-contributed projects └── voice-agents/ bharat-benefits, shoplabs-voice-agent cd examples/python pip install -r requirements.txt cp ../../.env.example .env Add your credentials python load and query sample.py cd examples/javascript npm install cp ../../.env.example .env Add your credentials npx tsx load and query sample.ts cd apps/next-js npm install cp ../../.env.example .env Add your credentials npm run dev Open http://localhost:3000 Sub-10 ms retrieval plugged into Pipecat's https://github.com/pipecat-ai/pipecat real-time voice pipeline — a customer support agent that actually keeps up with conversation. cd apps/pipecat-moss/pipecat-quickstart See README for setup and Pipecat Cloud deployment A privacy-first voice AI stack: Ollama for LLM inference, Moss for retrieval, Pipecat for real-time audio - the LLM and retrieval both run on your machine. cd apps/pipecat-moss/ollama-local docker compose up Full API reference: docs.moss.dev https://docs.moss.dev . | Framework | Status | Example | |---|---|---| | examples/cookbook/langchain/ DSPy https://github.com/stanfordnlp/dspy examples/cookbook/dspy/ LlamaIndex https://github.com/run-llama/llama index apps/moss-llamaindex/ CrewAI https://github.com/crewAIInc/crewAI examples/cookbook/crewai/ AutoGen https://github.com/microsoft/autogen examples/cookbook/autogen/ Haystack https://github.com/deepset-ai/haystack examples/cookbook/haystack/ Mastra https://mastra.ai examples/cookbook/mastra/ Pydantic AI https://ai.pydantic.dev examples/cookbook/pydantic-ai/ Pipecat https://github.com/pipecat-ai/pipecat apps/pipecat-moss/ LiveKit https://github.com/livekit/livekit apps/livekit-moss-vercel/ Vapi https://vapi.ai apps/vapi-moss/ ElevenLabs https://elevenlabs.io apps/elevenlabs-moss/ Agora https://www.agora.io/ apps/agora-moss/ Strands Agents https://github.com/strands-agents/sdk-python packages/strands-agents-moss/ Next.js https://nextjs.org apps/next-js/ VitePress https://vitepress.dev packages/vitepress-plugin-moss/ Vercel AI SDK https://sdk.vercel.ai packages/vercel-sdk/ Three parts: Moss Cloud - handles ingestion, document embedding, storage, and distribution. Point the SDK at it with a project ID and key. Index - your documents and their vectors, packaged as a single artifact that lives on Moss Cloud. Runtime - embedded in your application. It pulls indexes over HTTPS, holds them in memory, and serves queries locally. Once an index is loaded, queries don't leave your process - that's where the sub-10 ms latency comes from. Document changes flow through Moss Cloud and the runtime stays in sync. Server-side - moss Python and @moss-dev/moss Node.js 20+ embed the runtime in your backend. Use this when your agent runs on a server. Browser - @moss-dev/moss-web is a WebAssembly build that downloads the index and runs queries entirely client-side, no server required. Use this for static sites, browser extensions, and offline-first apps. See. examples/javascript-web/ Full Python SDK source code is available at sdks/python/ /usemoss/moss/blob/main/sdks/python . Here's where the community can have the most impact: New SDK bindings — Swift, Go, Elixir,... Framework integrations — CrewAI, Haystack, AutoGen Reranking support — plug in cross-encoder rerankers Doc-parsing connectors — PDF, DOCX, HTML, Markdown ingestion Examples and tutorials — if you build something with Moss, we'd love to feature it See our Contributing Guide /usemoss/moss/blob/main/CONTRIBUTING.md for setup instructions and our Roadmap /usemoss/moss/blob/main/ROADMAP.md for what's planned. Check out issues labeled good first issue https://github.com/usemoss/moss/labels/good%20first%20issue to get started. Discord https://moss.link/discord — ask questions, share what you're building GitHub Issues https://github.com/usemoss/moss/issues — bug reports and feature requests Twitter https://x.com/usemoss — announcements and updates BSD 2-Clause License /usemoss/moss/blob/main/LICENSE — the SDKs, examples, and integrations in this repo are fully open source. Built by the team at