{"slug": "your-documents-chunked-and-searchable-the-knowledge-base-in-bytechef", "title": "Your Documents, Chunked and Searchable: The Knowledge Base in ByteChef", "summary": "ByteChef has introduced a managed Knowledge Base feature that turns uploaded documents into a searchable RAG store, handling parsing, chunking, embedding, and indexing via a pgvector-backed Spring AI VectorStore. The feature offers multiple retrieval interfaces, including a Knowledge Base Search tool for AI agents, workflow actions, and a built-in search interface, while leaving deletion management to users.", "body_md": "TL;DR:AKnowledge Basein ByteChef is a managed RAG store: drop in documents (PDF, Markdown, Word, JSON, plain text - and scanned PDFs or images, through anOCRpath) and ByteChef parses,chunks(with size and overlap you pick per knowledge base),embeds, and indexes them in apgvector-backed Spring AI`VectorStore`\n\n. Retrieval is everywhere you'd want it: aKnowledge Base Search toolfor AI Agents (with tag filtering),Load/Search/Update/Delete actionsfor workflows, and a built-insearch interfacefor testing. Keeping the index current is yours to drive - re-run an ingestion workflow on whatever schedule you like - and the part to plan for deliberately is upstreamdeletes, which no refresh can spot on your behalf.\n\nEvery team building with agents arrives at the same sentence: *\"It should know our stuff.\"* The product docs. The runbooks. The policy PDFs. And the standard answer - RAG, retrieval-augmented generation - is conceptually simple: chunk the documents, embed the chunks, search by similarity, hand the best matches to the model.\n\nThe concept is a weekend. The *operation* is a project. Someone has to parse five file formats, pick chunk sizes, run an embedding model, host a vector database, keep the index in sync when the source-of-truth changes - and notice when a document is *deleted* upstream, because a knowledge base that confidently serves stale policy is worse than none at all. That's the gap between a RAG demo and a RAG system. ByteChef's **Knowledge Base** closes most of it for you - and this post is honest about the part it leaves to you.\n\nIn ByteChef, a Knowledge Base is a first-class object with its own home in the workspace: a named store of **documents**, each broken into **chunks**, each chunk embedded and indexed for semantic search. The layering is straightforward:\n\n`CL100K_BASE`\n\n, the same encoding GPT-4 uses, and prefers to break on sentence punctuation rather than mid-thought.`PgVectorStore`\n\nover a pgvector-enabled PostgreSQL database, built with the platform's configured `EmbeddingModel`\n\nand indexed with HNSW over cosine distance. There's no proprietary vector service to sign up for and no new operational vocabulary to learn: it's Postgres, backed up and monitored like the rest of your Postgres, and ByteChef creates and maintains the schema, the index, and the per-knowledge-base partitioning for you.And in the Knowledge Base workspace, none of this is a black box. You can open any document and **inspect its individual chunks** - and *edit* them, because sometimes the fix for a bad retrieval is one badly split paragraph - watch a document's indexing status, and, before any agent ever touches it, try queries against the KB in a built-in **search interface** to see exactly what retrieval returns.\n\nThere are three doors in, matched to three situations:\n\nWhichever door you use, everything downstream is identical - parse, chunk, embed, index. What differs is who pushes the button, and how often.\n\nAn index is a snapshot, and snapshots rot. Re-running ingestion is the easy half: put the **Load Data** workflow on a schedule and new and changed documents flow in on whatever cadence suits the material - nightly for a handbook, hourly for a status page.\n\nThe hard half is the one that quietly poisons RAG systems: **deletions**. A refresh that asks the upstream for \"everything that changed since Tuesday\" gets back the records that still exist - never the ones that vanished. Left alone, a document deleted at the source lives on in your knowledge base indefinitely, still retrieved, still quoted by your agent, long after somebody retracted it.\n\nThat half is yours to own today - ByteChef has no \"connect a source and forget it\" option that derives deletions for you. It's worth owning deliberately rather than discovering later, and the shape of the fix is simple enough: tag each ingested document with its upstream id, and on the runs where you can enumerate the full current upstream set, use the **Delete Documents** action - it removes documents by metadata filter - to drop whatever is no longer in it. Unglamorous, and far cheaper than the alternative, which is an agent confidently citing a policy that was withdrawn a month ago.\n\nA knowledge base earns its keep at retrieval time, and ByteChef exposes it on every surface:\n\nOne store, several ways in and out - and because retrieval is tag-filterable, a single knowledge base can serve multiple audiences (\"customer-facing\" vs. \"internal-only\") without maintaining parallel copies.\n\nIf you want to tune retrieval *itself* - query rewriting, expansion, custom joins over several stores - that's a different layer: ByteChef's RAG cluster elements, which work over any vector store you like. The Knowledge Base is the batteries-included end of the same spectrum: you don't configure retrieval, you just ask it questions.\n\nBy now you know the shape of this section. The primitives are Spring AI's; the product around them is ByteChef's:\n\n`PgVectorStore`\n\n+ `EmbeddingModel`\n\nare yours to configure and operateSame foundation as everything in this series - Spring AI under the hood - with the operational half of RAG, the half that actually hurts, absorbed into the platform.\n\nIf you're self-hosting, the Knowledge Base ships **disabled** - two switches turn it on:\n\n```\nbytechef:\n  ai:\n    knowledge-base:\n      enabled: true\n    vectorstore:\n      provider: pgvector\n      pgvector:\n        url: jdbc:postgresql://localhost:5433/bytechef_vectorstore\n        username: postgres\n        password: postgres\n```\n\nThat second block is the part worth planning for. The vector index lives in its **own** pgvector-enabled PostgreSQL database, separate from ByteChef's application database - the development compose file runs one for you alongside the main Postgres, and in production it's a second database to provision, monitor, and back up. Still Postgres, still nothing exotic, but not the *same* Postgres. ByteChef creates the schema and the index inside it on first use.\n\nOne more prerequisite, easy to miss: an **embedding model** has to be active for the environment. Without one there's nothing to turn chunks into vectors with, and the Knowledge Base page says so outright rather than quietly indexing nothing.\n\nThe distance between \"agents are impressive\" and \"agents are useful *here*\" is almost always knowledge - your documents, your data, your vocabulary. RAG closes that distance in principle; in practice it comes bundled with a parsing zoo, a vector database, an embedding budget, sync jobs, and the deletion problem nobody budgets for.\n\nByteChef's Knowledge Base packages most of it: documents in through upload, a workflow action, or an agent; chunks and vectors in pgvector; retrieval exposed as agent tools, workflow actions, and a human search box. What stays with you is the refresh policy - how often to re-run ingestion, and what to do about the deletes. Your agents get something to know, and you skip almost all of the machinery of knowing.\n\n*Have a folder of PDFs your agent should be quoting? Open ByteChef, create a **Knowledge Base*", "url": "https://wpnews.pro/news/your-documents-chunked-and-searchable-the-knowledge-base-in-bytechef", "canonical_source": "https://dev.to/bytechef/your-documents-chunked-and-searchable-the-knowledge-base-in-bytechef-h9d", "published_at": "2026-08-25 19:46:33+00:00", "updated_at": "2026-08-25 20:14:52.621295+00:00", "lang": "en", "topics": ["artificial-intelligence", "generative-ai", "ai-agents", "ai-infrastructure", "developer-tools"], "entities": ["ByteChef", "Spring AI", "pgvector", "PostgreSQL", "GPT-4", "CL100K_BASE", "HNSW"], "alternates": {"html": "https://wpnews.pro/news/your-documents-chunked-and-searchable-the-knowledge-base-in-bytechef", "markdown": "https://wpnews.pro/news/your-documents-chunked-and-searchable-the-knowledge-base-in-bytechef.md", "text": "https://wpnews.pro/news/your-documents-chunked-and-searchable-the-knowledge-base-in-bytechef.txt", "jsonld": "https://wpnews.pro/news/your-documents-chunked-and-searchable-the-knowledge-base-in-bytechef.jsonld"}}