RAG in Laravel: Embeddings and pgvector for a Knowledge-Base Bot A developer built a retrieval-augmented generation (RAG) pipeline in Laravel 11 using PostgreSQL and the pgvector extension, enabling a knowledge-base chatbot without external vector database services. The system stores document embeddings locally and retrieves relevant chunks for each query, addressing the problem of AI models lacking access to custom data. The developer also notes that for small knowledge bases (under 2,000 tokens), embedding the entire corpus in the system prompt with prompt caching is simpler and more effective. In the last post https://dev.to/adityakdevin we streamed AI responses over SSE. Now let's fix the problem every chatbot hits in week one: the model doesn't know your data. Ask it about your refund policy or your API docs and it either hallucinates or shrugs. RAG — retrieval-augmented generation — fixes that. Instead of hoping the model knows your content, you store your documents as embeddings, retrieve the few chunks relevant to each question, and hand only those to the model. Here's the whole pipeline in Laravel 11 with PostgreSQL and pgvector — no vector-database SaaS required. I recently shipped a chatbot for my own portfolio site. Total knowledge base: about 2,000 tokens of facts and FAQ. I didn't build RAG for it — I put the entire corpus in the system prompt and let prompt caching make it cheap. Simpler code, zero retrieval bugs, and the model sees everything on every question. The rule of thumb I use with clients: If you're in the first bucket, close this tab and go ship. Still here? Let's build. pgvector extension openai-php/laravel for embeddings + chat pgvector/pgvector-php for the Eloquent cast composer require openai-php/laravel pgvector/pgvector php artisan vendor:publish --provider="OpenAI\Laravel\ServiceProvider" On most managed Postgres RDS, Supabase, Neon, Laravel Forge boxes pgvector is one statement away: CREATE EXTENSION IF NOT EXISTS vector; One table stores chunks of your documents plus their embedding vectors. text-embedding-3-small produces 1536 dimensions — good quality, cheap $0.02 per million tokens . php use Illuminate\Database\Migrations\Migration; use Illuminate\Support\Facades\DB; use Illuminate\Support\Facades\Schema; return new class extends Migration { public function up : void { DB::statement 'CREATE EXTENSION IF NOT EXISTS vector' ; Schema::create 'document chunks', function $table { $table- id ; $table- string 'source' ; // e.g. "refund-policy.md" $table- unsignedInteger 'position' ; // chunk order within the source $table- text 'content' ; $table- timestamps ; } ; DB::statement 'ALTER TABLE document chunks ADD COLUMN embedding vector 1536 ' ; // HNSW index makes similarity search fast past ~10k rows DB::statement 'CREATE INDEX ON document chunks USING hnsw embedding vector cosine ops ' ; } }; The model: php namespace App\Models; use Illuminate\Database\Eloquent\Model; use Pgvector\Laravel\Vector; class DocumentChunk extends Model { protected $fillable = 'source', 'position', 'content', 'embedding' ; protected $casts = 'embedding' = Vector::class ; } Chunking strategy matters more than people admit. My default: split on headings/paragraphs, target ~500 tokens per chunk, and never split mid-sentence. Fancy overlap windows can wait until you have evidence you need them. php namespace App\Console\Commands; use App\Models\DocumentChunk; use Illuminate\Console\Command; use Illuminate\Support\Facades\File; use OpenAI\Laravel\Facades\OpenAI; class IngestDocs extends Command { protected $signature = 'rag:ingest {path : Directory of markdown files}'; public function handle : int { foreach File::files $this- argument 'path' as $file { $chunks = $this- chunk $file- getContents ; // One API call per file, not per chunk — batch input is supported $response = OpenAI::embeddings - create 'model' = 'text-embedding-3-small', 'input' = $chunks, ; DocumentChunk::where 'source', $file- getFilename - delete ; foreach $response- embeddings as $i = $embedding { DocumentChunk::create 'source' = $file- getFilename , 'position' = $i, 'content' = $chunks $i , 'embedding' = $embedding- embedding, ; } $this- info "{$file- getFilename }: " . count $chunks . ' chunks' ; } return self::SUCCESS; } / @return string / private function chunk string $text, int $targetChars = 2000 : array { $paragraphs = preg split '/\n{2,}/', $text ; $chunks = '' ; foreach $paragraphs as $p { $current = array key last $chunks ; if strlen $chunks $current + strlen $p $targetChars && $chunks $current == '' { $chunks = $p; } else { $chunks $current .= "\n\n" . $p; } } return array map 'trim', $chunks ; } } Run it: php artisan rag:ingest storage/docs . Re-running replaces a file's chunks, so ingestion is idempotent — wire it to your deploy or a scheduled job and content stays fresh. pgvector's <= operator is cosine distance. Lower = more similar. Embed the user's question, sort by distance, take the top handful: php namespace App\Services; use App\Models\DocumentChunk; use OpenAI\Laravel\Facades\OpenAI; class Retriever { / @return array{content: string, source: string} / public function search string $question, int $limit = 5 : array { $embedding = OpenAI::embeddings - create 'model' = 'text-embedding-3-small', 'input' = $question, - embeddings 0 - embedding; return DocumentChunk::query - selectRaw 'content, source, embedding <= ? AS distance', json encode $embedding - orderBy 'distance' - limit $limit - get - filter fn $c = $c- distance < 0.55 // relevance floor — tune on real queries - map fn $c = 'content' = $c- content, 'source' = $c- source - all ; } } That distance floor is your hallucination guard. If nothing scores under the threshold, the honest answer is "I don't know" — and you should say exactly that instead of feeding the model junk context. php namespace App\Services; use OpenAI\Laravel\Facades\OpenAI; class KnowledgeBot { public function construct private Retriever $retriever {} public function answer string $question : string { $chunks = $this- retriever- search $question ; if $chunks === { return "I couldn't find that in our documentation — try rephrasing, or contact support."; } $context = collect $chunks - map fn $c = " {$c 'source' } \n{$c 'content' }" - implode "\n\n---\n\n" ; $response = OpenAI::chat - create 'model' = 'gpt-4o-mini', 'max tokens' = 500, 'messages' = 'role' = 'system', 'content' = <<