How I Built a Secure, 3,072-Dim AI Document Indexer Using Next.js & Supabase.

Architecture of DocuIntel, a secure AI document indexer built with Next.js and Supabase that uses Gemini 2.0 Flash for multimodal parsing and a 3,072-dimensional vector embedding model. It details how the system enforces strict multi-tenant security through explicit SQL grants and user-specific storage folders to prevent unauthorized data access. The author also explains how a hardcoded system prompt forces the AI to extract clean Markdown text and smart metadata tags, improving semantic search accuracy.

Building a production-ready RAG Retrieval-Augmented Generation application from scratch is a very difficult and time consuming. You first get a new idea for a project or business but before you can write a single line of your actual core business logic, you find yourself spending weeks fighting with multimodal file parsing, configuring vector extensions, designing complex database architecture, and securing multi-tenant storage. The idea behind building DocuIntel was to solve that exact infrastructure headache. I wanted a template where I could just drop in my API keys, run a single setup script, and have a fully functioning, secure AI document portal ready to go. Here is a deep dive into the architecture, some of the technical hurdles I ran into, and how I solved them. The Multimodal Architecture DocuIntel needs to process a wide variety of inputs—PDFs, Word docs, images JPG/PNG , and even audio transcriptions MP3s . At first I was routing different file types through different third party parsing libraries like I was using tesseract OCR for extracting text from from images and pdf-parser for PDF files. But instead of doing that, I offloaded the heavy lifting directly to Gemini 2.0 Flash. Gemini’s native multimodal capabilities handle OCR and layout analysis beautifully. To ensure consistent, high-quality search results, I hardcoded a rigorous master SYSTEM PROMPT inside the backend processing route /api/process/route.ts . It forces the AI to behave like a clean data architect: js const SYSTEM PROMPT = You are an expert document parser for DocuIntel. Your goal is to extract text with 100% accuracy for semantic search indexing. RULES: 1. PRESERVE STRUCTURE: Use Markdown for headings, subheadings, and lists. 2. TABLES: Convert all data tables into clean Markdown table format. 3. NO CHAT: Do not say "Here is the text" or "I have processed the file." 4. NOISE REDUCTION: Ignore headers, footers, and page numbers. 5. SMART METADATA: At the very end of your output, add a section called '---METADATA---' and list 5-10 key entities or topics e.g., 'Company: Acme Corp', 'Date: 2024-01-01' . ; Why the Automated Metadata Tag Matters? By forcing Gemini to extract smart metadata tags right into the text, the vector embedding model gemini-embedding-2 captures high-value concepts. If a user searches for a specific company or date, the semantic search will surface the document even if that detail only appeared once in fine print. Navigating the New Supabase Security Standards Supabase has shifted to a "Secure by Default" model which will come into force by 30th May 2026. Previously, creating a table in the public schema automatically exposed it to the Data API. Now, new tables require explicit grants, or your frontend client libraries supabase-js will get a 42501 permission error—even if Row-Level Security RLS is enabled. To future-proof the setup, my SQL initialization script applies explicit grants directly to the authenticated user roles right after creating the tables and vector search RPC functions: -- Create the documents table with 3,072-dimensional vector support create table if not exists public.documents id uuid primary key default uuid generate v4 , file name text not null, file url text not null, content text, user id uuid references auth.users id , user email text not null, embedding vector 3072 , -- Optimized for high-res gemini-embedding-2 created at timestamp with time zone default timezone 'utc'::text, now ; -- EXPLICIT GRANTS Fixes 42501 Permission Errors grant select, insert, update, delete on table public.documents to authenticated; grant all on table public.documents to service role; Hardened Multi-Tenant Storage Policies Since the app deals with private user documents i.e. User A must never be able to discover or access User B's files, I have structured the Supabase Storage bucket so that every uploaded file is dynamically sandboxed into a folder named exactly after the user's unique authenticated ID auth.uid . Here are the folder-level RLS storage policies that enforce that rule on every single request: -- Restrict file uploads to the user's own UID folder create policy "Users can upload their own documents" on storage.objects for insert to authenticated with check bucket id = 'documents' AND storage.foldername name 1 = auth.uid ::text ; -- Restrict file reading to the user's own UID folder create policy "Users can view their own documents" on storage.objects for select to authenticated using bucket id = 'documents' AND storage.foldername name 1 = auth.uid ::text ; Keeping the Frontend Clean & Readable On the frontend Next.js 14 + TypeScript + Tailwind CSS , I wanted a highly scalable UI. A minor but common mess I see in boilerplate projects is massive, nested ternary operators in the JSX to handle dynamic file icons. To keep things clean and performant, I grouped the file extension arrays and used standard JavaScript .includes methods to dynamically assign Lucide React icons using a single class utility string: Wrapping Up: Building these layers from scratch took extensive testing, debugging environment variables, and reading through updated security documentation. But once it's configured, it works like magic: a user uploads an asset, Gemini reads and indexes it with a 3,072-dimension vector embedding, and you can instantly query your documents conceptually rather than just matching keywords. If you are planning to build an AI document product for a client or launching your own micro-SaaS, you don't have to spend your weekend configuring this infrastructure from zero. I've packaged this exact production-ready foundation—complete with the 1-click database initialization script, Next.js frontend, and pre-configured API routes—into a developer-friendly boilerplate. Setting this up from scratch took me weeks of debugging. The boilerplate gets you there in 10 minutes. https://dhritiman.gumroad.com/l/docuintel https://dhritiman.gumroad.com/l/docuintel