# How I Built a Secure, 3,072-Dim AI Document Indexer Using Next.js & Supabase.

> Source: <https://dev.to/dhritich20baruah/how-i-built-a-secure-3072-dim-ai-document-indexer-using-nextjs-supabase-1bbf>
> Published: 2026-05-20 03:20:19+00:00

Building a production-ready RAG (Retrieval-Augmented Generation) application from scratch is a very difficult and time consuming.

You first get a new idea for a project or business but before you can write a single line of your actual core business logic, you find yourself spending weeks fighting with multimodal file parsing, configuring vector extensions, designing complex database architecture, and securing multi-tenant storage.

The idea behind building DocuIntel was to solve that exact infrastructure headache. I wanted a template where I could just drop in my API keys, run a single setup script, and have a fully functioning, secure AI document portal ready to go.

Here is a deep dive into the architecture, some of the technical hurdles I ran into, and how I solved them.

**The Multimodal Architecture**

DocuIntel needs to process a wide variety of inputs—PDFs, Word docs, images (JPG/PNG), and even audio transcriptions (MP3s).

At first I was routing different file types through different third party parsing libraries like I was using tesseract OCR for extracting text from from images and pdf-parser for PDF files.

But instead of doing that, I offloaded the heavy lifting directly to Gemini 2.0 Flash. Gemini’s native multimodal capabilities handle OCR and layout analysis beautifully.

To ensure consistent, high-quality search results, I hardcoded a rigorous master SYSTEM_PROMPT inside the backend processing route (/api/process/route.ts). It forces the AI to behave like a clean data architect:

``` js
const SYSTEM_PROMPT = `
  You are an expert document parser for DocuIntel. 
  Your goal is to extract text with 100% accuracy for semantic search indexing.

  RULES:
  1. PRESERVE STRUCTURE: Use Markdown for headings, subheadings, and lists.
  2. TABLES: Convert all data tables into clean Markdown table format.
  3. NO CHAT: Do not say "Here is the text" or "I have processed the file." 
  4. NOISE REDUCTION: Ignore headers, footers, and page numbers.
  5. SMART METADATA: At the very end of your output, add a section called '---METADATA---' 
     and list 5-10 key entities or topics (e.g., 'Company: Acme Corp', 'Date: 2024-01-01').
`;
```

Why the Automated Metadata Tag Matters?

By forcing Gemini to extract smart metadata tags right into the text, the vector embedding model (gemini-embedding-2) captures high-value concepts. If a user searches for a specific company or date, the semantic search will surface the document even if that detail only appeared once in fine print.

**Navigating the New Supabase Security Standards**

Supabase has shifted to a "Secure by Default" model which will come into force by 30th May 2026.

Previously, creating a table in the public schema automatically exposed it to the Data API. Now, new tables require explicit grants, or your frontend client libraries (supabase-js) will get a 42501 permission error—even if Row-Level Security (RLS) is enabled.

To future-proof the setup, my SQL initialization script applies explicit grants directly to the authenticated user roles right after creating the tables and vector search RPC functions:

```
-- Create the documents table with 3,072-dimensional vector support
create table if not exists public.documents (
  id uuid primary key default uuid_generate_v4(),
  file_name text not null,
  file_url text not null,
  content text,               
  user_id uuid references auth.users(id),
  user_email text not null,   
  embedding vector(3072),     -- Optimized for high-res gemini-embedding-2
  created_at timestamp with time zone default timezone('utc'::text, now())
);

-- EXPLICIT GRANTS (Fixes 42501 Permission Errors)
grant select, insert, update, delete on table public.documents to authenticated;
grant all on table public.documents to service_role;
```

**Hardened Multi-Tenant Storage Policies**

Since the app deals with private user documents i.e. User A must never be able to discover or access User B's files, I have structured the Supabase Storage bucket so that every uploaded file is dynamically sandboxed into a folder named exactly after the user's unique authenticated ID (auth.uid()).

Here are the folder-level RLS storage policies that enforce that rule on every single request:

```
-- Restrict file uploads to the user's own UID folder
create policy "Users can upload their own documents"
on storage.objects for insert to authenticated
with check (
  bucket_id = 'documents' AND 
  (storage.foldername(name))[1] = auth.uid()::text
);

-- Restrict file reading to the user's own UID folder
create policy "Users can view their own documents"
on storage.objects for select to authenticated
using (
  bucket_id = 'documents' AND 
  (storage.foldername(name))[1] = auth.uid()::text
);
```

**Keeping the Frontend Clean & Readable**

On the frontend (Next.js 14 + TypeScript + Tailwind CSS), I wanted a highly scalable UI. A minor but common mess I see in boilerplate projects is massive, nested ternary operators in the JSX to handle dynamic file icons.

To keep things clean and performant, I grouped the file extension arrays and used standard JavaScript .includes() methods to dynamically assign Lucide React icons using a single class utility string:

Wrapping Up:

Building these layers from scratch took extensive testing, debugging environment variables, and reading through updated security documentation. But once it's configured, it works like magic: a user uploads an asset, Gemini reads and indexes it with a 3,072-dimension vector embedding, and you can instantly query your documents conceptually rather than just matching keywords.

If you are planning to build an AI document product for a client or launching your own micro-SaaS, you don't have to spend your weekend configuring this infrastructure from zero.

I've packaged this exact production-ready foundation—complete with the 1-click database initialization script, Next.js frontend, and pre-configured API routes—into a developer-friendly boilerplate.

Setting this up from scratch took me weeks of debugging. The boilerplate gets you there in 10 minutes.

[https://dhritiman.gumroad.com/l/docuintel](https://dhritiman.gumroad.com/l/docuintel)