{"slug": "i-built-an-app-that-ocrs-every-screenshot-on-your-phone-here-s-the-architecture", "title": "I built an app that OCRs every screenshot on your phone — here's the architecture", "summary": "A developer built Screenshot Vault, an app that OCRs every screenshot on a phone and makes them searchable. The app uses on-device Google ML Kit for text recognition to keep images private, and sends only extracted text to Gemini for categorization. The developer chose a staged, resumable processing approach and moved the AI API key to a server-side proxy to avoid exposing it in the client.", "body_md": "We all have that folder of 500+ screenshots we can never find anything in. I got tired of scrolling through mine looking for a receipt from three months ago, so I built Screenshot Vault — an app that reads the text in every screenshot on your phone and lets you search it like Google.\n\n*Here's how it's built, and a few decisions that ended up mattering more than I expected.*\n\nThe stack\n\nExpo (React Native) + Expo Router — file-based routing, EAS for native builds\n\n@react-native-ml-kit/text-recognition — on-device OCR via Google ML Kit\n\nexpo-sqlite — local persistence\n\nexpo-media-library — reading the device's Screenshots album\n\nGemini 3.6 Flash — categorization, titles, tags, called through a small Next.js API route\n\nNext.js + Vercel — landing page + the backend proxy\n\n**Decision #1**: *OCR stays fully on-device*\n\nMy first instinct was to upload screenshots to a server for processing. Then I actually thought about what's in a typical screenshot folder — bank OTPs, payment confirmations, private chats, addresses. Uploading that to a database I control is a liability for me and a real risk for users if anything ever leaks.\n\nGoogle ML Kit's text recognition runs entirely on-device, for free, with no server round-trip. So OCR happens locally, and the only thing that ever leaves the phone is the already-extracted text — sent to Gemini for a lightweight categorization call, not the image itself. This also happens to be cheaper to run at scale, since I'm not paying for image storage or bandwidth.\n\n**Decision #2**: **Don't block the user on a full scan**\n\nIf someone installs this with 500 existing screenshots, running OCR + AI on all of them before showing anything would mean a 10-20 minute wait on first launch. That's an instant uninstall.\n\nInstead, processing is staged and resumable:\n\n```\nCREATE TABLE screenshots (\n  id TEXT PRIMARY KEY,\n  uri TEXT,\n  text_content TEXT,\n  category TEXT,\n  ai_title TEXT,\n  tags TEXT,\n  ocr_done INTEGER DEFAULT 0,\n  ai_done INTEGER DEFAULT 0,\n  created_at INTEGER\n);\n```\n\n**On open:**\n\n*If the app gets closed mid-scan*, **it just picks up from wherever the flags left off next time** — no reprocessing, no lost work. This is basically the same idea Google Photos uses for its \"preparing your library\" background indexing.\n\n**Decision #3**:**_ Never put the AI API key in the client_**\n\nEarly version called Gemini directly from the app with the key in an env var. Realized pretty quickly that anyone who installs the APK can extract that key and rack up usage on my bill — env vars prefixed EXPO_PUBLIC_ get bundled straight into the client, no exceptions.\n\nMoved it behind a single Next.js API route on the same Vercel project as the landing page:\n\n```\nexport async function POST(req: NextRequest) {\n  const { text } = await req.json();\n  const apiKey = process.env.GEMINI_API_KEY; // server-only, never shipped to client\n\n  const res = await fetch(GEMINI_ENDPOINT, {\n    method: 'POST',\n    headers: { 'x-goog-api-key': apiKey },\n    body: JSON.stringify({ /* prompt asking for category + title + tags in one call */ }),\n  });\n  // ...parse and return\n}\n```\n\nOne call returns category + title + tags together instead of three separate requests — matters a lot on Gemini's free tier, which caps out fast.\n\nStill early and Android-only right now, testing with a small group before a wider release. If you want to follow along or try it when it's ready: screenshot-vault-lac.vercel.app\n\nCurious if anyone's tackled similar on-device vs. cloud tradeoffs for AI features — happy to talk through any of this in the comments.", "url": "https://wpnews.pro/news/i-built-an-app-that-ocrs-every-screenshot-on-your-phone-here-s-the-architecture", "canonical_source": "https://dev.to/raj_chavan524/i-built-an-app-that-ocrs-every-screenshot-on-your-phone-heres-the-architecture-2ik3", "published_at": "2026-08-13 06:13:46+00:00", "updated_at": "2026-08-13 06:44:47.322473+00:00", "lang": "en", "topics": ["artificial-intelligence", "computer-vision", "ai-products", "developer-tools"], "entities": ["Screenshot Vault", "Google ML Kit", "Gemini", "Expo", "Next.js", "Vercel"], "alternates": {"html": "https://wpnews.pro/news/i-built-an-app-that-ocrs-every-screenshot-on-your-phone-here-s-the-architecture", "markdown": "https://wpnews.pro/news/i-built-an-app-that-ocrs-every-screenshot-on-your-phone-here-s-the-architecture.md", "text": "https://wpnews.pro/news/i-built-an-app-that-ocrs-every-screenshot-on-your-phone-here-s-the-architecture.txt", "jsonld": "https://wpnews.pro/news/i-built-an-app-that-ocrs-every-screenshot-on-your-phone-here-s-the-architecture.jsonld"}}