How I Handle 10,000 Email Generations per Day on a Shoestring Budget

A developer behind ColdMailAI built a model-routing, semantic-caching, and request-batching pipeline that generates over 10,000 personalized cold emails per day for roughly $505 a month. Routing about 70% of requests to cheaper models cut average cost per email from $0.02 to $0.004, while a vector-DB cache with a 0.92 similarity threshold hit about 40% and batching similar prompts reduced API calls by 60%. The stack runs on Vercel, Supabase, Pinecone, Upstash Redis, and OpenAI, keeping the product viable at $9 per user.

When I launched ColdMailAI https://getcoldmail.com , I had one goal: keep the price at $9/month and still handle serious volume. Six months later, we're generating over 10,000 personalized cold emails per day. Here's the exact technical playbook that makes it possible without burning through cash. Let's do the math. If you're calling GPT-4 for every email generation: At $9/month per user, you'd need 333-1,000 paying users just to cover API costs. That's before servers, domains, or your coffee budget. Not every email needs GPT-4-level intelligence. I built a routing layer that classifies generation requests: js function routeModel request { const complexity = analyzeComplexity request ; if complexity.requiresResearch { return 'gpt-4o'; // Deep personalization } if complexity.hasTemplate { return 'gpt-4o-mini'; // Template variation } return 'gpt-4o-mini'; // Simple generation } Result: 70% of requests go to cheaper models. Average cost dropped from $0.02 to $0.004 per email. Here's what most people miss: cold emails to similar prospects in the same industry follow similar patterns. I built a semantic cache: js async function generateWithCache prospect { const embedding = await getEmbedding prospect.context ; const cached = await vectorDB.search embedding, { threshold: 0.92 } ; if cached { // Modify cached result slightly instead of full generation return await cheapVariation cached.result, prospect ; } const result = await fullGeneration prospect ; await vectorDB.store embedding, result ; return result; } Cache hit rate after 2 weeks of operation: ~40% . Those hits cost almost nothing. Instead of processing emails one-by-one with individual API calls, I batch similar requests: php User Request - Redis Queue - Batch Aggregator every 30s | Group by industry/template | Single API call per batch | Distribute results back Batching similar prompts reduced my API calls by 60% while maintaining personalization quality. Here's what runs ColdMailAI in production: | Component | Choice | Monthly Cost | |---|---|---| | Hosting | Vercel serverless | $20 | | Database | Supabase free tier + $25 pro | $25 | | Vector DB | Pinecone starter | $0 | | Queue | Upstash Redis | $10 | | AI APIs | OpenAI with routing | ~$400 | | Email verification | ZeroBounce | $50 | | Domain/DNS | Cloudflare | $0 | | Monitoring | Sentry free tier | $0 | | Total | | ~$505/month | At $9/user with 200+ users, the margins actually work. The key is that costs scale sub-linearly because of caching and batching. This was the biggest unlock. Instead of generating a perfect email in one shot, I break it into stages: Users see Stage 1 results in under 2 seconds. Stages 2-3 run asynchronously, which spreads the computational load and lets me use cheaper batch processing. You don't need massive infrastructure funding to run an AI product at scale. Smart routing, aggressive caching, and batch processing got me from "this will bankrupt me" to "this is a real business" in about 3 months. If you're building something similar, I'm happy to answer questions in the comments. And if you want to see the end result, check out ColdMailAI https://getcoldmail.com - it's $9/month because I actually figured out how to make that price work. Building in public. Follow me for more technical deep dives on running an AI SaaS on a shoestring budget.