I Spent $5 on OpenRouter While Building AI Features for an NGO App

A developer building AI features for the PSS NGO website spent about $5 on OpenRouter during development. The project integrates a knowledge base with MongoDB and embeddings to control AI-generated content, and encountered issues with output limits and factual contradictions. The developer emphasizes the importance of application-level control over AI responses.

I've been working on the PSS website for quite some time, and recently a lot of my work went into its AI features. The application already handles sessions, events, staff, blog posts, gallery, testimonials, donations, settings, and an admin CMS. For the AI work, I wanted two things: The stack is Next.js, React, TypeScript, MongoDB, Mongoose, Clerk, Cloudinary, Tailwind CSS, shadcn/ui, TanStack Query and OpenRouter . During development, my OpenRouter dashboard reached about $4.99 in total spend . The $5 isn't the interesting part. What happened while building the features is. I didn't want the AI to answer PSS-related questions only from its general knowledge. I needed information that the application could control and update. So I built a knowledge flow around approved PSS content: Approved PSS knowledge ↓ Chunking ↓ Embeddings ↓ MongoDB ↓ Relevant knowledge retrieved ↓ AI context ↓ Response / content generation The stored knowledge includes the content plus details such as its source, authority, status and visibility. That also gives me a way to separate public knowledge from internal AI guidance and policies. The idea I ended up using is simple: The knowledge base decides what PSS teaches. The model decides how to communicate it. That became much more important once I started checking the generated content. The admin CMS also has AI-assisted blog creation. AI can help generate: I didn't want one button to generate an entire blog post from nothing. For smaller fields, the admin can run individual AI actions. Full article generation is a separate flow. The article generation can use the information already entered in the editor: Title Category Tags SEO title SEO description Excerpt Writing instruction Relevant PSS knowledge ↓ AI model ↓ Generated article ↓ Article editor This is much more useful than sending something generic like: "Write a blog post about meditation." The application already has context. The AI should use it. My first full-article generation didn't finish. The logs showed: model: openai/gpt-5-mini task: full article promptTokens: 1013 completionTokens: 1800 finishReason: length The application reported: AI generation reached the model's output limit. I increased: AI CONTENT MAX TOKENS from 1800 to 2000 , and after testing again, the generation completed. It was a small configuration issue, but it reminded me that an AI feature has limits that a normal API call doesn't have. The request can succeed while the output is still incomplete. The application needs to detect and handle that. The generated article looked good. That was actually the problem. When I compared it with the approved PSS knowledge, I found some contradictions. The approved information included specific guidance around: The generated article introduced more general meditation advice, including things like resting the hands in the lap or on the knees, allowing the eyes to remain open, and suggesting specific practice durations. Those suggestions may sound reasonable for meditation in general. But the website isn't publishing generic meditation advice. It represents PSS. So the output can be well written and still be wrong for the application . That was the most useful thing I found during testing. Before this work, I mostly thought of retrieval as a way to give the model better information. Now I also see it as a way to keep the application's content under control. The embeddings help find relevant information, but the application still needs to know: So it isn't simply: documents → embeddings → model There is an application around it. Getting the backend generation working wasn't enough. While using the admin workflow, I also found that the AI actions needed to behave like normal application features. I worked on things such as: Adding AI doesn't remove normal UX problems. It gives you a few more. My OpenRouter dashboard showed approximately: The dashboard also showed usage from GPT-5.4, GPT-5 Mini and Text Embedding 3 Small . One clarification: 73.5M is the token-volume metric shown by the dashboard . I wouldn't describe it as 73.5M tokens of generated text. Most of this usage came from development, testing AI flows, model calls and embeddings. The useful part wasn't getting a lot of AI work done for $5. It was being able to see what experimentation actually costs. The 94.7% cache hit rate also made me think more about repeated context and what these costs could look like with more traffic. I'm trying not to judge an AI feature only by whether the API returns a successful response. I also need to ask: Some of these are things I've already encountered during development. Others are areas I still want to make more systematic with testing. The AI system isn't finished. If I started this part of the project again, I'd think about the knowledge and failure cases earlier. Not only: Which model should I use? But also: What information should the AI use? What information should it never use? What happens when there is no relevant information? What happens when the model doesn't finish? How do I know the generated content is acceptable? The model is only one part of the feature. The rest is application engineering. The biggest thing I learned wasn't about a particular model. It was that adding AI to an existing application means dealing with a lot more than an API call: The AI can generate a very good-looking article. I still have to ask: Is this actually what we want the application to say? For this project, that question matters more than whether the model produced a polished paragraph. You can see the application here: If you're building AI into an existing application, I'd be interested to hear what surprised you after the first successful API response.