Architecture Notes on VerusCite #
I have released an application, VerusCite, that checks academic documents for hallucinated references.
You can see an example of the output without signing up. While most public sector agencies will not be interested in that tool (it is more for editors or academics), I figured a post on Crime De-Coder discussing the architecture would be worthwhile.
Long story short, I made the application as cheap as possible (both for hosting and LLM processing). I will only need to change the architecture if I start processing thousands of articles a day. So if you want help building similar applications for your agency I am your guy.
I will detail two parts: first, how the agentic LLM processing works, and second, the actual hosting infrastructure.
LLM Processing
First, VerusCite is an actual agentic application. I know that is a buzzword, but the way this works is, you upload a document, it extracts the citations in step 1, in step 2 it does a check against an official database of documents (Crossref), and then if the citation was not found in step 2, it goes to an agent to do web search and verify the accuracy in step 3.
It is done in a way that is about as cheap as possible for the application to run. The citation extraction stage costs on average $0.02 to $0.03 per paper. The checking Crossref stage is free, and so the major costs of the program are via the agentic web search.
Each stage (including agentic web search) is conducted with cheaper models (either OpenAI's Luna or Gemini Flash Lite models). In total, it averages me around $0.50 per paper to process (which I have in my published benchmark data). Around half of the cost is web search, which is really a hard limit, as all providers of this range from $5 to $14 per 1000 searches. So using Perplexity web search saves me a few dollars over OpenAI or Google's native web search directly.
Hosting the Application
The application is hosted on a VPS (a Digital Ocean Droplet with 4 gigs of RAM). This costs me around $30 per month total. (I round up to $40, I pay $5 to email with Cloudflare, and I also have backups in S3, which are basically free.)
The biggest issue with this is throughput – I need to make sure not to overload my RAM. It is a Docker-deployed application mostly running Python code and a Postgres database. The front end is Flask (plus a Redis cache for some widgets on the site).
The backend is more interesting, as I need to take into account queuing. Queuing is needed both for memory management, as well as rate limiting with the APIs. So basically every service I have has rate limiting and a queue in a particular process. A single user has a queue (so if you upload two papers, the second paper does not start until the first is finished). This is all handled via multiple Celery queues that each have their own specified memory.
Since I have built the LLM processing to be very svelte, I do not need to worry about token limits. If I extract 100 citations in your paper, it actually does the citation-by-citation checking in parallel (the other queues basically handle rate limiting in the application).
Next Steps and Expansion
So this $40 per month application can currently handle a throughput of several thousand articles per day. So if I need to change the architecture, I will already be making a significant amount of revenue.
The next easiest step, if I needed to expand, would simply be buying a bigger VPS. I could essentially expand to 10k articles per day (or more) by simply increasing the size of the VPS RAM.
I am not sure the actual market for this is larger than that, but if that still became a problem, some of the sub-components can simply be isolated on different VPSs instead of deployed on a single machine. For example, Postgres could be hosted on a different machine. (In terms of database size, I would need to process high five figures of articles before I even need to worry about the size of Postgres as well).
I do not say all these things because I expect many of the police departments I work with at Crime De-Coder to care all that much about the technical details. I am giving these details as there are a ton of grifters in the AI space. Vibe coding has made it so neophytes can make a nice-looking website without a strong understanding of the underlying code.
If you need to deploy an agentic LLM application, make it secure, and meet throughput requirements, you should get in contact. It is one of the things I have developed strong expertise in over the course of my career.