# Engineering Reliable AI Document Scanning: Prompts and Schemas > Source: > Published: 2026-08-22 19:32:16+00:00 Managing short-term rentals and homestays usually involves a chaotic mix of spreadsheets, WhatsApp messages, and scattered paper receipts. ** Propio** was built to replace that chaos with a purpose-built financial tracking platform. One of the most powerful features in Propio is **Smart Scan** — an AI-powered OCR system that allows property managers to drop a receipt, invoice, or booking confirmation into the app and have all the relevant financial fields extracted and categorized automatically. But building an AI that reads documents is easy; building one that extracts data *reliably enough for financial records* is incredibly difficult. If your AI hallucinates a tax amount or miscategorizes an expense, you haven't saved the user time—you've created a data integrity nightmare. Here is a deep dive into how Propio's document scanning architecture works, focusing on the critical importance of system prompts, output structuring, and fallback mechanisms. Document processing is slow. Relying on a synchronous HTTP request to wait for an LLM to parse a multi-page PDF is a recipe for browser timeouts and frozen UIs. Propio handles this asynchronously: `AgentTask` is created in MongoDB with a `PENDING` status, and the Task ID is returned to the client immediately.`asyncOcrAgentProcess` ).`COMPLETED` , the extracted data pre-fills the expense or booking form for a quick human review.This "fire-and-forget with polling" approach ensures the UI remains snappy regardless of how long the AI takes to "think." The biggest mistake when using LLMs for data extraction is asking for free-text or loosely formatted JSON. LLMs are eager to please and will often include conversational filler (e.g., *"Here is the extracted data:"*), which breaks standard JSON parsers. Propio enforces strict structured outputs at the API level using the `@google/genai` SDK's schema definition. ``` js const config = { responseMimeType: "application/json", responseSchema: { type: Type.OBJECT, properties: { recordDate: { type: Type.STRING, description: "This is receipt/invoice/billing date", }, category: { type: Type.STRING, enum: ExpenseCategories, // e.g., ["Electricity", "Water", "Maintenance"...] }, amount: { type: Type.NUMBER, }, paymentMode: { type: Type.STRING, enum: ExpensePaymentOptions, }, vendorName: { type: Type.STRING, }, error: { type: Type.STRING, nullable: true, }, }, }, systemInstruction: [{ text: expenseParsingSystemInstruction }], }; ``` By enforcing `responseMimeType: "application/json"` and providing a strict OpenAPI-style schema, we guarantee that the output will be parseable JSON matching our exact database requirements. We even pass our application's ENUMs (`ExpenseCategories` ) directly into the schema to ensure the AI categorizes the expense into a bucket our frontend actually supports. Even with a forced JSON schema, the AI needs strict behavioral boundaries. Propio's `expenseParsingSystemInstruction` acts as a rigid set of rules designed to prevent hallucination and enforce normalization. Financial data must be exact. The prompt explicitly states: ``` "Extract only data that is explicitly visible in the document. Never guess, infer, or fabricate values. If a field is missing or unclear, return null for that field." ``` Raw OCR data is messy. Dates come in various formats (MM/DD/YY, DD-MMM-YYYY), and amounts often include currency symbols or commas. We instruct the model to normalize this on the fly: `recordDate` → `YYYY-MM-DD` format.`amount` → numeric value only (remove currency symbols, commas, text).What happens if a user uploads a photo of their cat instead of a receipt? The agent needs an escape hatch. ``` FAILURE CONDITIONS: If the file is corrupted, blank, unsupported, not readable, or not a financial document, return only: { 'error': 'Seems like file is not a valid document' } ``` Because our schema includes a nullable `error` string, the AI can legally fulfill the JSON requirement while still rejecting the document. AI APIs go down. They get rate-limited. Models get overloaded. If your feature relies on a single model endpoint, your feature *will* break. To ensure near-zero downtime for the Smart Scan feature, Propio implements a multi-model cascade with exponential backoff. ``` js const Models = ["gemma-4-31b-it", "gemini-2.5-flash-lite"]; const generateContent = async (models, config, contents) => { const delayMs = 1000 * 2; const maxRetryPerModel = 2; for (const model of models) { for (let attempt = 1; attempt <= maxRetryPerModel; attempt++) { try { const response = await ai.models.generateContent({ model, config, contents, }); if (response.text) return response; } catch (error) { // If it's a 429 Rate Limit, break and try the next model immediately if (error instanceof ApiError && error.code === 429) { break; } // Otherwise, wait with exponential backoff and retry this model if (attempt <= maxRetryPerModel) { const waitTime = delayMs * Math.pow(2, attempt - 1); await sleep(waitTime); } } } } throw new Error("Model API call error. All models failed."); }; ``` This wrapper iterates through an array of preferred models. If the primary model fails due to a standard error, it retries with exponential backoff. Crucially, if it receives a `429 Too Many Requests` error, it immediately aborts retrying the current model and seamlessly cascades to the next fallback model. The user rarely notices a delay. Building reliable AI features isn't just about sending a prompt to an endpoint. It requires treating the AI as an unreliable function that needs strict guardrails. By combining **asynchronous processing** for UX, **strict JSON schemas** for structural integrity, **rigid system instructions** for data accuracy, and **multi-model cascading** for resilience, Propio turns AI document scanning from a novelty into a dependable tool for property managers.