{"slug": "i-used-hindsight-to-remember-human-invoice-decisions", "title": "I Used Hindsight to Remember Human Invoice Decisions", "summary": "A developer built VendorSense, an accounts-payable agent that combines PDF invoice extraction, Groq-based LLM reasoning, and a Hindsight memory layer so that prior human approval and rejection decisions inform future invoice evaluations. The system recalls vendor-specific history — amount patterns, payment terms, purchase-order formats, bank details, and past exceptions — and injects it into the model prompt before a decision, then retains confirmed human outcomes as new memories. Invoices are routed to AUTO_PROCESS or an exception path, with bank-detail changes always requiring human review.", "body_md": "The first useful question I asked about invoice automation was not “Can the model read an invoice?” It was “What happens when the same vendor sends its fiftieth invoice?”\n\nReading one invoice is straightforward. The harder problem is context. A human reviewer remembers that a vendor normally uses a particular purchase-order format, stays within a familiar amount range, keeps the same payment terms, and has had previous invoices approved after specific checks. A stateless model has to rediscover that context every time.\n\nI built VendorSense around that gap. It is an Accounts Payable agent that combines invoice extraction, Hindsight memory, and LLM reasoning so previous human decisions can influence future invoice evaluations.\n\nThe system I built\n\nVendorSense follows a deliberately narrow workflow:\n\nInvoice\n\n  ↓\n\nExtract invoice details\n\n  ↓\n\nHindsight Recall\n\n  ↓\n\nGroq reasoning\n\n  ↓\n\nAUTO-PROCESS or EXCEPTION\n\n  ↓\n\nHuman confirmation when required\n\n  ↓\n\nHindsight Retain\n\n  ↓\n\nFuture invoices use that experience\n\nThe application is built with Python and Streamlit. PDF invoices are parsed with pypdf, Groq handles reasoning, and Hindsight provides the memory layer.\n\nThe interface exposes invoice intake, an agent pipeline, processing metrics, an exception view, and a dedicated Vendor Memory view. I wanted the memory loop to be visible instead of hiding it behind an API call.\n\nFigure 1 — The VendorSense dashboard makes the invoice pipeline and learning activity visible.\n\nAUTO_PROCESS represents simulated Accounts Payable routing; the system does not execute a real payment.\n\nThe application also records the processed invoice and keeps the uploaded file in its archive, separate from learned Hindsight experience.\n\nWhy I put Hindsight before reasoning\n\nThe most important architectural decision was the ordering of operations.\n\nI did not want the agent to analyze an invoice and then optionally look up history. I wanted history to become part of the model's input before the decision was made.\n\nFor each invoice, VendorSense constructs a query from the current vendor and invoice context. It asks Hindsight to retrieve previous approvals, rejections, amount patterns, payment terms, purchase-order patterns, verified bank information, previous exceptions, human decisions, and learned vendor behavior.\n\nThe recall call is:\n\nresult = hindsight.recall(\n\n    bank_id=BANK_ID,\n\n    query=query,\n\n    max_tokens=2500,\n\n    budget=\"mid\",\n\n)\n\nI then place the returned memories alongside the current invoice:\n\nuser_prompt = f\"\"\"\n\nCURRENT INVOICE:\n\n{json.dumps(invoice, indent=2)}\n\nHINDSIGHT MEMORY:\n\n{memory_text}\n\nEvaluate this invoice.\n\n\"\"\"\n\nThis creates a useful separation:\n\nCurrent invoice\n\nHindsight memory\n\nWhat is happening now\n\nWhat happened before\n\nAmount, PO, terms, bank details\n\nPrevious decisions and vendor patterns\n\nCurrent evidence\n\nHistorical context\n\nThat separation is what makes Hindsight useful here. The model is no longer forced to treat every invoice as a clean slate.\n\nFigure 2 — VendorSense keeps the processed invoice as part of the application's invoice workflow and archive.\n\nMemory is evidence, not truth\n\nThe next problem was more subtle.\n\nIf I give a model historical context, it is tempting to treat that context as authoritative. I decided not to.\n\nThe reasoning instructions in VendorSense explicitly establish:\n\nHindsight memory is evidence, not absolute truth.\n\nThe same instructions tell the agent to behave conservatively when there is little relevant experience, recognize familiar vendor patterns when the evidence supports them, route meaningful deviations to an exception path, and require review when bank details change.\n\nThe important part: humans create the learning signal\n\nA model recommendation should not automatically become a fact about the vendor.\n\nVendorSense therefore learns from confirmed human outcomes.\n\nWhen an invoice is routed for review, the reviewer can approve or reject it and add a note. The application turns that outcome into an explicit learning event:\n\nexperience = f\"\"\"\n\nVendorSense AP learning event.\n\nVendor: {invoice.get('vendor')}\n\nInvoice ID: {invoice.get('invoice_id')}\n\nAmount: ₹{invoice.get('amount')}\n\nPurchase Order: {invoice.get('purchase_order')}\n\nPayment Terms: {invoice.get('payment_terms')}\n\nBank ending: {invoice.get('bank_account_last4')}\n\nHuman decision: {decision}\n\nHuman review note:\n\n{reason}\n\nThis is a human-confirmed AP experience.\n\nUse this experience when evaluating future invoices\n\nfrom this vendor.\n\n\"\"\"\n\nThen the learning event is retained in Hindsight:\n\nhindsight.retain(\n\n    bank_id=BANK_ID,\n\n    content=experience,\n\n)\n\nFigure 3 — The Recall and Retain implementation that connects VendorSense to Hindsight memory.\n\nThe learning loop is therefore:\n\nAgent recommendation\n\n        ↓\n\nHuman confirmation\n\n        ↓\n\nConfirmed experience\n\n        ↓\n\nHindsight Retain\n\n        ↓\n\nFuture Hindsight Recall\n\nI prefer this to letting the model learn from its own outputs. Recommendations do not become trusted vendor knowledge until a human has confirmed the outcome.\n\nThe before-and-after moment\n\nThe easiest way to see the value of memory is to process two invoices from the same vendor.\n\nImagine an invoice from Apex Industrial Supplies arriving for the first time. VendorSense can extract the vendor, amount, purchase order, payment terms, and bank information, but it has little vendor-specific history to work with.\n\nThe safe response is to evaluate the invoice conservatively and bring a human into the loop.\n\nThe human confirms the outcome.\n\nVendorSense retains that experience.\n\nNow another Apex invoice arrives.\n\nThe invoice is not evaluated in isolation anymore. Hindsight can return the previous experience, including the prior human decision and recurring vendor patterns. The reasoning layer can use that context to distinguish a routine invoice from something that deserves attention.\n\nThe behavioral change can be summarized simply:\n\nFirst interaction\n\nLater interaction\n\nLittle vendor-specific context\n\nRelevant vendor experience is available\n\nConservative evaluation\n\nHistorical context informs the evaluation\n\nHuman decision creates learning data\n\nPrevious learning is recalled\n\nAgent starts with little experience\n\nAgent can use accumulated experience\n\nLearning should not make the agent careless\n\nThere is an equally important counterexample.\n\nSuppose Apex becomes a familiar vendor. Previous invoices match expected patterns. Several have been confirmed by humans.\n\nThen the bank details change.\n\nA memory-powered workflow that blindly trusted its history could reason:\n\n“I know this vendor, so this is probably fine.”\n\nVendorSense is designed not to do that.\n\nThe model output includes a bank_change_detected flag, and the application enforces the exception path:\n\nresult = json.loads(content)\n\nif result.get(\"bank_change_detected\") is True:\n\n    result[\"decision\"] = \"EXCEPTION\"\n\nreturn result\n\nThat gives the system a useful rule:\n\nA strong historical pattern should improve recognition of normal behavior, not erase sensitivity to meaningful change.\n\nFamiliarity is not permanent permission.\n\nFigure 4 — The human-review view makes the evidence and review decision visible before a confirmed outcome becomes learning data.\n\nMaking the memory layer inspectable\n\nAnother practical lesson was that memory should not feel magical.\n\nThe invoice workflow shows the current invoice, retrieved Hindsight memory, the agent decision, confidence, reason, evidence, and the human review/learning action.\n\nA dedicated Vendor Memory view makes the same lifecycle easier to inspect.\n\nThat gives the user a traceable sequence instead of a black box:\n\nCurrent invoice\n\n     ↓\n\nMemory retrieved\n\n     ↓\n\nReasoning\n\n     ↓\n\nRecommendation\n\n     ↓\n\nHuman confirmation\n\n     ↓\n\nMemory updated\n\nWhat I learned building it\n\nCalling a memory system and displaying old records is easy. The value appears when those memories enter the reasoning step and alter what the agent can conclude.\n\nI do not want the agent to teach itself that its own previous recommendation was correct. A confirmed outcome is a clearer learning signal.\n\nHindsight gives the agent context, but the reasoning layer still has to evaluate whether that context applies to the current event. “This happened before” is not the same thing as “this is safe now.”\n\nThe bank-account change path is deliberately enforced rather than left entirely to model interpretation. Some conditions should be obvious in application logic.\n\nVendorSense focuses on one workflow: evaluating invoices in the context of vendor history. That narrow scope makes the entire loop easier to trace from recall to reasoning to human confirmation to retention.\n\nWhere Hindsight fits\n\nHindsight is not an add-on to VendorSense. It is the component that changes the information available to the reasoning layer.\n\nWithout memory, the agent can still extract an invoice and produce a recommendation.\n\nWith Hindsight agent memory, the agent can retrieve previous vendor experiences and use confirmed human decisions as context for future evaluations.\n\nThe implementation uses the Hindsight GitHub repository and Hindsight documentation as the memory layer around that loop.\n\nThe rest of the application handles invoice intake, authentication, interface state, and operational records. Keeping those concerns separate makes the architecture easier to reason about and leaves room to move application persistence to durable infrastructure without changing the memory contract.\n\nThe pattern I would reuse\n\nThe most reusable part of VendorSense is not the invoice parser or the dashboard.\n\nIt is the learning loop:\n\nRecall the past.\n\nReason about the present.\n\nAsk a human when the evidence is insufficient.\n\nRetain the confirmed outcome.\n\nUse it the next time.", "url": "https://wpnews.pro/news/i-used-hindsight-to-remember-human-invoice-decisions", "canonical_source": "https://dev.to/harini_varma_a5c641756f3e/i-used-hindsight-to-remember-human-invoice-decisions-4ilb", "published_at": "2026-09-28 21:02:05+00:00", "updated_at": "2026-09-28 21:19:55.630501+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-tools", "artificial-intelligence"], "entities": ["VendorSense", "Hindsight", "Groq", "Streamlit", "Python", "pypdf"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/i-used-hindsight-to-remember-human-invoice-decisions", "markdown": "https://wpnews.pro/news/i-used-hindsight-to-remember-human-invoice-decisions.md", "text": "https://wpnews.pro/news/i-used-hindsight-to-remember-human-invoice-decisions.txt", "jsonld": "https://wpnews.pro/news/i-used-hindsight-to-remember-human-invoice-decisions.jsonld"}}