# Human-in-the-Loop Validation for AI-Generated Financial Models > Source: > Published: 2026-10-17 11:54:42+00:00 An AI system can produce a spreadsheet that looks correct, contains plausible formulas, and returns a coherent financial result while still encoding the wrong assumptions. That is a difficult failure to detect because spreadsheet quality has several dimensions. A workbook may be syntactically valid but semantically wrong. Its formulas may calculate correctly while referencing the wrong inputs. Its output may be internally consistent but depend on unsupported source data. These are different failure modes, and they require different controls. For financial models, the engineering objective should not be to make a language model responsible for every step. It should be to separate interpretation, calculation, validation, and professional approval so that each can be tested independently. This article develops one possible implementation pattern for AI-assisted financial modelling. It is an educational architecture, not a description of any proprietary product. Before designing a validation pipeline, distinguish five objects: These objects should not collapse into a single generated answer. Consider a hypothetical development feasibility model. An AI system extracts a construction cost from a project document and proposes entering it into a workbook. The extraction may be correct, but that does not establish that the figure is current, applies to the relevant asset, or uses the correct unit. Even after the assumption has been approved, the resulting calculation still needs validation. A useful validation system therefore asks separate questions: A single confidence score cannot adequately answer all six. Language models are useful for interpreting ambiguous instructions, extracting information, identifying relationships, and proposing structured data. They should not be the sole authority for consequential arithmetic. A safer design separates the workflow into distinct stages: ``` Source documents and workbook | v AI-assisted extraction | v Structured candidate assumptions | v Schema and provenance validation | v Human approval of material inputs | v Deterministic calculation engine | v Independent validation checks | v Human review and release ``` This separation creates meaningful test boundaries. For example, if a residual land value changes unexpectedly, an engineer can investigate whether the cause was an extracted input, a changed assumption, a formula defect, or a validation failure. The architecture does not guarantee that every error will be caught, but it makes errors easier to locate and investigate. The calculation engine should receive typed, validated inputs and execute explicitly defined formulas. It should not need to infer the intended mathematics from a natural-language explanation each time a calculation runs. A common weakness in AI-assisted workflows is treating generated prose as if it were a reliable data interface. Instead, require the model to propose structured records. The exact schema will depend on the application, but a candidate assumption might look like this: ``` { "name": "construction_cost", "value": 1250, "unit": "currency_per_square_metre", "source_reference": "document_03", "source_location": "page_12", "status": "pending_review" } ``` The number and unit above are illustrative, not market evidence. A production schema would normally need additional fields, such as currency, applicable area definition, source date, geographic scope, extraction status, and approval metadata where relevant. The important point is that the value is not accepted merely because the model returned valid JSON. Validation should cover several layers: **Structural validation** **Semantic validation** **Provenance validation** A schema can enforce structure. It cannot, by itself, establish that an assumption is commercially reasonable. Once assumptions are approved, calculations should run through deterministic code or a controlled spreadsheet calculation engine. Deterministic means that the same defined inputs, formulas, and calculation rules produce the same result under the same execution conditions. It does not mean the model is economically correct. Validation should include at least four categories. Confirm that formulas use the intended cells, units, and dependency relationships. Detect broken references, unexpected hard-coded values, circular dependencies where prohibited, and missing required formulas. Where a spreadsheet is involved, the validation layer should distinguish a formula cell from a displayed value. A cached result alone may not establish that the formula was recalculated using the latest inputs. Where applicable, test identities and reconciliations defined by the model specification. Examples include: The checks must be derived from the actual model specification. Not every financial model has the same identities or conventions. Reject or flag invalid inputs, incompatible units, missing required values, impossible dates, and calculations that divide by zero. A value being numeric does not mean it is valid for the model. Maintain a set of known test cases. After changing formulas, extraction logic, or calculation code, compare the results with approved expected outcomes. Tests should cover normal cases, boundary conditions, missing data, and known failure scenarios. Expected results must come from independently checked calculations or approved specifications, not from the same model output being tested. A model can pass isolated formula checks while behaving badly when assumptions change. Sensitivity testing examines whether outputs respond in ways consistent with the model's defined relationships. For a hypothetical development model, useful tests might include: These are diagnostic tests, not universal assertions about every model. For example, increasing construction cost would generally be expected to reduce residual land value in a conventional residual valuation model when all other inputs remain fixed and the additional cost is not offset elsewhere. If the output increases instead, the system should flag the behavior for investigation. The test does not automatically prove that the model is defective. There may be a legitimate dependency or offsetting rule. The purpose is to make the behavior explicit and explainable. A practical test suite should distinguish: Directional checks need careful design. Hard-coding intuitive rules without accounting for model dependencies can produce false alarms. Adding a human approval button is not sufficient. The reviewer needs the information and authority required to make a meaningful decision. A useful review interface should expose: Review priority should reflect materiality and uncertainty. A minor formatting issue does not require the same treatment as an unsupported assumption that materially changes a financial result. A possible approval policy is: | Condition | System action | |---|---| | Required source missing | Block the assumption | | Unit or schema invalid | Reject or quarantine the candidate | | Material assumption lacks approval | Prevent release | | Calculation reconciliation fails | Block release pending investigation | | Sensitivity test produces an unexplained result | Escalate for review | | All required checks pass | Permit the next review stage | Passing automated checks should not be treated as proof that a model is suitable for a real-world decision. The approval policy must also define who can approve an assumption, what evidence is required, how overrides are recorded, and when an approval expires or must be revisited. A useful audit trail records the sequence of changes, not just the final workbook. For each material assumption, record the source reference, extracted value, proposed interpretation, approved value, reviewer, timestamp, and reason for any override. For each calculation run, record the model version, input version, calculation version, validation results, and output version. Where AI is involved, also consider recording the model or service version and the relevant processing configuration, subject to privacy, security, and retention requirements. This is important because an identical output can arise from different assumptions, while a changed output can result from either changed inputs or changed calculation logic. Auditability requires enough information to reconstruct why a result was produced. A generated explanation that merely sounds plausible is not a substitute for that record. **Mistake 1: Treating valid JSON as correct data.** A schema proves that a response meets structural constraints. It does not prove that a source was interpreted correctly or that the value is appropriate. **Mistake 2: Asking the same AI system to verify its own answer.** A second model-generated explanation may identify useful issues, but it is not independent evidence of correctness. Independent checks should use source evidence, deterministic calculations, explicit invariants, or separately established expected results. **Mistake 3: Checking only the final output.** A plausible headline result can hide errors in intermediate calculations, units, timing, or assumptions. Validate the calculation chain as well as the final number. **Mistake 4: Treating human approval as a rubber stamp.** Reviewers need access to evidence and the ability to reject, amend, or escalate proposed assumptions. Otherwise, the control exists mainly on paper. **Mistake 5: Confusing model correctness with decision quality.** A model may calculate exactly as specified and still be unsuitable because its assumptions are weak, its evidence is stale, or relevant risks are omitted. Teams building this capability can start with a narrow, testable scope. Evaluation should measure more than whether the pipeline completes. Useful measures include unsupported-assumption detection, source-traceability coverage, validation failures caught, false-positive rates, reviewer override frequency, and reproducibility across controlled test cases. These measures need explicit definitions and representative test data. No performance figure should be claimed without a documented evaluation method. Human-in-the-loop validation is most effective when it is designed into the system rather than added at the end. AI can help interpret documents, propose structured assumptions, and explain relationships. Deterministic calculation, independent validation, source traceability, and accountable human approval address different parts of the problem. The engineering objective is not to eliminate professional judgment. It is to ensure that the judgment is applied to a result whose assumptions, evidence, calculations, and limitations can be inspected. For a financial model, a plausible number is only the beginning of the validation process.