# Building Intelligent Document Parsing Systems: Balancing AI Capabilities with Deterministic Fail-Safes

> Source: <https://gist.github.com/JawadulHadi/0ab89d96e0abb545779522b82263e728>
> Published: 2026-09-27 05:14:55+00:00

- *Jawad Ul Hadi, Backend Lead / Architect — AI-First Systems Design*

I built a NestJS module that turns timesheets in six formats (PDF, DOCX, XLSX, XLS, PNG, JPG) into one fixed eight-column payroll export. "AI-first" meant weighing AI at every design decision, and then using it only where it pays for itself. Deterministic parsing does the work. An LLM is a bounded, redacted, budget-capped fallback that can never fail an upload. AI was also part of how the module was built: planning, implementation, and an AI code review that caught ten real bugs before merge.

## HR operations receive timesheets from client sites in whatever format each site uses: weekly grids with repeated In/Out/Hours blocks, individual timecards with dated punches, monthly day-numbered sheets, scans and phone photos. Payroll needs one thing: an eight-column workbook with **one row per employee per worked day**.

```
Employee Number | First Name | Last Name | Full Name | Final Department | Employee Start | Final End | Final Hours
8092            | Jane       | Doe       | Jane Doe  | Afternoon Shift  | 8/24/2026 14:30| 8/25/2026 0:00 | 9.00
```

The rules are simple to state and easy to get wrong:

- **If the source has it, extract it. If not, leave it blank.** Nothing is inferred and nothing is back-filled from the HR database.
- AM/PM must survive. An overnight shift's end time moves to the next day.
- Start and end are real Excel datetimes, not strings. Hours are decimals.

## AI-first does **not** mean "send everything to an LLM". It means AI is considered first at every design decision, and then placed where it earns its cost, latency and privacy risk. I applied that in two places.

``` php
flowchart TD
    U[Upload] --> V{Validate MIME + size}
    V --> R{Route by format}
    R -- XLSX / XLS / DOCX --> N[Native table extraction]
    R -- Digital PDF --> T[Text layer to grid]
    R -- PNG / JPG --> O[OCR to grid]
    N & T & O --> P[Deterministic parser<br/>weekly grid / timecard]
    P --> Q{Quality gate<br/>confident?}
    Q -- yes --> F[Flatten to 8 columns]
    Q -- no --> K{AI enabled?}
    K -- no --> F
    K -- yes --> AI[Redact PII, then LLM tool call,<br/>then re-parse and re-score]
    AI --> B{Better than deterministic?<br/>ties go to deterministic}
    B --> F
    F --> X[Specimen XLSX + audit record]
```

Timesheets are *tabular*. A parser that understands the layout is faster, free and reproducible, and it is right most of the time. The LLM handles the leftovers: noisy OCR, odd layouts, ambiguous rows. So the cost model is:

```
cost per document = P(AI invoked) × cost per AI call
                  ≈ 0.40 × $0.0055  (small model, ~3k in / ~500 out tokens)
                  ≈ $0.0022   →  about $22 per month at 10,000 documents
```

Quoting a flat "cost per LLM call" would have overstated the bill 2–10×, because clean digital files never reach the model.

**Guardrails on the AI path:**

| Concern | Control | 
|---|---|
| Privacy | Emails, URLs, SSNs and phone numbers are redacted before any external call. Only a text grid is sent, never the file and never a signed URL. | 
| Determinism | Temperature 0, forced tool use, JSON-schema output, versioned prompt | 
| Trust | The AI output is re-parsed and re-scored by the *same* deterministic pipeline, and it wins only if it scores strictly higher | 
| Cost | A pre-flight token estimate is checked against a per-upload USD budget. If it's over, the call is skipped | 
| Availability | Short timeout plus a circuit breaker. Any failure returns `null` , so the deterministic result ships | 
| Audit | Provider, model, prompt version, tokens, cost estimate, confidence and decision reason are stored per upload | 
| Logs | Status codes only: no prompts, no document text, no model output | 

With no API key configured, the module runs fully deterministic at zero AI cost. AI is a switch, not a dependency.

| Phase | How AI was used | What I kept for myself | 
|---|---|---|
| Proposal | Drafted the cost model and compared providers (Anthropic, OpenAI, Google, open-weight) per invoked call | Scope boundaries, the decision to keep AI off the critical path, the provider recommendation | 
| Contract | Turned the sample export into explicit rules (blank-not-inferred, overnight, native datetimes) | Signing off the contract as the source of truth | 
| Build | Claude Code implemented the layered NestJS module against the house patterns (guards, response envelopes, repositories) | Architecture, the simplification calls, reviewing every diff | 
| Review | An AI code review of the rewrite found **10 real bugs** | Triage, fix design, regression tests | 
| Verification | Ran the parser against the real input template instead of trusting unit tests | Deciding what "done" means | 

The review is the part I'd highlight. A "lean" rewrite removed about 8,800 lines: extra provider abstractions, a Python OCR sidecar, dead indirection. Simplifying that aggressively also dropped things that mattered:

- The **JWT guard disappeared** from the controller, so every route was unauthenticated and crashed on`user._id` .
- An SDK import had no matching dependency, so the app would have **crashed on boot** .
- Storage returned `file://` paths instead of private cloud URLs.
- Scanned PDFs produced **empty exports marked "completed"** : silent data loss.
- A phone-number redaction regex also matched ISO dates and runs of decimal hours, so the LLM received corrupted data.
- Weekday detection matched on a 3-letter prefix, so "**Mon** thly Total" became a second Monday and hours were counted twice.
- A timezone "correction" shifted every exported time by five hours, because the Excel library writes dates from UTC.

Each fix came with a regression test that fails on the old code. Then I ran the parser against the real supplied template, not synthetic fixtures. It returned **zero rows**: that layout uses day numbers (`1 In / 1 Out / 1 Hours`), not weekday names. The unit tests were green, and the acceptance criterion was not. That gap is now the top item on the backlog instead of a production surprise.

| Decision | Alternative | Why | 
|---|---|---|
| Deterministic-first, LLM fallback | LLM extracts everything | Cheaper, reproducible, auditable. The LLM covers the long tail | 
| Re-parse the AI output with the same parser | Trust the model's JSON | One definition of "correct", and it makes AI and non-AI results comparable | 
| Deterministic wins ties | Prefer the newer or smarter path | Stable output. AI has to be *better* to be adopted | 
| Store floating wall-clock times (printed time in UTC fields) | Apply a business-timezone offset | Excel writers serialise from UTC. The export must show what was printed | 
| Reject scanned PDFs with a clear 422 | OCR them via a vision model | No rasteriser in the stack, and sending raw files breaks the "text only" privacy rule. A clear error beats a silent empty export | 
| One concrete AI service | Provider interface + factory + DI tokens | One active vendor. Swapping means rewriting ~150 lines. The export contract doesn't change | 
| Blank beats inferred | Look up missing IDs or departments in HR | The source document is the source of truth, and the export stays auditable | 

NestJS 11 · TypeScript 5 · MongoDB (Mongoose) · Google Cloud Storage (private, signed URLs) · ExcelJS · SheetJS · mammoth · pdf-parse · tesseract.js · Anthropic Messages API (tool use) · Jest

Dated: 27-Sep-2026
