After processing thousands of bank statements, invoices, and receipts through Claude Vision API, I've learned that financial document OCR is harder than it looks. Here's what actually works in production.
Traditional OCR tools like Tesseract or AWS Textract struggle with financial documents for three reasons:
1
with l
or 0
with O
creates accounting errors. A single misread digit can break double-entry bookkeeping.Traditional OCR gives you raw text. You still need to write hundreds of lines of regex to parse it into structured data.
Claude Vision doesn't just extract text — it understands document structure. You give it an image and a prompt like:
"Extract this bank statement into JSON with transaction date, description, debit, credit, and balance columns."
Claude returns structured JSON directly. No regex. No manual column detection.
Input: Bank statement PDF (converted to PNG)
Prompt:
Extract all transactions from this bank statement. Return JSON with:
- header: {accountNumber, statementPeriod, bankName}
- transactions: [{date, description, debit, credit, balance}]
Rules:
- Dates in YYYY-MM-DD format
- All amounts as numbers (no currency symbols)
- If a field is unclear, use null (never guess)
Output:
{
"header": {
"accountNumber": "****1234",
"statementPeriod": "2024-01-01 to 2024-01-31",
"bankName": "Chase Bank"
},
"transactions": [
{
"date": "2024-01-03",
"description": "Amazon.com",
"debit": 49.99,
"credit": null,
"balance": 1450.01
},
{
"date": "2024-01-05",
"description": "Salary Deposit",
"debit": null,
"credit": 3500.00,
"balance": 4950.01
}
]
}
No parsing code. No regex. Just structured data ready for your database.
Problem: Users upload phone photos of statements — blurry, skewed, poor lighting.
Solution: Preprocess images before sending to Claude:
from PIL import Image, ImageEnhance
def preprocess_image(img_path):
img = Image.open(img_path)
img = img.convert('L')
enhancer = ImageEnhance.Contrast(img)
img = enhancer.enhance(2.0)
if img.size[0] > 2000:
ratio = 2000 / img.size[0]
img = img.resize((2000, int(img.size[1] * ratio)))
return img
Result: Accuracy improved from 78% to 94% on mobile-captured statements.
Problem: Statements can be 5-10 pages. Sending all pages in one request:
Solution: Process first + last page only for most use cases:
For full transaction history, batch-process middle pages and merge results.
import anthropic
def extract_statement_summary(pdf_pages):
"""Extract key info from first and last page only"""
client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
first_page = pdf_pages[0]
response = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
messages=[{
"role": "user",
"content": [
{
"type": "image",
"source": {
"type": "base64",
"media_type": "image/png",
"data": first_page
}
},
{
"type": "text",
"text": "Extract account number, statement period, opening balance as JSON."
}
]
}]
)
summary = json.loads(response.content[0].text)
return summary
Cost savings: $0.15 per statement → $0.03 per statement (5× reduction)
Problem: Claude occasionally misreads 1,234.56
as 123456
or 12.34
.
Solution: Add validation rules in your prompt:
Rules for amount extraction:
1. All amounts must have exactly 2 decimal places
2. If you see "1,234.56", extract as 1234.56
3. If you see "1234", extract as 1234.00
4. If unclear whether "1234" means 1234.00 or 12.34, use null
5. Never guess — uncertain values must be null
Also validate in code:
def validate_amount(amount):
if amount is None:
return None
return round(float(amount), 2)
Result: Decimal errors dropped from 3.2% to 0.4%.
Problem: Claude Sonnet 4 sometimes rate-limits during peak hours.
Solution: Implement model fallback:
MODELS = [
"claude-sonnet-4-20250514", # Primary
"claude-3-5-sonnet-20241022", # Fallback
"claude-3-haiku-20240307" # Last resort
]
def extract_with_fallback(image_data):
for model in MODELS:
try:
response = client.messages.create(
model=model,
max_tokens=12000,
messages=[...]
)
return response
except anthropic.RateLimitError:
continue
raise Exception("All models rate-limited")
Result: 99.7% uptime even during peak usage.
Problem: Real-world statements have weird formats:
(1234.56)
instead of -1234.56
Solution: Explicit edge case handling in prompts:
Edge cases:
- Amounts in parentheses like "(123.45)" mean negative (debit)
- If a transaction has no date, use the statement end date
- If balance column is empty, calculate it from previous balance +/- amount
- "Pending" transactions go in a separate "pending" array
And post-process in code:
def normalize_transaction(txn):
if txn['debit'] and '(' in str(txn['debit']):
txn['debit'] = -float(txn['debit'].strip('()'))
if not txn['date']:
txn['date'] = statement_end_date
return txn
Financial document OCR can get expensive. Here's what we learned:
| Optimization | Cost Impact | Accuracy Impact |
|---|---|---|
| Process first+last page only | -80% | -5% (acceptable for summaries) |
| Use Haiku for simple receipts | -90% | -2% (receipts are easier) |
| Batch similar documents | -30% | +3% (context helps) |
| Prompt caching (reuse bank-specific rules) | -50% | No change |
Current costs: $0.03 per bank statement, $0.01 per receipt using this setup.
After 10,000+ documents processed:
| Document Type | Accuracy | Notes |
|---|---|---|
| Digital bank statements (PDF) | 98.2% | High contrast, clean layout |
| Scanned bank statements | 94.1% | Preprocessed with contrast enhancement |
| Mobile photos of statements | 91.7% | Users must follow photo guidelines |
| Invoices (structured) | 96.8% | Consistent format helps |
| Receipts (printed) | 89.4% | Small text, low contrast |
| Handwritten receipts | 72.3% | Use case too hard for automation |
"Accuracy" = extracted data matches manual review.
Claude Vision isn't perfect for:
For these cases, consider AWS Textract + custom parsing or on-premise OCR.
Want to test Claude Vision on your own statements? I built CleanStmt as a free tool to convert bank statements to Excel/CSV using the techniques above.
Source code for the preprocessing pipeline: GitHub (coming soon)
What's your experience with financial document OCR? Drop a comment if you've hit similar challenges or found better solutions.