{"slug": "i-asked-gemini-to-read-300-year-old-portuguese-parish-records", "title": "I Asked Gemini to Read 300-Year-Old Portuguese Parish Records", "summary": "Google's Gemini AI can extract structured genealogical data from 300-year-old Portuguese parish records, according to a benchmark of 84 images tested by a genealogist. The model must expand abbreviations, modernize historical spellings, and recover faded marginalia to produce JSON output in Portuguese, preserving only what the page states. The test evaluates whether AI can reliably identify people and relationships from digitized parish books held by the Aveiro District Archive and the University of Coimbra Archive.", "body_md": "Can you read this page?\n\nThis page was written in a Portuguese parish in 1693.\n\nIt holds three baptism records in running cursive, densely abbreviated.\n\nThe first child was baptized “aos 30 de agosto de 1693 annos” — on August 30, 1693.\n\nThe second, “aos 2 de 7.bro” — *7bre* being the period abbreviation for September.\n\nThe third entry records only one parent, and the ink has faded to the edge of legibility.\n\nIn the left margin, the priest repeated each child’s name and hamlet — *An.to / Vila*, *Mathias / Poutena* — a seventeenth-century finding aid.\n\nI ask language models to turn pages like this into structured genealogical data—not a full transcription, but the people and relationships needed to identify someone and connect them to a family.\n\nFor this page, the expected answer is:\n\n```\n[\n  {\n    \"arquivo\": \"PT-ADAVR-PAND13-1-3_m00009.jpg\",\n    \"assentos\": [\n      {\n        \"tipo\": \"batismo\",\n        \"data\": \"1693-08-30\",\n        \"local\": \"Bemposta\",\n        \"nome_principal\": \"Maria\",\n        \"nome_do_pai\": \"Domingos Jorge\",\n        \"nome_da_mae\": \"Isabel Francisca\"\n      },\n      {\n        \"tipo\": \"batismo\",\n        \"data\": \"1693-09-02\",\n        \"local\": \"Vila\",\n        \"nome_principal\": \"Antonio\",\n        \"nome_do_pai\": \"Antonio Francisco\",\n        \"nome_da_mae\": \"Maria Francisca\"\n      },\n      {\n        \"tipo\": \"batismo\",\n        \"data\": \"1693-09-22\",\n        \"local\": \"Poutena\",\n        \"nome_principal\": \"Matias\",\n        \"nome_do_pai\": \"Manoel José\"\n      }\n    ]\n  }\n]\n```\n\nThis JSON is the human reference: my reading of the page, record by record.\n\nThree choices in this reference shape the evaluation.\n\nFirst, the schema is in Portuguese—the language of the corpus and my native language.\n\nEach *assento* (record) includes a *tipo* (type), date, place, *nome_principal* (the person the record concerns), and any parents named in the document.\n\nSecond, the reference records only what the page states. The third entry names a father but no mother, so the annotation preserves that absence rather than filling the gap.\n\nThird, the task requires more than character recognition.\n\nAbbreviations must be expanded (*7.bro* → September, *M.a* → Maria), historical spellings modernized, and place names recovered from cramped marginalia—in the third entry, from ink that is barely visible.\n\nThis image is one of 84 in the benchmark. The rest of this article examines what models get right, what they get wrong, and what changes when I let them investigate the image instead of answering in one shot.\n\n## Why This Extraction Matters[#](#why-this-extraction-matters)\n\nI started building my own family tree in 2022.\n\nThat hobby quickly led me to Portuguese parish records, to the slow, rewarding work of reading them by hand, and to the question behind this article: can current AI reliably extract the people mentioned in these documents?\n\nFor anyone researching Portuguese ancestry, parish records are the backbone source.\n\nFrom the sixteenth century until civil registration took over in the early twentieth century, the local parish was where baptisms, marriages and deaths were written down.\n\nMany of these books are now [digitized and published online](https://tombo.pt) by Portuguese archives, including the two whose collections I use here: the [Aveiro District Archive (ADAVR)](https://adavr.dglab.gov.pt/) and the [University of Coimbra Archive (AUC)](https://www.uc.pt/auc/).\n\nBut digitized pages are not searchable data (yet).\n\nFinding an ancestor today usually means guessing which books might mention them and then reading, page after page of this *pleasant* calligraphy.\n\nWhat an (amateur) genealogist actually needs from each record is relational:\n\n| Record type | Fields that matter most | Why |\n|---|---|---|\n| Baptism | Child’s name; baptism date; parents’ names; location | In early records the baptism date is often the only dated fact about a person; the parents link the child to the previous generation |\n| Baptism, after ~1700 | Paternal and maternal grandparents, when present | Later records increasingly name them, expanding the tree two generations at once |\n| Marriage | The couple’s names; each spouse’s parents | Marriages connect two lineages — and are often found by tracing all of a couple’s children |\n| Death | The deceased’s name; spouse or father, when present | These disambiguate the person and preserve family links |\n\nAn isolated name rarely solves a search; a name, plus parents, plus a place usually does.\n\nThat is why this benchmark measures the extraction of people and relationships, not full-text transcription quality.\n\n## The Benchmark Data[#](#the-benchmark-data)\n\nI built the reference set by reading 84 digitized images from 17 parish books — nine from ADAVR, eight from AUC — and manually extracting the same structured fields shown for the opening page: 434 records spanning 1601 to 1876.\n\nI expect most of it to be correct given my few years of hobbyist experience reading and studying hundreds of these pages, but I don’t expect perfection.\n\nThe set was composed to be roughly balanced across the three record types and three centuries.\n\nThe 434 records split into 140 baptisms, 135 marriages and 159 deaths.\n\nBy century, 173 date from the seventeenth, 131 from the eighteenth and 123 from the nineteenth (some records don’t have dates in the visible page).\n\nAn image can have up to two pages and hold anywhere from 2 to 21 records — about 5 on average.\n\nThe balance lets me check whether older, harder-to-read records really score worse instead of letting one easy century dominate the average.\n\n## What The Model Is Asked To Do[#](#what-the-model-is-asked-to-do)\n\nFor each benchmark item, the model receives one digitized image and must identify every parish record in reading order.\n\nThe instruction itself is short and is sent in Portuguese:\n\n```\nExtraia cada assento desta página seguindo o schema anexado\n\nNormalize os nomes para versões modernas e abreviações para o nome completo. Por exemplo:\nJoze - José\nJoam - João\nMagdalena - Madalena\nAnna - Ana\nRoiz - Rodrigues\nAntônio - Antonio\nManuel - Manoel\nRosaria - Rosália\nBaptista - Batista\nIzabel - Isabel\n\n[a list of valid place names for the parishes in the benchmark]\n\nPreencha o campo arquivo com: PT-ADAVR-PAND13-1-3_m00009.jpg\n```\n\nIn English: extract every record on the page following the attached schema, normalize names to modern versions and abbreviations to full names, and identify the image in the arquivo field.\n\nThe output must identify the record type and date, then capture the people and family relationships relevant to that type.\n\nBaptisms and deaths revolve around a principal person; marriages name the bride and groom.\n\nFields unsupported by the document may be omitted, and the schema provides no place for witnesses or godparents.\n\nTwo additional conventions remove ambiguity before scoring.\n\nHistorical names and abbreviations are normalized to the expected modern form: “Joseph”, “Joze” and “José” can all be the same name in these books, as can “Manuel” and “Manoel”.\n\nWithout an explicit convention, a model that modernizes a spelling and a model that transcribes letter by letter would each be penalized against a reference written in the other style.\n\nThe bracketed section is a list of valid place names for the parishes in the benchmark, grouped by municipality and accompanied by historical spelling variants.\n\nI compiled it through research separate from the annotation, mirroring my own practice of checking candidate places when the handwriting is ambiguous or a name is unfamiliar.\n\nA subtle way of leaking data would be using the golden answers to extract the location, but we don’t make this type of mistake here (only other types).\n\nTogether, these rules define a valid extraction before the deterministic grader compares it with the human reference.\n\n## How Scoring Works[#](#how-scoring-works)\n\nEach image earns two numbers: the ordered Transcription Score and an unordered diagnostic.\n\n*(While writing this post I noticed that Transcription Score was not a good name for the metric, as this is an extraction task, but bear with me)*\n\nBoth operate on the annotated fields in the human reference.\n\nEach field has equal weight, predicted fields without a corresponding annotation are ignored, and the image score is the total credit divided by the number of evaluable reference fields.\n\nText fields earn `1 − Levenshtein distance / length of the longer string`\n\n.\n\nThis gives partial credit to imperfect readings that may still help a researcher find the right page.\n\nA search for a João Manuel married to a Maria da Silva should still surface a page where the model read “João Martins” but preserved Maria da Silva and the place; a reading unrelated to the real name earns close to zero, and the credit fades smoothly between these extremes.\n\nFor dates, string similarity prices errors badly: 1600 and 1700 differ by one character and by a century, while the 7th and the 17th of a month differ more in text and still land a researcher in the right weeks.\n\nSo dates use temporal distance instead: `max(0, 1 − days of difference / 56)`\n\n.\n\nAn exact date earns full credit, and an error of eight weeks or more earns zero. Reference dates are scored only when complete; a missing reference date earns a point only when the model also returns null.\n\nOne structural rule overrides field similarity: if the record type is wrong, every field in that record scores zero. A baptism interpreted as a death changes the event that gives its names and dates meaning.\n\nThe ordered score pairs predicted and reference records by position. A missing or invented record can therefore shift every record that follows it.\n\nFor example, suppose the reference order is `Maria, Antonio, Matias`\n\n, but the prediction skips Antonio and returns `Maria, Matias`\n\n.\n\nThe ordered score compares Matias with Antonio and leaves Matias unmatched; the unordered score restores the Maria–Maria and Matias–Matias matches, leaving only Antonio unmatched.\n\nThe unordered diagnostic uses the same field credits but finds the highest-scoring one-to-one assignment between complete predicted and reference records.\n\nA large gap between the two scores indicates an ordering problem; two low scores indicate that the extraction itself failed.\n\nOnly the model’s final message is scored. Invalid JSON receives zero and is also recorded in the invalid_json metric.\n\n## Experiment Results[#](#experiment-results)\n\nThe experiment runs two conditions on Gemini 3.5 Flash Lite and Gemini 3.6 Flash: T0, a single-pass extraction, and T1, the same task inside an agentic harness.\n\n*Chart layout adapted from the beautiful DiligenceBench.*\n\nThe diagram shows the ordered Transcription Score averaged across the 84 images.\n\nWithin each model, the environment helps: +0.17 for the budget model and +0.08 for the stronger one.\n\nThe unordered diagnostic follows the same pattern, rising from 0.46 to 0.63 on Gemini 3.5 Flash Lite and from 0.70 to 0.77 on Gemini 3.6 Flash.\n\nThe weaker reader gains more from the tools — the same pattern DiligenceBench reports on its harness ladder, where added rungs lift the weakest models most.\n\nAcross the two models, capability is the larger effect: the 3.6 Flash single pass already beats the 3.5 Lite with tools, and the best extraction belongs to the strongest model inside the harness.\n\nThe unordered score stays only 0.02 to 0.04 above the ordered one, so on average the loss concentrates on field extraction rather than record ordering.\n\nThe harness raises total cost from US$ 0.56 to US$ 8.74 on the budget model and from US$ 2.70 to US$ 47.47 on the stronger one, increases of 15.6× and 17.6×.\n\nThe incremental dollar buys less as the model improves: US$ 8.2 for +0.17 on the budget model, US$ 44.8 for +0.08 on the stronger one.\n\nInvalid JSON appears only in the Gemini 3.6 Flash T1 run — three of 84 images fail to parse — a direct cost of emitting free-text JSON without the API’s schema enforcement.\n\n## Do Older Records Score Worse?[#](#do-older-records-score-worse)\n\nYes, but the effect looks more like a seventeenth-century penalty than a smooth decline with age.\n\n*Field-weighted ordered Transcription Score over dated reference records; seven records without a visible year are excluded.*\n\nIn every model-environment pair, the seventeenth century scores below both later centuries.\n\nTwo baptism entries from Vilarinho do Bairro show what “older and harder to read” means in this corpus:\n\n| 1693 | 1795 |\n|---|---|\n\n*Crops from ADAVR baptism books PT-ADAVR-PAND13-1-3 and PT-ADAVR-PAND13-1-7.*\n\nThe later hand is darker and more regular, while the earlier page is faded, densely abbreviated and physically damaged.\n\nAbbreviations get really crazy as you reach the 1600-1500s.\n\nInterestingly, Gemini 3.6 Flash T1 improves clearly from the seventeenth to the eighteenth and then the nineteenth century.\n\n## What Separates T0 And T1, Exactly?[#](#what-separates-t0-and-t1-exactly)\n\nIn T0, the image and prompt go into one API call, and the JSON comes out. The response is constrained by the strict JSON Schema attached to the request, reasoning effort sits at medium, and every image gets exactly one attempt.\n\nIn T1, each model runs inside an agentic harness with the same core prompt and sampling settings.\n\nThe harness is [pi](https://pi.dev/) (v0.82.0), executing in a [Docker sandbox](https://github.com/PrimeIntellect-ai/verifiers) that carries image tooling: Python with Pillow and OpenCV, ImageMagick, Tesseract OCR, and a JSON Schema validator, with the same output schema.\n\nTo point the agent to that file, the prompt’s first line asks the model to read the schema file before starting and answering with valid JSON.\n\nAcross multiple turns, the model can write and run its own scripts: split a double page, cut out individual records, enlarge faded lines, adjust contrast, try OCR on a stubborn passage, and validate the output file before answering.\n\nOnly the final message is scored, exactly as in T0.\n\nThe design was inspired by [DiligenceBench](https://www.paperinstruments.com/blog/diligence-bench), an equity-research benchmark that grades agent submissions with dense, task-specific rubrics.\n\nIts reference harnesses form a controlled ladder: a base loop, where the model searches the web and answers from conversation context alone; a generic sandbox, which adds shell, Python and writable files on a persistent workspace; and a domain harness, which adds specialized tools and instructions.\n\nThere the sandbox rung beat the base loop for fifteen of the sixteen models tested — the same weights scored measurably higher once the model could execute and keep state.\n\nMy T0 and T1 mirror the first two rungs: the loop becomes a single vision call, the sandbox gains image tooling, and the question moves from “which model reads best in one shot?” to “what changes when the model can investigate the image?”.\n\n## Why Did I Test Only Gemini Models?[#](#why-did-i-test-only-gemini-models)\n\nIn my preliminary tests, Gemini 3.5 Flash Lite — the budget option — read these documents significantly better than every GPT and Claude model I tried, so the comparison stays within the family.\n\nExternal evidence points the same way: Gemini models are the strongest general-purpose entries on the [CHURRO leaderboard](https://stanford-oval.github.io/Churro/leaderboard.html) for historical document transcription, ahead of the best Claude and GPT results and behind only the benchmark authors’ own fine-tuned specialist model.\n\n[Independent testing](https://generativehistory.substack.com/p/gemini-3-solves-handwriting-recognition) on eighteenth- and nineteenth-century English manuscripts reached similar conclusions, with Gemini 3 Pro achieving error rates comparable to a professional typist and far ahead of both competitors.\n\nOne variation I didn’t test, but might in the future, is to run stronger agentic models with Gemini as a tool for vision.\n\n## A Few Interesting Rollouts[#](#a-few-interesting-rollouts)\n\n### When Decomposition Recovers Structure[#](#when-decomposition-recovers-structure)\n\nThis image from 1636 and 1637 contains twelve marriage entries across two pages.\n\nIn T0, Gemini 3.5 Flash Lite examined the complete image once, drifted between death and baptism, returned a single baptism, and scored 0.00.\n\nIn T1, it split the scan into two pages, divided each page into vertical bands, created twelve entry-level crops, and increased the contrast on selected regions.\n\nIt also tried Tesseract, found the available OCR unsuitable for the seventeenth-century Portuguese handwriting, and returned to visual inspection.\n\nThe smaller views exposed the repeated marriage formula, including *recebi por palavras de presente* (“received” is a common language formula for marriages that indicate the bride and groom went to meet the record author), and made the document’s structure legible.\n\nThe agent returned twelve marriage objects and raised the score to 0.26.\n\nThe output still read eleven of the twelve years as 1631 instead of 1636 or 1637, omitted every parent, and recovered many names only partially.\n\n### One Page Where T1 Was Worse[#](#one-page-where-t1-was-worse)\n\nOn this page of four death entries, T1 performed worse than T0 for Gemini 3.5 Flash Lite.\n\nT0 scored 0.90, while T1 scored 0.60.\n\nBoth conditions found four records, classified all four as deaths, and kept them in the correct order.\n\nThe entire 0.30 gap came from the dates.\n\nThe reference year is 1606, which T0 preserved across all four records, with three exact dates and one six days late.\n\nT1 used 1666 for every record, putting all four dates beyond the scorer’s eight-week window and receiving zero date credit.\n\nIt considered 1660 and 1666, opened the complete image, located files, read the schema, and validated the JSON.\n\nA standalone *1606* appears between the first and second entries, yet none of those steps corrected the year.\n\nThe validator could establish only that dates such as `1666-03-07`\n\nmatched the required format.\n\n### A Rare Self-Correction[#](#a-rare-self-correction)\n\nOn this heavily crossed-out page, *Branca* (Blank) is one of the few legible words.\n\nGemini 3.6 Flash initially constructed a complete but unsupported baptism around it: a child named Branca, baptized on 24 November 1688 in Troviscal or Palhaça, with Manuel and Maria as parents.\n\nBranca is, indeed, a valid first name in Portuguese.\n\nThe model then reconsidered the large flourishes, crossed-out writing, and placement of the word.\n\n“I’m now certain ‘Branca’ does not refer to the name of a baptized individual.”\n\nIt interpreted the scan as a page marked blank and returned an empty array.\n\nThe Gemini 3.5 Flash Lite runs in both T0 and T1 also returned an empty array, although their traces did not show the same explicit construction and withdrawal of a full record.\n\n### The Opening Page, Side by Side[#](#the-opening-page-side-by-side)\n\nIf you are curious about the opening page, Gemini 3.6 Flash T1 scored 0.79 on that image. Its final JSON compares with the human reference as follows:\n\n| Record | Field | Human reference (ground truth) | Gemini 3.6 Flash T1 |\n|---|---|---|---|\n| Image | `arquivo` |\n`PT-ADAVR-PAND13-1-3_m00009.jpg` |\n`PT-ADAVR-PAND13-1-3_m00009.jpg` |\n| 1 | `tipo` |\n`batismo` |\n`batismo` |\n| 1 | `data` |\n`1693-08-30` |\n`1693-08-30` |\n| 1 | `local` |\n`Bemposta` |\n`Bemposta` |\n| 1 | `nome_principal` |\n`Maria` |\n`Maria da Assunção` |\n| 1 | `nome_do_pai` |\n`Domingos Jorge` |\n`Domingos Jorge` |\n| 1 | `nome_da_mae` |\n`Isabel Francisca` |\n`Isabel Ferreira` |\n| 2 | `tipo` |\n`batismo` |\n`batismo` |\n| 2 | `data` |\n`1693-09-02` |\n`1693-09-02` |\n| 2 | `local` |\n`Vila` |\n`Vila de Vilarinho` |\n| 2 | `nome_principal` |\n`Antonio` |\n`Antonio` |\n| 2 | `nome_do_pai` |\n`Antonio Francisco` |\n`Antonio Ferreira` |\n| 2 | `nome_da_mae` |\n`Maria Francisca` |\n`Apolonia Garcia` |\n| 3 | `tipo` |\n`batismo` |\n`batismo` |\n| 3 | `data` |\n`1693-09-22` |\n`1693-09-22` |\n| 3 | `local` |\n`Poutena` |\n`Poutena` |\n| 3 | `nome_principal` |\n`Matias` |\n`Matias` |\n| 3 | `nome_do_pai` |\n`Manoel José` |\n`Manoel Ferreira` |\n| 3 | `nome_da_mae` |\n— | `Maria Ferreira` |\n\n*The em dash marks a field absent from the human reference rather than an unread value.*\n\n## Limitations and Conclusions[#](#limitations-and-conclusions)\n\nAcross these four configurations, upgrading the reader buys more quality per dollar than placing Gemini 3.5 Flash Lite inside the harness.\n\nThe score itself requires careful interpretation.\n\nA score of 0.74 includes partial credit for near matches and cannot be read as 74% of records being exactly correct.\n\nThe benchmark is also small and specific: 84 images, 434 records, 17 books, and two Portuguese archives.\n\nI tried to be unbiased, but the images form a curated evaluation set from locations where I have ancestors rather than a random sample of the archives.\n\nI produced the reference transcription myself, without an independent second annotator or paleographic adjudication.\n\nThat keeps the annotation policy consistent, while leaving some historical readings open to human error or interpretation.\n\nDue to self-funded budget constraints, I only generated one answer per image. Without repeated samples, I cannot estimate how much of the image-level difference comes from model stochasticity.\n\nI am keeping the dataset private for now to reduce the chance that it accidentally enters future model training data and contaminate the benchmark.\n\nThese are obscure archival scans, and I consider it unlikely that equivalent high-quality annotations or transcripts already exist in a public corpus.\n\nI would love to see reinforcement learning environments targeting historical and genealogical tasks like this one though.\n\nMy ideal end state is a model reliable enough to extract every person and relationship in a record—including witnesses and godparents—while linking each claim to its source image.\n\nDownstream systems could use these records to build family trees and follow a person through baptisms, marriages, deaths, and appearances in other people’s records.\n\nFreely searchable online, this evidence-linked archive could become an unprecedented map of ordinary lives and the relationships connecting them across centuries.\n\nAnd it’s likely easier than [reading burned scrolls](https://scrollprize.org/)!\n\n## A Note On Verifiers[#](#a-note-on-verifiers)\n\nI first implemented this evaluation with my own custom code.\n\nI later moved the benchmark to [Verifiers](https://github.com/PrimeIntellect-ai/verifiers), Prime Intellect’s open-source framework for building LLM evaluation and reinforcement-learning environments.\n\nI made that migration to learn the framework through a real task whose data and failure modes I already understood, and because the same environment abstraction can later support reinforcement learning.\n\nIn the resulting environment, a Verifiers `Taskset`\n\nloads one multimodal task per image, including the image, prompt, and private reference records.\n\nThe task exposes the ordered Transcription Score as its reward and records the unordered score and invalid JSON rate as additional metrics.\n\nThe execution harness remains a separate choice, which allowed me to run the same taskset through a single model call in T0 or through pi in T1 without rewriting the scoring logic.\n\nVerifiers also preserved the calls, tool interactions, usage, errors, rewards, and metrics that made the case studies and reconstructed cost analysis possible.\n\n**Seja o primeiro a saber das novidades em Machine Learning. Me siga no**", "url": "https://wpnews.pro/news/i-asked-gemini-to-read-300-year-old-portuguese-parish-records", "canonical_source": "https://mariofilho.com/gemini-portuguese-parish-records/", "published_at": "2026-07-28 22:08:46+00:00", "updated_at": "2026-07-28 22:22:28.806166+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-tools"], "entities": ["Google Gemini", "Aveiro District Archive", "University of Coimbra Archive"], "alternates": {"html": "https://wpnews.pro/news/i-asked-gemini-to-read-300-year-old-portuguese-parish-records", "markdown": "https://wpnews.pro/news/i-asked-gemini-to-read-300-year-old-portuguese-parish-records.md", "text": "https://wpnews.pro/news/i-asked-gemini-to-read-300-year-old-portuguese-parish-records.txt", "jsonld": "https://wpnews.pro/news/i-asked-gemini-to-read-300-year-old-portuguese-parish-records.jsonld"}}