A small, not-so-smart model went from 10% to 99.6% at reading property deeds for $1.15, and it says 'check this line' instead of bluffing

A developer built DeedReader, an open-source tool that parses metes-and-bounds property deed descriptions using a fine-tuned Qwen3-8B model (LoRA, trained on Tinker for about $2.50) and verifies every computed number in deterministic code. In the developer's tests on synthetic deeds, the untuned base model read 10% of deeds exactly right and produced 5 of 60 wrong with no warning, while the tuned system flags mismatches as NEEDS HUMAN REVIEW, points to the offending line, and searches one-digit corrections to close the parcel. The developer cautions that all figures come from synthetic deeds and real-deed accuracy is unmeasured.

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady https://dev.to/challenges/hacktoberfest-week3-2026-10-19 N 86°29' E, 103.2 feet. That is one line of a property deed. Walk the rest of the lot and you should end up where you started. You end up 27 feet away. One digit is wrong. Maybe a clerk typed 103.2 for 130.2 . Maybe the scan smudged. Maybe the AI that read it got creative. The drawing you'd make still looks fine, and nobody finds out until the survey crew does. DeedReader reads that deed, closes the shape in plain code, plots it on a real satellite map, and when something doesn't add up it refuses to draw it. It stamps the result NEEDS HUMAN REVIEW , points at the line, and tries every one-digit fix to find the one that closes the parcel. In the example above: 103.2 → 130.2 . Every parcel of land has a legal description, and it reads like this: Commencing at a cedar post at the corner of Tract 7, thence S 46°30′11″ W 1,011.25 feet to a corner post, thence along a curve to the right, R = 757.19 feet, to a point in the center of the road, thence N 23°16′52″ W 1,206.14 feet to a 4 inch concrete monument… Someone has to turn that into a drawing. Convert every bearing. Plot every side, curved ones included. Check the shape closes. Work out the acreage. That someone is a surveyor, a title abstractor or a paralegal, and the job takes fifteen to thirty minutes a deed . The scary part isn't the time. It's that one wrong digit quietly poisons everything downstream , and the drawing still looks plausible. The failure you have to fear isn't "the tool is wrong." It's "the tool is wrong and didn't tell me." | Option | What it gives you | What it can't tell you | |---|---|---| | By hand | A drawing you trust | It takes 15–30 minutes a deed | | A regex parser | Instant and free | Only reads clean N 41°00' E calls, not 1800s prose or numbers spelled out in words | | A base open model, no training | Reads messy prose | 10% exactly right in my test, and 5 of 60 wrong with no warning at all | | DeedReader | A closed parcel on a real map in seconds | Says NEEDS HUMAN REVIEW when anything doesn't add up, and points at the line | DeedReader is the lazy version of that job: paste, glance, approve. Why it fits Slow and Steady: the tortoise didn't win by being fast. It won by never doing something stupid. A small model, a dollar of training, plain code for every number, and a system that would rather say "I don't know" than be quick and wrong. It automates the slowest job in land work, and earns trust the slow way: by showing its arithmetic. The video above is the whole tour. The studio replays real recorded outputs of my fine-tuned model on deeds it never saw in training, while every safety check runs live in your browser . Paste, glance, approve. A clean deed: every check passes, the parcel closes to 1:2.9 million, and the studio reports 43.44 acres. Click any sentence and its side lights up in the drawing. Export to SVG, DXF or GeoJSON in one click. Break it on purpose. "Simulate a misread" corrupts a single number. Three independent checks fail at once, the banner turns red, and it names the line: And the lazy payoff, side by side: Paste a metes-and-bounds legal description. Get structured calls, a closure check done in plain code, a plotted parcel, and an honest NEEDS HUMAN REVIEW whenever anything is off. A fine-tuned open-weight model Qwen3-8B + LoRA, trained on Tinker https://thinkingmachines.ai/tinker/ for about $2.5 does the reading; deterministic code does everything that must be right. Built for the DEV Hacktoberfest Open-Source AI Challenge, Week 3. Read EVAL REPORT.md https://github.com/advikdivekar/deed/EVAL REPORT.md before trusting any number here: everything is measured on synthetic deeds, and real-deed accuracy is unmeasured. | base Qwen3-8B, 3-shot | tuned v2 | | |---|---|---| | deeds read exactly right in-distribution | 10% n=60 | 99.6% n=3,000 | | flagged for a human | 82% | 0.4% | | silent errors wrong AND unflagged | 5 of 60 | 2 of 3,000 0.07%, 95% upper bound 0.24% ; 0 if 87 link = 0.87 chain | | training cost | n/a | about $1.15 est. | Unseen style/unit, zero-shot: spelled-out numbers 85.6% exact 14.4% flagged, 0 silent … The rule behind all of it: the model reads, code decides. Language models read messy text brilliantly and are not accountable for arithmetic. So Qwen3-8B plus a LoRA does exactly one job, turning prose into structured calls. Everything after that is plain code with tests: closure, area, unit conversion feet, chains, poles, varas, links and plotting. Then five independent checks have to agree before an answer may look confident: The data: truth first, prose second. There is no free pile of labelled real deeds, so I turned the problem around. A generator draws a closed polygon with known bearings and distances , then writes it up in many styles: symbols, dashes, words, archaic chains and links, and damaged OCR, with easements, deed-book numbers and stray acreage as distractors. The truth exists before the text does, so every label is exact. I deliberately held back one writing style every number spelled out in words and one unit Texas varas to see what the model does with things it has never met. Qwen3-8B is a small model. Out of the box it doesn't know what a deed is, and it has nowhere near the knowledge of a giant LLM. It scored 10%. Same model, same prompt, same checks: the only thing that changed was one dollar and fifteen cents of Tinker training. That's the point. I didn't need a model that knows everything. I needed one that does one narrow job perfectly, and fine-tuning is how a small model gets there. | | base Qwen3-8B, untrained | Tinker LoRA | |---|---|---| | Exactly right | 6 / 60 10% | 2,988 / 3,000 99.6% | | Sent to a human | 82% | 0.4% | | Wrong and unflagged | 5 / 60 | 2 / 3,000 0 by physical length | | Cost | n/a | $1.15 | My goal wasn't a good number. It was a good number at the lowest price I could defend. Here is exactly how. Tinker bills per token, so I started with the output. My first format was JSON at 547 tokens per deed. I replaced it with one line per call: POB iron pin L N 41 0 0 E 121.9 ft | iron pin C N 3 45 22 E 366.15 ft R 944.33 ft L | stone Code converts it back to JSON. 145 tokens, 3.8× smaller , which cuts training, evaluation and local inference all at once, and costs nothing to run. Training per million tokens: Qwen3-8B $0.44 vs Qwen3.5-4B $0.737. A smaller model is not automatically a cheaper one. Qwen3-8B was cheaper to train and strong enough for the job. tc = await svc.create lora training client async base model="Qwen/Qwen3-8B", rank=32 loss only on the answer tokens, never on the deed prose Datum model input=full :-1 , loss fn inputs=dict weights= 0.0 len prompt -1 + 1.0 len completion , target tokens=full 1: fb = await tc.forward backward async batch, loss fn="cross entropy" op = await tc.optim step async AdamParams learning rate=lr at s , beta1=0.9, beta2=0.95, eps=1e-8 2,300 training deeds , batch 16, peak learning rate 2e-4 with warmup and linear decay, 429 steps, about 2.6M tokens. Loss is masked to the answer, so the model learns to write calls instead of memorizing deed prose. Steps are pipelined step k+1 is submitted before step k returns , a checkpoint is saved every epoch, and a budget guard prices every job from token counts and refuses to launch one that would burn half the cap. Loss falls from 1,806 on step 1 to under 0.1 by step 41. I didn't train on everything and hope. Four runs, one epoch each, scored on the same 500 held-out deeds: | training deeds | est. cost | read exactly right | wrong and unflagged | |---|---|---|---| | 250 | $0.04 | 38.6% | 19 | | 500 | $0.08 | 92.6% | 0 | | 1,000 | $0.17 | 99.4% | 0 | | 2,000 | $0.34 | 99.6% | 0 | Seventeen cents buys 99.4%. And look at the zeros: between 250 and 500 deeds the model stops making silent mistakes. That jump, not the accuracy, is where it becomes safe to trust. I paired the base model against the fine-tune on the same 60 deeds , deed by deed: | | base, untrained | fine-tuned | |---|---|---| | exactly right | 6 / 60 10%, CI 4.7–20.1% | 60 / 60 | | wrong and unflagged | 5 | 0 | | deeds only this model got right | 0 | 54 | Every deed the base model got right, the fine-tune also got right, and it won the other 54. Paired sign test: p < 0.0001. Then the real exam: 3,000 fresh deeds , generated before I froze the validator and scored exactly once. 2,988 exactly right. 99.6%. Of the 12 imperfect outputs, 10 were caught by parse and closure alone. The other 2 are one ambiguity in my own generator: it writes 87 links as "87 link" in one style and "0.87 chain" in another, which is the same physical length. By physical length, the silent count is 0 . I report both. Held out on purpose, zero-shot, from the first model: | unseen test | exactly right | wrong and unflagged | |---|---|---| | numbers spelled out in words | 85.6% | 0 14.4% flagged for review | | Texas varas | every deed flagged | 0 | Varas is the fun one. The model read every bearing and distance correctly but wrote the unit as var , a token it had never seen, and the strict decoder rejected it. So everything was flagged and nothing wrong got through. That is the safety net doing its job on a surprise. I didn't patch the decoder to accept each spelling the model improvised, because that would be fitting the test set. The real fix was data: 150 examples each , retrain, score on fresh deeds: | fresh test set | n | exactly right | wrong and unflagged | |---|---|---|---| | in distribution | 3,000 | 99.6% | 2 0 by physical length | | spelled-out numbers | 250 | 99.6% | 0 | | varas | 250 | 100% | 0 | Onboarding a new style or unit is a 150-example, sub-dollar job. That's the workflow Tinker makes cheap enough to just do. Evaluation was the bigger bill $1.35 vs $2.46 for all training , so every sampled output is cached and the checks score from the cache. Improving the validator and re-scoring 3,000 deeds cost $0. One deed through Tinker costs about $0.00015. I downloaded the 369 MB adapter and ran it with MLX on the Apple GPU over a 4-bit Qwen3-8B base: 150 of 150 deeds read exactly right, 0 flagged, median 7.5 s per deed. M3 Pro, 18 GB, no network, no API key. | item | est. cost | |---|---| | baseline, no training | $0.11 | | learning-curve runs | $0.29 | | first full run | $1.01 | | final model 2,300 deeds, 3 epochs | $1.15 | | all evaluation 4,000+ deeds | $1.35 | | everything on Tinker | about $3.80 | Estimated from token counts at Tinker's list prices. I'd rather you hear these from me, with numbers. ; or thence What went wrong on the way and made it better : my validator flagged perfect answers until I ran it on the generator's own ground truth and fixed three reading bugs. Two silent errors slipped past closure at 500 deeds a dropped arc-minute and a flipped curve direction , which is exactly why the independent second reader exists. A closed API gives you a prompt box. Open weights gave me a steering wheel. The honest caveat: training ran on Tinker's servers on generated deeds only. No real person's land record was ever sent anywhere, and at run time nothing leaves the laptop. I built DeedReader with Claude Code as a pair, from a written plan with a cost estimate up front and a budget guard on every training job. Best Use of Tinker. The LoRA fine-tune pipelined forward/backward passes, loss-masked completions, per-epoch checkpoints , the baseline, the data-size learning curve, 4,000+ evaluated deeds and the adapter download behind the offline story all ran on Tinker. $1.15 for the final model, about $3.80 for everything. Same base model, same prompt, same checks: 10% 6/60 → 99.6% 2,988/3,000 , and every deed the base model got right, the fine-tune got right too. Built solo, with Claude Code. Three epochs. $1.15. 10% became 99.6%. If that sounds like a good trade, a ❤️ or 🦄 helps more people find it. What's the strangest phrase you've ever seen in a property deed? Mine so far is "to a white oak, thence along the run." Leave yours in the comments.