Using system-one models inside high-throughput data pipelines

TypeSafe's system-one model Jev cut model costs by 99.56% and raised measured call-time throughput to 7.35× the baseline while staying within 0.5 percentage points of Fable's accuracy on an entity resolution pipeline joining donors and committees across Ohio campaign finance data, according to a TypeSafe writeup by Hrishi Olickel. On a difficult segment of the Ohio dataset, Fable cost $9.83 for 1.2 MB of prepared evidence and took 844.64 seconds, while an optimized hank reduced cost to $0.3012 at 715.08 seconds, and Jev as the primary workhorse inside a five-family review stage with Luna reached $0.0000361 per KB at 10.5 KBps against Fable's 1.4 KBps. TypeSafe estimated a full four-file Ohio snapshot adjudicated by Fable would cost about $9,614, and said Jev is the final piece needed to scale these workflows to terabytes of data.

Using system-one models inside high-throughput data pipelines Hrishi Olickel· Jev TypeSafe's system-one model is a quantum leap forward for data processing pipelines, if harnessed correctly. We used Jev to accelerate an entity resolution pipeline to join donors and committees across Ohio campaign finance data. Hank-assisted resolution pipelines that use Jev as the workhorse reduced model costs by 99.56% and increased measured call-time throughput to 7.35× the baseline while retaining near-Fable quality. But it's not all roses. Applying the wrong signal or prompting the wrong way causes large mistakes at scale. No free lunches, only subsidised ones. We'll talk about the results, explain the task of entity resolution, and at the end we'll provide some prompts and principles that can work for integrating models like Jev into AI data pipelines. Results On a segment of the Ohio dataset that we selected to be difficult, using Fable to join entities costs $9.83 for 1.2 MB of prepared evidence, and takes 844.64 seconds. An optimized hank like the one below is an alternative to direct comparisons using a model or a code-based approach. Here, it reduces model costs to $0.3012 for 715.08 seconds a 32.6× reduction in cost but about the same time . Using only Jev to adjudicate drops accuracy significantly, but Jev as the primary workhorse inside a five-family review stage with Luna provides 226x lower cost $0.0000361 per KB , 7.35x higher throughput 10.5 KBps in call-seconds against Fable's 1.4 , while being within 0.5 percentage points of Fable in accuracy. Jev is the final missing piece for something we set out to do 18 months ago; to scale these workflows up to Terabytes of data. The job The task is simple. Can we resolve entities across the Ohio Campaign Finance dataset? Entities here could be donors, committees, companies, etc. This is the task we have been working on as part of a much larger ingestion and data pipelining test designed to measure 4a, our internal superagent that can autonomously build and self-optimize AI data pipelines. In practice, ER is incredibly difficult. Humans make honest mistakes - middle names are a common example. Filing A Filing B Same person? DANIEL R. OKAFOR DANIEL OKAFOR JR A middle initial on one record, a suffix on the other; same street address. KAREN L. KAREN L An initial and punctuation inside the given-name field. WALTER H. BRENNAN SR. WANDA BRENNAN Shared surname and address, but two different given names. We also move houses and jobs, change names, get married, omit information, and overall shenaniganize our lives. Then there's alleged hostile obfuscation - large donors and companies attempting to spread out detection with different entity names, honest looking mistakes, etc etc. Our ER pipeline resolves these problems using atoms - bundles of equivalent donor evidence. We put rows together into atoms, and models decide on what these bundles are a company, a person, a committee, a joint donor or an unnamed record , and how they're related same person, same household? . As we join more and more atoms, the dataset slowly resolves itself. Here's one case: 11 rows, five atoms, two people. Ordinary code makes the atoms and keeps every original row. Jev judges which atoms represent the same donor, then whether the two donors share a household. The household is a connection between people, not permission to merge them. This is where Jev comes in. If you were so inclined either rich or a frontier lab , you could have a frontier LLM adjudicate every atom and bundle. However - even if you could pay for all of this our estimate for the four-file Ohio snapshot is about $9,614 for Fable - speed makes it difficult to do so. Even the best of us only have about 3 billion seconds of life. Additionally, we have found that at scale, using cheaper models with additional review steps can surpass the accuracy of even frontier models deployed raw. Prompts Here are the things we learned making Jev useful for this task. Telling Jev what counts as enough evidence. Our first questions spent too much time telling it what wasn't proof. This was the original yes criterion for a household link: WrapCopyTextThis specific relationship is supported between distinct compatible entities. This is the criterion we replaced it with: WrapCopyTextBoth records give the same residential street address same house number and street , and the two are distinct persons. The revised instructions also say that DR and DRIVE don't break an address match, and that a shared employer, business address or PO box doesn't establish a household. On the same 13 development questions, Jev went from answering no to all 13 to yes to all 13, matching all 13 reference answers. Those are development results, not the final benchmark score. We made the same change to identity. Here is the actual revised same criterion: WrapCopyTextThe same donor. Sufficient: the same surname and the same or clerically equivalent given name spelling, nickname, initial, or a middle name or suffix present on only one record at the same residential street address, or at the same postal code with the same employer. Differences only in employer/occupation, one blank field, or address formatting do not distinguish donors. Much like prompting in the 3.5-turbo days, more examples and more specific choices make things easier. Same for humans: we all prefer MCQs over essays Conclusion More work remains to be done in integrating intelligence classifiers across data problems, especially when it comes to pipelining the raw performance on offer. This is the largest change we've seen in architecture since the flash gemini, deepseek, glm series of models to change how we structure data work in production. We are already seeing significant bumps in accuracy, cost and speed across other tasks like mining, selection, sanitization, and a lot more. We'll share results as we can make them ready for publication. Data You can download the publication-safe experiment data and the Ohio campaign-finance sources here. Runnable seven-row example and full offline report — extract the bundle, then run bun example.ts --dry-run from the publication folder. This makes no model call. The small example demonstrates classification and joining labels back to rows, not the full resolver.