The core question driving this project is a heavy one: Can an LLM actually reach the same logical conclusions as the great scientists and thinkers of the past, or is it just a sophisticated parrot of modern internet slang? This isn't just a novelty project; it's a deep dive into whether dataset curation and domain expertise can outperform raw scale.
The Technical Breakdown of Bart #
Most people don't realize how much work goes into the "data" side of the AI workflow before you even touch a GPU. The Unbounded Labs team had to clean one of the largest vintage datasets available—Harvard's Institutional Books—which involved shrinking a massive 242B token mess down to a refined 23B tokens.
Here is what makes this deployment interesting from a research perspective:
Model Scale: 2.82B parameters.Training Data: 20.1B tokens of pre-1931 English.Efficiency: The final training run took only 5 days on a single H100, maintaining a 60% Model Flops Utilization (MFU) throughout.Benchmarking: Since standard benchmarks like MMLU are useless for 19th-century prose, they built "Vintage CORE," a suite of 20 custom benchmarks specifically for vintage LLMs.SFT Dataset: They released a massive supervised fine-tuning (SFT) dataset consisting of 416k graded question-and-answer pairs, all grounded in pre-1930s text.
Why "Vintage" Matters for AI Research #
There is a specific type of signal-to-noise problem in modern LLM training. Today’s datasets are filled with Reddit arguments, SEO-optimized blog posts, and social media chatter. By restricting the corpus to pre-1931 text, the researchers are testing the limits of reasoning and linguistic structure without the "pollution" of the modern web.
They even ran 10 hours of autonomous research on a single H100, conducting 100 experiments that led to 26 distinct improvements. That is a highly efficient way to approach prompt engineering and hyperparameter tuning when you don't have a massive compute cluster at your disposal.
How to check it out #
If you want to see how a model trained on Victorian-era sensibilities and early 20th-century logic handles a conversation, you can actually interact with it. They've open-sourced almost everything—the datasets, the methodology, the training code, and the evaluations.
You can find the model weights and more details here:
Huggingface: https://huggingface.co/jbduran/bartholomew-sft
Demo: https://www.unboundedlab.com/chat/bartholomew
Full Article: https://www.unboundedlab.com/blog/bartholomew
It’s a refreshing reminder that sometimes, to understand the future of LLM agents and reasoning, we need to look back at how we used to write. The team is currently looking for compute grants and mentors to scale these experiments further, which is a massive undertaking for a group that's managed to do this much on an $800 budget.
**A $13B price tag for the AI developer playground everyone 22h ago
Hugging Face might actually be worth $13 billion right now 1d ago
Developers are losing their minds over a new open-source model 2d ago
Open source AI is the only way to stop a few giants from owning 8d ago
Open source AI is shifting its center of gravity toward China 9d ago
Strands Agents and LeRobot make robotics deployment way easier 10d ago
Next Continual Learning can actually produce frontier-tier models →