Training a 2.

Unbounded Labs has released Bartholomew, a 2.82B-parameter LLM trained on 20.1B tokens of pre-1931 English text, achieving a 60% Model Flops Utilization on a single H100 in a 5-day run. The team cleaned Harvard's Institutional Books dataset from 242B to 23B tokens, built a 20-benchmark suite called Vintage CORE, and released a 416k-question SFT dataset. The project demonstrates that curated vintage data can rival modern-scale training, with all code, weights, and datasets open-sourced.

Training a 2. The core question driving this project is a heavy one: Can an LLM actually reach the same logical conclusions as the great scientists and thinkers of the past, or is it just a sophisticated parrot of modern internet slang? This isn't just a novelty project; it's a deep dive into whether dataset curation and domain expertise can outperform raw scale. The Technical Breakdown of Bart Most people don't realize how much work goes into the "data" side of the AI workflow before you even touch a GPU. The Unbounded Labs team had to clean one of the largest vintage datasets available—Harvard's Institutional Books—which involved shrinking a massive 242B token mess down to a refined 23B tokens. Here is what makes this deployment interesting from a research perspective: Model Scale: 2.82B parameters. Training Data: 20.1B tokens of pre-1931 English. Efficiency: The final training run took only 5 days on a single H100, maintaining a 60% Model Flops Utilization MFU throughout. Benchmarking: Since standard benchmarks like MMLU are useless for 19th-century prose, they built "Vintage CORE," a suite of 20 custom benchmarks specifically for vintage LLMs. SFT Dataset: They released a massive supervised fine-tuning SFT dataset consisting of 416k graded question-and-answer pairs, all grounded in pre-1930s text. Why "Vintage" Matters for AI Research There is a specific type of signal-to-noise problem in modern LLM training. Today’s datasets are filled with Reddit arguments, SEO-optimized blog posts, and social media chatter. By restricting the corpus to pre-1931 text, the researchers are testing the limits of reasoning and linguistic structure without the "pollution" of the modern web. They even ran 10 hours of autonomous research on a single H100, conducting 100 experiments that led to 26 distinct improvements. That is a highly efficient way to approach prompt engineering and hyperparameter tuning when you don't have a massive compute cluster at your disposal. How to check it out If you want to see how a model trained on Victorian-era sensibilities and early 20th-century logic handles a conversation, you can actually interact with it. They've open-sourced almost everything—the datasets, the methodology, the training code, and the evaluations. You can find the model weights and more details here: Huggingface: https://huggingface.co/jbduran/bartholomew-sft Demo: https://www.unboundedlab.com/chat/bartholomew Full Article: https://www.unboundedlab.com/blog/bartholomew It’s a refreshing reminder that sometimes, to understand the future of LLM agents and reasoning, we need to look back at how we used to write. The team is currently looking for compute grants and mentors to scale these experiments further, which is a massive undertaking for a group that's managed to do this much on an $800 budget. A $13B price tag for the AI developer playground everyone 22h ago /en/news/7555/ Hugging Face might actually be worth $13 billion right now 1d ago /en/news/7522/ Developers are losing their minds over a new open-source model 2d ago /en/news/7384/ Open source AI is the only way to stop a few giants from owning 8d ago /en/news/6712/ Open source AI is shifting its center of gravity toward China 9d ago /en/news/6532/ Strands Agents and LeRobot make robotics deployment way easier 10d ago /en/news/6405/ Next Continual Learning can actually produce frontier-tier models → /en/news/7688/