{"slug": "training-a-2", "title": "Training a 2.", "summary": "Unbounded Labs has released Bartholomew, a 2.82B-parameter LLM trained on 20.1B tokens of pre-1931 English text, achieving a 60% Model Flops Utilization on a single H100 in a 5-day run. The team cleaned Harvard's Institutional Books dataset from 242B to 23B tokens, built a 20-benchmark suite called Vintage CORE, and released a 416k-question SFT dataset. The project demonstrates that curated vintage data can rival modern-scale training, with all code, weights, and datasets open-sourced.", "body_md": "# Training a 2.\n\nThe core question driving this project is a heavy one: Can an LLM actually reach the same logical conclusions as the great scientists and thinkers of the past, or is it just a sophisticated parrot of modern internet slang? This isn't just a novelty project; it's a deep dive into whether dataset curation and domain expertise can outperform raw scale.\n\n## The Technical Breakdown of Bart\n\nMost people don't realize how much work goes into the \"data\" side of the AI workflow before you even touch a GPU. The Unbounded Labs team had to clean one of the largest vintage datasets available—Harvard's Institutional Books—which involved shrinking a massive 242B token mess down to a refined 23B tokens.\n\nHere is what makes this deployment interesting from a research perspective:\n\n**Model Scale:** 2.82B parameters.**Training Data:** 20.1B tokens of pre-1931 English.**Efficiency:** The final training run took only 5 days on a single H100, maintaining a 60% Model Flops Utilization (MFU) throughout.**Benchmarking:** Since standard benchmarks like MMLU are useless for 19th-century prose, they built \"Vintage CORE,\" a suite of 20 custom benchmarks specifically for vintage LLMs.**SFT Dataset:** They released a massive supervised fine-tuning (SFT) dataset consisting of 416k graded question-and-answer pairs, all grounded in pre-1930s text.\n\n## Why \"Vintage\" Matters for AI Research\n\nThere is a specific type of signal-to-noise problem in modern LLM training. Today’s datasets are filled with Reddit arguments, SEO-optimized blog posts, and social media chatter. By restricting the corpus to pre-1931 text, the researchers are testing the limits of reasoning and linguistic structure without the \"pollution\" of the modern web.\n\nThey even ran 10 hours of autonomous research on a single H100, conducting 100 experiments that led to 26 distinct improvements. That is a highly efficient way to approach prompt engineering and hyperparameter tuning when you don't have a massive compute cluster at your disposal.\n\n## How to check it out\n\nIf you want to see how a model trained on Victorian-era sensibilities and early 20th-century logic handles a conversation, you can actually interact with it. They've open-sourced almost everything—the datasets, the methodology, the training code, and the evaluations.\n\nYou can find the model weights and more details here:\n\n```\nHuggingface: https://huggingface.co/jbduran/bartholomew-sft\nDemo: https://www.unboundedlab.com/chat/bartholomew\nFull Article: https://www.unboundedlab.com/blog/bartholomew\n```\n\nIt’s a refreshing reminder that sometimes, to understand the future of LLM agents and reasoning, we need to look back at how we used to write. The team is currently looking for compute grants and mentors to scale these experiments further, which is a massive undertaking for a group that's managed to do this much on an $800 budget.\n\n[**A $13B price tag for the AI developer playground everyone 22h ago](/en/news/7555/)\n\n[Hugging Face might actually be worth $13 billion right now 1d ago](/en/news/7522/)\n\n[Developers are losing their minds over a new open-source model 2d ago](/en/news/7384/)\n\n[Open source AI is the only way to stop a few giants from owning 8d ago](/en/news/6712/)\n\n[Open source AI is shifting its center of gravity toward China 9d ago](/en/news/6532/)\n\n[Strands Agents and LeRobot make robotics deployment way easier 10d ago](/en/news/6405/)\n\n[Next Continual Learning can actually produce frontier-tier models →](/en/news/7688/)", "url": "https://wpnews.pro/news/training-a-2", "canonical_source": "https://promptcube3.com/en/news/7691/", "published_at": "2026-08-25 20:08:26+00:00", "updated_at": "2026-08-25 20:13:42.235966+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-products", "ai-infrastructure"], "entities": ["Unbounded Labs", "Bartholomew", "Harvard Institutional Books", "Vintage CORE", "Hugging Face", "jbduran"], "alternates": {"html": "https://wpnews.pro/news/training-a-2", "markdown": "https://wpnews.pro/news/training-a-2.md", "text": "https://wpnews.pro/news/training-a-2.txt", "jsonld": "https://wpnews.pro/news/training-a-2.jsonld"}}