{"slug": "overfitted-a-900kb-transformer-to-compress-a-100mb-csv-into-7mb", "title": "Overfitted a 900KB Transformer to Compress a 100MB CSV into 7MB", "summary": "A developer built an experiment using an overfitted 900KB transformer and arithmetic coding to compress individual files, achieving 7MB compression on a 100MB NYC taxi CSV and 21MB on a 100MB enwik9 slice, though compression and decompression are slow at around 45 minutes each.", "body_md": "| ||||||||||||||||||||||||||||||||||||||||||||\n22 points by |\nI built an experiment that uses an overfitted transformer and arithmetic coding to compress individual files. Instead of training the model to generalize, I train a 900KB transformer to memorize a single file and predict the next byte. Those predictions are fed into an arithmetic coder to produce the compressed output. On a 100MB NYC taxi CSV, it compresses to about 7MB (~0.5 bits/byte). On a 100MB slice of enwik9, it compresses to about 21MB (~1.68 bits/byte). It's pretty slow right now (roughly 20–30 minutes of training and 45 minutes each for compression and decompression on my AMD 7800XT). Checkout the repo - https://github.com/samyak112/pym-particles\n| |||||||||||||||||||||||||||||||||||||||||||\n|", "url": "https://wpnews.pro/news/overfitted-a-900kb-transformer-to-compress-a-100mb-csv-into-7mb", "canonical_source": "https://news.ycombinator.com/item?id=48644463", "published_at": "2026-06-23 13:11:27+00:00", "updated_at": "2026-06-26 03:33:18.879347+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-research"], "entities": ["AMD 7800XT", "NYC taxi CSV", "enwik9"], "alternates": {"html": "https://wpnews.pro/news/overfitted-a-900kb-transformer-to-compress-a-100mb-csv-into-7mb", "markdown": "https://wpnews.pro/news/overfitted-a-900kb-transformer-to-compress-a-100mb-csv-into-7mb.md", "text": "https://wpnews.pro/news/overfitted-a-900kb-transformer-to-compress-a-100mb-csv-into-7mb.txt", "jsonld": "https://wpnews.pro/news/overfitted-a-900kb-transformer-to-compress-a-100mb-csv-into-7mb.jsonld"}}