# tufazip: Tufa Labs Squeezes 1 GB of Wikipedia into 105.39 MB With a Transformer

> Source: <https://explainx.ai/blog/tufazip-tufa-labs-neural-text-compressor-enwik9-105-39-mb-large-text-compression-benchmark-2026?utm_source=rss&utm_medium=feed&utm_campaign=blog>
> Published: 2026-11-20 00:00:00+00:00

**Recommendation:** If you train a compressor or model online, keep encoder and decoder updates fully deterministic, since any drift makes the decoder fail to reproduce the file.
More in the playbook "Fine-tuning and data": <https://wpnews.pro/playbooks/training-data>

Tufa Labs says it shrank 1 GB of Wikipedia to 105.39 MB, losslessly, with a single neural network. The lab's [announcement on X](https://x.com/tufalabs/status/2123350647273631907?ref=explainx) calls tufazip "the best entry on the Large Text Compression Benchmark using a neural network as its sole predictor," and adds that everything above it "mixes a transformer with hand-built models."

The size figure holds up. Matt Mahoney's [Large Text Compression Benchmark](https://mattmahoney.net/dc/text.html?ref=explainx) lists tufazip v2 at 105,388,353 bytes total. The cost is the other half of the story: about 7.5 days of compute on two Nvidia B200 GPUs for each direction. This post explains how it works, what is verified, and why a text compressor is a useful lens on language models.

## Quick reference

| Question | Answer | 
|---|---|
| What is it? | A lossless text compressor driven by a transformer's next-token predictions | 
| Result | enwik9 (10^9 bytes of Wikipedia) to 105,388,353 bytes, decoder and weights included | 
| Bits per byte | About 0.843 (the best online-only run reached 0.848) | 
| Model | 12-layer decoder-only transformer, 163M parameters | 
| Hardware and time | Two Xeon Platinum 8570 CPUs, two B200 GPUs; about 7.5 days each way, per the benchmark page | 
| License | MIT for tufazip 0.1.0, per the benchmark page | 
| Author | Tommy He, Tufa Labs | 
| Caveat | Tuned to one file; the author raises a Goodhart concern | 

## Why compression measures language models

A compressor's job is to predict what comes next. Arithmetic coding turns a probability for the next symbol into a number of bits: the more confident and correct the prediction, the fewer bits needed. The ideal code length equals the summed cross-entropy of the model's predictions, which is the same quantity used as the training loss of a language model.

So a language model is a compressor, and a better one compresses more. That is why the [Large Text Compression Benchmark](https://mattmahoney.net/dc/text.html?ref=explainx) exists. It ranks programs by the compressed size, including the decompressor, of enwik9, the first 10^9 bytes of an English Wikipedia XML dump. Counting the decoder stops a program from hiding the data inside its own weights.

For readers who want the vocabulary, we cover [tokens](https://explainx.ai/blog/what-are-llm-tokens), the [context window](https://explainx.ai/blog/llm-context-window-explained-2026) and [quantization](https://explainx.ai/blog/what-is-ai-model-quantization-complete-guide-2026) in separate guides. The same ideas appear here: a model predicts tokens, a window limits what it sees, and quantization shrinks stored weights.

## How tufazip works

The [Tufa Labs write-up](https://tufalabs.ai/research/tufazip/?ref=explainx), dated October 14, 2026, describes the design.

**The predictor.** A 12-layer, decoder-only transformer with 163M parameters. It uses rotary position embeddings, SwiGLU MLPs and RMSNorm. Tokenization is reversible with a 4K word vocabulary. An arithmetic coder turns the predicted probabilities into the output stream.

**Online learning.** The encoder and decoder start from identical weights and run the same loop: predict, encode or decode, update. Because both sides learn the same way from the same decoded text, the learned weights never need to be transmitted. The cost is that updates must be fully deterministic, or the decoder will drift and fail to reproduce the file.

**Replay.** Periodically the model retrains on a window of text it has already coded. The best online run used a 35M-token window and 0.25 replay dropout with a 192-token context. Growing the window from 15M to 35M tokens cut the payload from 107.22 to 105.92 MB, a 1.21% gain, for about 40% more estimated FLOPs.

**An offline prior.** The final model starts from a compact, offline-trained prior. The page says a 34M-parameter offline model was quantized to 4.33 MB. The "Cycle2" layout trains two shared blocks repeated across twelve steps, stores each once, and then copies them into twelve independent layers that specialize during online learning.

The trade is explicit in the page's results table:

| Configuration | Payload (MB) | Decoder and weights (MB) | Total (MB) | 
|---|---|---|---|
| Online v1 | 105.924 | 0.055 | 105.979 | 
| L6 (six-layer prior) | 99.173 | 10.945 | 110.118 | 
| C2 (final) | 100.958 | 4.430 | 105.388 | 

The C2 prior saved 4.97 MB of payload, cost 4.38 MB to store, and netted 0.59 MB, a 0.56% gain over the online v1 run. The stronger L6 prior predicted better but cost too much to ship.

## What the benchmark page confirms

The benchmark's own entry for tufazip, as of this writing, adds independent detail:

- tufazip 0.1.0, submitted September 7, 2026, totals 105,979,069 bytes.
- tufazip v2, submitted September 30, 2026, totals 105,388,353 bytes, using transformer weights trained offline through cycle 2 and quantized. Compression time does not include offline training time.
- Both entries use code in C, Python and CUDA, with preprocessor code from Fabrice Bellard, and are tagged as transformer-based.
- Compression or decompression takes 7.5 days on 2 Intel Xeon Platinum 8570 CPUs with 28 GiB of memory and 2 Nvidia B200 GPUs with 43 GiB of memory.
- The benchmark lists tufazip 0.1.0 as free, open source (MIT); it states no license for the v2 entry.
- The v2 notes give about 7.3 days to compress and 7.5 days to decompress, and the main-table timing columns for v2 repeat the 0.1.0 values, so treat "about 7.5 days" as approximate.

One wrinkle worth flagging: the X announcement is from November 19, but the benchmark entries date from September, and the write-up from October 14. The November post is a public push of an existing result, not a new submission. The write-up says the entry ranked #2 at submission. On the benchmark's main table, sorted by total size, tufazip v2 now shows at 16th, behind cmix-family and other hybrid entries, so the "best neural-only" claim and the overall rank are different things.

## What is verified and what is only claimed

| Claim | Status | 
|---|---|
| enwik9 compresses to 105,388,353 bytes | Matches both the write-up and the benchmark table | 
| Best neural-only entry | Tufa Labs' claim; consistent with the benchmark's own algorithm tags but we did not audit every entry | 
| Everything above it mixes a transformer with hand-built models | Tufa Labs' claim | 
| 7.5 days per direction on two B200 GPUs | Stated on the benchmark page | 
| Beats NNCP v3.2 (107.26 MB) | Stated in the write-up | 
| Hardware and wall-clock in the write-up | Not reported there; the benchmark page supplies them | 

The write-up is open about limits: results apply to one fixed file, so the score may not reflect general ability, and the author raises a Goodhart concern about optimizing this proxy. Larger models did not help in the tested settings: on enwik9 a 267M model was slightly worse than a 188M one, and on enwik8 gains stopped around 77M parameters.

## Why it costs 7.5 days

Decompression mirrors compression. The decoder repeats the same online learning and replay, so it pays the same compute. A normal compressor such as gzip or zstd decompresses in seconds because decoding is a lookup. Here decoding means retraining a transformer as it goes. That is the nature of an online-learning compressor, and it is why this is a benchmark result and not a tool.

If you want a practical angle on compression for LLM workflows, our explainer on [gzip and faster first token in Claude Code](https://explainx.ai/blog/what-is-gzip-claude-code-faster-first-token-compression-explained-2026) covers a case where simple compression helps real latency, and the [KV cache guide](https://explainx.ai/blog/what-is-kv-cache-llm-inference-explained-guide-2026) covers where inference memory goes.

## How tufazip compares with other neural compressors

The write-up compares only with NNCP v3.2 from 2023, at 107.26 MB. Fabrice Bellard's NNCP and his ts_zip are the best-known neural compressors, and the benchmark page names transformer and LSTM models among the leading approaches, along with context mixing. We found no ts_zip figure on the Tufa Labs page, so we do not compare against it here.

The other neural-compression stories we have covered point the same way: models shrink things when they are allowed to do heavy work. See [PrismML's near-lossless compression of a 27B model](https://explainx.ai/blog/prismml-bonsai-2-27b-near-lossless-compression-2026) for compressing model weights instead of text.

## How to try it

The repository is [Tufalabs/tufazip](https://github.com/Tufalabs/tufazip?ref=explainx). The [v2 release notes](https://github.com/Tufalabs/tufazip/releases/tag/v2?ref=explainx) call for Linux x86-64, exactly two Nvidia B200 GPUs and a pinned Python 3.12.3 environment. Treat that as the intended setup. There is no sensible way to run a full enwik9 pass on a laptop.

What you can do without the hardware:

1. **Read the write-up.** The replay and prior-cost trade-offs are a good lesson in why parameter count is not the only lever.
2. **Check the benchmark table.** It lists every entry with its sizes and timings.
3. **Try small files.** Compressing a small corpus will show the idea, though the setup is tuned to enwik9.
4. **Use the idea elsewhere.** Online adaptation plus deterministic updates is a pattern for any system where both ends must stay in sync.

## What people are asking

### Is this the same as ChatGPT compressing text?

It uses the same family of model, a transformer predicting the next token. The difference is that the predictions drive an arithmetic coder, and everything must be exactly reproducible.

### Could this make my downloads smaller?

Not today. At 7.5 days per gigabyte it is a research result.

### Why Wikipedia?

enwik9 is the fixed test file of the benchmark, so results are comparable across years.

### Does a better compressor mean a smarter model?

Better prediction on text means a better model of that text. The author cautions that optimizing one file is a proxy and can be gamed, so it is a signal, not a verdict.

## Related reading

*Figures are accurate as of November 20, 2026. The benchmark table and the repository may change as new entries and releases arrive.*
