Taco: Cross platform LLM based compressor, eliminates floating point

Taco 0.1.0, an alpha proof-of-concept lossless compressor released on 2026-10-19, folds and unfolds a 3,672-byte English text sample to 648 bytes with an identical SHA-256 hash (2fab1ef7a1c70cd4e39ddfd88a8d66389ae07a403111db131326ae425460652e) across 7 backends spanning 3 operating systems, 2 CPU architectures and 4 processor types. Taco's HardShell engine runs a GPT-2 language model in integer arithmetic only, so every CPU and GPU produces bit-identical predictions and a file folded on one machine unfolds on all the others. The author states most of the code, tests and documentation were written by Anthropic's Claude under his direction, and warns Taco is slow and not a replacement for zstd or xz.

Hard shell. No floats. Same taco on every machine. Taco is a lossless compressor that uses a language model GPT-2 . The model predicts the next piece of the data, and a range coder stores only what the model did not predict. The engine of Taco, HardShell , runs the model with integer arithmetic only . Thus every CPU and GPU calculates the same predictions, bit for bit. A file that one machine compresses "folds" decompresses "unfolds" on every other machine. taco fold notes.txt makes notes.txt.taco taco unfold notes.txt.taco restores notes.txt, byte for byte Warning Alpha: a proof of concept version 0.1.0, 2026-10-19 . Taco is an experiment. It is slow, and it is not a replacement for zstd or xz. Keep a second copy of data that you fold. Read Limits limits before you use it. Note How this code was written. Most of the code, the tests and the documents in this repository were written by Claude an AI model by Anthropic with Claude Code, under the direction of the author, to get to results faster. The author made the design decisions. Trust the code only as far as the tests show: conformance vectors, the cross-machine check, fuzzing and the comparison of every fast path with the reference path. This is the point of Taco. Speed and ratio come second. The release program of version 0.1.0 folded the demo sample demo/samples/sample.txt , 3,672 bytes, --profile text and unfolded it again on 7 backends: 3 operating systems, 2 CPU architectures and 4 kinds of processors. Each fold ran on the backend in the row, and each unfold restored the sample byte for byte measured, 2026-10-19 : | Machine | System | Backend | SHA-256 of the .taco file | |---|---|---|---| | Apple M2 | macOS ARM | CPU NEON | 2fab1ef7a1c70cd4e39ddfd88a8d66389ae07a403111db131326ae425460652e | | Apple M2 | macOS ARM | GPU Metal | 2fab1ef7a1c70cd4e39ddfd88a8d66389ae07a403111db131326ae425460652e | | AMD Ryzen 7 5800H | Linux x86-64 | CPU AVX2 | 2fab1ef7a1c70cd4e39ddfd88a8d66389ae07a403111db131326ae425460652e | | NVIDIA RTX 3060 Laptop | Linux x86-64 | CUDA | 2fab1ef7a1c70cd4e39ddfd88a8d66389ae07a403111db131326ae425460652e | | AMD Ryzen 7 5800H | Windows x86-64 | CPU AVX2 | 2fab1ef7a1c70cd4e39ddfd88a8d66389ae07a403111db131326ae425460652e | | NVIDIA RTX 3060 Laptop | Windows x86-64 | CUDA | 2fab1ef7a1c70cd4e39ddfd88a8d66389ae07a403111db131326ae425460652e | | NVIDIA RTX 3060 Laptop | Windows x86-64 | GPU Vulkan | 2fab1ef7a1c70cd4e39ddfd88a8d66389ae07a403111db131326ae425460652e | Thus a file that any of these machines folds unfolds on all of the others. To add a row, run ./try on your machine Try it try-it and compare the hash. More checks of the same property, with more inputs, are in Same bits on different machines same-bits-on-different-machines . There are two ways. Both give the same .taco bytes. 1. Open the Releases page https://github.com/tac0mania/taco/releases and download taco-demo-0.1.0.zip about 120 MB . It has the programs for macOS Apple Silicon , Linux x86-64 and Windows x86-64 , the model file gpt2.hsm , and a one-command demo. If you want only the program for one system, download taco-0.1.0-<system .tar.gz or .zip for Windows ; it does not include the model file. 2. Unpack the zip and open a terminal in the unpacked folder taco-demo-0.1.0 . 3. Run ./try macOS, Linux or try.cmd Windows . The demo folds and unfolds a short English text 3,672 bytes , checks it, and prints the SHA-256 of the .taco file. It takes about 10 s to 1 minute. The end of the output: PASS: the file came back byte for byte. Original: 3672 bytes. Folded: 648 bytes. SHA-256 of the .taco file: 2fab1ef7a1c70cd4e39ddfd88a8d66389ae07a403111db131326ae425460652e This hash is the same on every machine and every backend. Run the demo on a second computer and compare. To fold your own files, use ./taco fold notes.txt and ./taco unfold notes.txt.taco in the same folder Windows: taco.cmd . Details and troubleshooting: demo/README.md https://github.com/tac0mania/taco/blob/main/demo/README.md . You need: rustup https://rustup.rs it installs Rust 1.99.0 from rust-toolchain.toml , uv https://docs.astral.sh/uv/ for the Python tools, a C compiler for zstd , about 1.5 GB of disk and 8 GB of memory. A GPU is optional. git clone https://github.com/tac0mania/taco.git && cd taco uv run --project tools python -m baseline.fetch GPT-2 weights; the tool checks each SHA-256 uv run --project tools python -m convert.gpt2 makes tools/data/out/gpt2.hsm about 1 minute shasum -a 256 tools/data/out/gpt2.hsm must be aaa03eb96e9ad795...b51001ba5c2 cargo build --release --locked -p taco-cli export TACO MODEL="$PWD/tools/data/out/gpt2.hsm" ./target/release/taco fold demo/samples/sample.txt --profile text -o /tmp/sample.taco shasum -a 256 /tmp/sample.taco must be the hash in section A ./target/release/taco unfold /tmp/sample.taco -o /tmp/sample.txt cmp demo/samples/sample.txt /tmp/sample.txt && echo "restored byte for byte" - The converter calculates with exact fractions, so your gpt2.hsm is the same file, byte for byte, as the one in the release. If its hash is different, do not use it. - Your .taco file must have the hash of section A. Then your build and the release program are compatible: each one unfolds the files of the other. - scripts/release/build.sh <target makes the release archive itself. It builds with fixed paths and times, so two builds of one commit on one machine gave the same binary, byte for byte measured on Linux . BUILDINFO in each archive records the commit and the SHA-256 of the program. Caution Taco is slow. With a GPU, it folds text at about 85 to 129 KB/s and unfolds at about 26 to 31 KB/s. With the CPU only, it folds at about 0.3 to 2 KB/s and unfolds at about 0.1 to 0.5 KB/s measured . Start with a file of a few KB. Taco is a compressor whose output does not depend on the machine. Its parts are: - an open specification of integer-only GPT-2 inference docs/adr/0004-hardshell-numerics.md ; - an exact converter: it reads each float32 weight as mantissa x 2^exponent and calculates with exact fractions, so every machine makes the same model file ADR 0014 ; - implementations for the CPU NEON, AVX2 , NVIDIA GPUs CUDA and other GPUs wgpu: Vulkan, Metal, DX12 , which tests show give the same bits; - conformance vectors that connect the model to the bytes of real .taco files. Taco is not the first or the best compressor with a language model: - ts zip https://bellard.org/ts zip/ Fabrice Bellard makes smaller files and is faster. Its page also states that its result does not depend on the GPU or the CPU. - The ideas under Taco have earlier work: language models with entropy coders LLMZip https://arxiv.org/abs/2306.04050 , integer networks for compression that decode the same on every platform Ballé et al., 2019 https://research.google/pubs/integer-networks-for-data-compression-with-latent-variable-models/ , and integer-only Transformers I-BERT https://proceedings.mlr.press/v139/kim21d.html . What Taco adds is the combination: an inspectable integer specification, a repeatable model conversion, several independent implementations, and tests that show they agree. All numbers are measured unless they are labelled "published" or "est.". Details are in docs/research/ . The sizes do not include the model file 127 MB , which both machines must have. | File | Original | Taco | xz -9e | zstd -19 | |---|---|---|---|---| | English book 32 KiB | 32,768 | 8,432 | 12,400 | 12,463 | | HTML page | 24,603 | 4,759 | 7,652 | 7,713 | | English 4 KiB | 4,096 | 1,237 | 1,684 | 1,599 | | Server log 16 KiB | 16,384 | 3,129 | 3,164 | 3,139 | | Excel file binary | 8,192 | 8,285 stored raw | 1,672 | 2,097 | Against another compressor with a language model, on the whole alice29.txt 152,089 bytes : | Compressor | Bytes | Bits per byte | |---|---|---| | xz -9e | 48,492 published | 2.55 | | Taco | 38,939 | 2.05 | | ts zip | 21,713 published | 1.14 | ts zip is 44% smaller. It uses a larger and newer model RWKV, 169M parameters; GPT-2 small has 124M , and Taco's profile text restarts the model context at each 16 KiB block. The 16 Loghub log files 4,458,875 bytes in total , the defaults of the profile logs blocks of 64 KiB, context overlap 512 , whole files for the other compressors docs/research/2026-10-19-overlap-cli.md , docs/research/2026-10-19-logs-profile.md : | Compressor | Bits per byte | |---|---| | zstd -19 --long=27 | 0.597 | | xz -9e | 0.554 | | Taco, profile logs | 0.494 10.8% smaller than xz | | paq8px -8 a slow context-mixing compressor | smaller than Taco: 0.348 against 0.600 est. on 6 other whole logs | Speed of the profile logs with the GPU mixer RTX 3060 at 95 W, CUDA, commit 1493af3 : | File | Fold | Unfold | |---|---|---| | Apache 2k.log , 171 KB | 4.5 s was 208.8 s on the CPU | 53.8 s was 90.1 s | | apache access.log , 2.37 MB | 70 to 98 s | 255 to 280 s | What can make logs smaller not in this version : the skip rule, 4.8 to 7.3% smaller est., simulation, ADR 0017 ; and a model fine-tuned on the logs of known systems, 53.2% smaller on those systems but 4.6% larger on other systems measured, frozen; Experiments experiments-that-do-not-ship . | Model | Bits per byte 16 text files, 42,256 bytes | |---|---| | GPT-2 small, float | 1.867 | | GPT-2 small, integer gpt2.hsm | 1.871 +0.2% | The +0.2% is a total. Single files change from −9.0% to +6.6%. The conversion saturates large activations, which changes the predictions; against a float model with the same saturation, the integer model is 2.3% larger docs/research/2026-10-18-integer-gpt2.md §6 . | What | Speed | |---|---| | Engine, decode over a full 1,024-token window | 20,472 tokens/s | | taco fold , English text | 85 to 129 KB/s | | taco unfold , English text | 26 to 31 KB/s | | taco fold , CPU only | about 0.3 to 2 KB/s | | taco unfold , CPU only | about 0.1 to 0.5 KB/s | For comparison, zstd -3 compresses at hundreds of MB/s and xz -9e at about 0.7 MB/s. ts zip states "up to 1 MB/s" on an RTX 4090 published, a different GPU . This test checks that the memory does not grow with the file ADR 0018 . It used 200,000,000 bytes of random words and the small test model, on the devbox CPU. | Run | Largest memory RSS | |---|---| | fold, file to file | 254 MiB | | fold, stdin to stdout | 276 MiB | | unfold, file to file | 38 MiB | | unfold, stdin to stdout | 35 MiB | Both unfolds restored the input byte for byte. Its times say nothing about Taco's speed: the test model has random weights, so every block was stored with zstd or raw, and unfold ran no model step. The real speed is in the table above. The largest files folded with GPT-2 so far are 2.4 to 2.7 MB measured . | Check | Result | |---|---| | Mac ARM folds, Linux x86 unfolds, and the reverse scripts/xcheck , 12 inputs | 12 of 12 restored each way; 12 of 12 .taco files identical | | Native Windows x86 against the Mac scripts/wcheck | 9 of 9 .taco files identical, 9 of 9 restored | | CPU, CUDA and wgpu Metal, GL | the same bytes, 14 of 14 GPU cases | | CI on each push: Linux x86-64 and Linux ARM64 | the same 50 conformance hashes | | Fuzzing: damaged containers, coder streams and model files | millions of cases, 0 failures docs/security.md | - No floating-point arithmetic in fold or unfold. The engine, the coder, the mixer and the GPU kernels use integers only. scripts/check-no-floats.sh checks this on each push. - The same result on every machine, when each operation is fully specified. Integers alone are not sufficient. Taco makes these conditions rules CLAUDE.md §1 : - every sum has an accumulator that is wide enough, with the bound written in the code, or it wraps on overflow in a defined way; - every rounding and every clamp has one written rule ADR 0004 ; - signed division uses floor-division helpers. Then a compiler, a CPU type or a GPU driver has no freedom to change a result. Tests check these conditions. Tests cannot prove that the code obeys them everywhere. - The model file is the same on every machine. The Mac and Linux make the same gpt2.hsm , byte for byte. Its SHA-256 is aaa03eb96e9ad7957432168c8ac6b4a99ed55d57e637e4786a664b51001ba5c2 . - A wrong result is never silent. Each .taco file stores the SHA-256 of the original data, and taco unfold checks it. Important The checksum finds an error. It cannot repair one. If taco unfold fails, try again with --backend cpu or on a different machine. If the error happened during taco fold a bug or a hardware fault on that machine , no decoder can restore the file. Keep the original until you have unfolded the .taco file one time. These problems can still occur, as in every program: a bug in our code, a bug in a GPU driver Taco runs a self-test before it uses a GPU, and uses the next backend if the test fails , or a hardware fault. In each case the SHA-256 check finds the error. | Hardware | Backend | Result | |---|---|---| | Apple M2 macOS | CPU NEON , GPU wgpu on Metal | bit-exact | | AMD Ryzen 7 5800H Linux, native Windows | CPU AVX2 | bit-exact | | NVIDIA RTX 3060 Laptop | CUDA | bit-exact | | NVIDIA RTX 3060 Laptop | wgpu on OpenGL Linux , DX12 with DXC Windows | bit-exact | | AMD Radeon iGPU Ryzen 7 5800H , native Windows | wgpu on DX12 with DXC | bit-exact | | NVIDIA RTX 3060, AMD Radeon iGPU, native Windows | wgpu on Vulkan | bit-exact 4 op tests; the full model tests did not run yet | | GitHub Actions: Linux x86-64, Linux ARM64 | CPU | the same 50 conformance hashes | Not tested yet: Intel GPUs; AMD GPUs on Linux; AMD discrete GPUs; macOS on Intel; big-endian and 32-bit CPUs; phones and browsers. GPU notes: - AMD and Intel GPUs need no special code. They use wgpu and the normal GPU driver. - The CUDA backend needs compute capability 8.0 or more RTX 30 series, A100 and later . Older NVIDIA GPUs use wgpu or the CPU. The bits are the same; only the speed changes. - On Windows, DX12 with the old FXC shader compiler cannot compile 7 of the 14 kernels. The self-test then fails, and Taco uses the next backend. Vulkan, Metal and DX12 with DXC work. | Limit | Details | |---|---| | Slow | See Speed speed-rtx-3060-laptop-gpu-95-w . A GPU is strongly recommended. | | Small files unfold slowly | One block decodes one token after the other: about 10 to 13 s for a 16 KiB block on the RTX 3060 est. from measured times . | | English only | GPT-2 is an English model. On German and French, Taco is larger than PPMd; on Chinese, larger than gzip measured . | | Binary data | Photos, video, archives and other binary files are stored with zstd or raw. Use zstd for them. | | Logs | The profile logs is smaller than xz, but paq8px is smaller still measured . | | The model file | 127 MB. Both machines must have the same file. The release does not download it yet. | | Untrusted model files | A model file can state a very large KV cache finding F6, open . Load only the published gpt2.hsm . | | Wide models | Model-file version 2 runs on the CPU and with CUDA, not on wgpu yet. The default gpt2.hsm is version 1 and runs everywhere. | | Format | Container version 2 is frozen: later versions of Taco read the files of this version. | | Release 0.1.0-alpha.1 | Its programs fold the default profile auto on the CPU use --profile text for the GPU , and a crafted .taco file can use about 11 GB in GPU unfold F2; use --backend cpu for untrusted files . Both are fixed on main ; the GPU logs speeds above are from main . | A symbol that the model expects with probability p costs about −log₂ p bits. For each position, the model gives a probability to each of the 50,257 GPT-2 tokens. The range coder divides a number range in proportion to these probabilities. The decoder runs the same model, divides the range in the same way, and finds the token. The decoder must calculate exactly the same probabilities as the encoder. If one frequency is different by one unit, the decoder can select a wrong token, and all output after it is wrong. Floating-point results can change with the order of additions, the chip, the math library and the batch size. Specified integer results do not change. Thus HardShell uses i8 weights, i32 and i64 sums, one rounding rule, and tables for GELU and exp that are stored in the model file. php flowchart LR A input bytes -- B blocks B -- C tokenizer<br/ byte-level BPE C -- D HardShell<br/ integer GPT-2 D -- E frequency table E -- F range coder F -- G .taco file<br/ model hash + SHA-256 G -. unfold: same model,<br/ same table .- H original bytes<br/ checksum verified - Blocks are independent, so they run in parallel, and damage stays in one block. - A block that does not become smaller is stored zstd or raw . The worst-case growth is the 104-byte header and 5 bytes for each block. - The profile logs adds a match model and a mixer for repeated lines. The profile auto the default selects the smaller result for each block. - The file format is in ADR 0006 and ADR 0011. The model file format is in ADR 0004. The full explanation of every part, with the measurements and the reasons for each decision, is in docs/core-architecture.md https://github.com/tac0mania/taco/blob/main/docs/core-architecture.md . If you are new to compression or machine learning, start with docs/primer.md https://github.com/tac0mania/taco/blob/main/docs/primer.md : it explains IEEE 754 floating point, entropy, range coding, transformers and quantisation from the start. See Try it, B b-build-the-same-program-yourself . taco fold <input -o <output -c --profile text|logs|auto --backend auto|cuda|gpu|cpu taco unfold <input.taco -o <output -c --backend auto|cuda|gpu|cpu | Option | Meaning | |---|---| | -o <path , -c | The output file, or stdout. The input - is stdin. | | -m <path.hsm | The model file. Default: the TACO MODEL environment variable. | | --profile | Fold only. text GPT-2 , logs GPT-2, match model, mixer or auto default . | | --backend | cuda , gpu wgpu , cpu or auto default: the first that passes a self-test . The output is the same on every backend. | | --block-size , --overlap | Fold only. Block size default 16 KiB for text , 64 KiB for logs and auto and context overlap. | | --threads <n | Worker threads. The output does not change. | | -f , -q , -V | Replace an output file; no progress output; show the version. | Files of any size stream with bounded memory Large files large-files-streaming-a-memory-test-not-a-speed-test . taco --help and docs/man/taco.1 have all the details. The alpha shows one property: the same bits on every machine. These items come later. None of them changes that property, and files from the alpha stay readable. | Item | Expected gain | |---|---| | Speculative decoding in taco unfold ADR 0017 | Unfold 1.8 to 3.7 times faster est., simulation | | Skip rule mixer rule 2 for the profile logs | Logs 4.8 to 7.3% smaller est., simulation on 18 log files | | Smaller blocks for small text files | Unfold of a 25 KB file from 13.7 s to 4.7 s, for 2.3% more bytes; a 427 KB file from 17.7 s to 10.7 s, for 1.0% more measured | | More context for the profile text | Smaller files not measured | | Range coder on the GPU | Removes one CPU-GPU round trip for each token not measured | | CUDA for compute capability 6.1 and 7.5 | Faster on older NVIDIA GPUs not measured | | Model download in taco docs/design/model-distribution.md | No manual model setup | | Other models with an automatic converter docs/design/anymodel.md | Designed | A model fine-tuned on logs "Route A", frozen . distilgpt2 6 layers fine-tuned on 12 Loghub systems, against GPT-2 and the general compressors. taco fold --profile logs , the first 16 KiB of each file, sizes in bytes measured, docs/research/2026-10-19-route-a.md §8 : | Sample 16,384 bytes each | xz -9e | zstd -19 | Taco, GPT-2 | Taco, fine-tuned | |---|---|---|---|---| | 4 log systems seen in training Android, HDFS, Mac, OpenStack | 10,872 | 10,897 | 7,672 | 3,593 | | 4 log systems not seen in training Apache, HealthApp, Proxifier, Zookeeper | 4,664 | 4,578 | 3,709 | 3,879 | - On seen systems, the fine-tuned model is 67% smaller than xz and 53% smaller than GPT-2. - On unseen systems, it is 4.6% larger than GPT-2 but still 17% smaller than xz . - Its engine is 1.5 times faster than GPT-2 30,319 against 20,472 tokens/s , because it has 6 layers instead of 12. - Thus a model fine-tuned on the logs of one company would be the strong case. It is frozen, because the alpha is about the same bits on every machine, not about ratio. Details: docs/core-architecture.md https://github.com/tac0mania/taco/blob/main/docs/core-architecture.md §8.6. - ts zip publishes no log results, so this table has no ts zip column. ---|---|---| | A model fine-tuned on logs distilgpt2, 6 layers, trained on 12 Loghub systems; "Route A" | On logs of the 12 trained systems: 53.2% smaller than GPT-2 profile logs . On 4 unseen systems: 4.6% larger. Engine 1.5 times faster 30,319 against 20,472 tokens/s, smaller model | Frozen: the alpha is about cross-machine bits, not ratio. Details: docs/core-architecture.md https://github.com/tac0mania/taco/blob/main/docs/core-architecture.md §8.6 | Read CLAUDE.md https://github.com/tac0mania/taco/blob/main/CLAUDE.md before you change code. It has the rules that keep Taco bit-exact. The most important rule: no floating-point arithmetic in the prediction path. | Document | Contents | |---|---| | docs/core-architecture.md https://github.com/tac0mania/taco/blob/main/docs/core-architecture.md | How Taco works, end to end, with the measurements | | docs/primer.md https://github.com/tac0mania/taco/blob/main/docs/primer.md | The background: floating point, compression, language models | | HANDOFF.md https://github.com/tac0mania/taco/blob/main/HANDOFF.md | The current state and the next steps | | docs/ARCHITECTURE.md https://github.com/tac0mania/taco/blob/main/docs/ARCHITECTURE.md | The code map: crates, tools and permitted dependencies | | docs/design/plan.md https://github.com/tac0mania/taco/blob/main/docs/design/plan.md | The design | | docs/adr/ https://github.com/tac0mania/taco/blob/main/docs/adr | The decisions | | docs/research/ https://github.com/tac0mania/taco/blob/main/docs/research | The measurements | | docs/security.md https://github.com/tac0mania/taco/blob/main/docs/security.md | The threat model and the fuzzing results | | docs/ci.md https://github.com/tac0mania/taco/blob/main/docs/ci.md | CI and the cross-machine conformance check | scripts/check.sh tags, no floats, fmt, clippy -D warnings, all tests scripts/xcheck the cross-machine check: Mac ARM and devbox x86 , 12 inputs A change to the coder, the model file, the tables or the container changes the bytes of .taco files. It needs a new format version, a reader for the old version, new conformance vectors and an ADR CLAUDE.md §1 and §8 . Apache-2.0. See LICENSE https://github.com/tac0mania/taco/blob/main/LICENSE . The GPT-2 weights are from OpenAI modified MIT licence and are not in this repository. tools/baseline downloads them.