{"slug": "i-built-a-readability-test-for-my-own-compression-format-it-scored-0-24", "title": "I built a readability test for my own compression format. It scored 0/24.", "summary": "A developer built a readability test for their compression format ctxfold, which scored 0/24 for folded CSV against GPT-4o-mini, revealing that models struggle with reconstruction rules. The CSV encoder factored out shared prefixes, making the format unreadable for LLMs, leading to a fix where CSV folding is now pipeline-mode only.", "body_md": "*A negative result, its root cause, and the feature it produced.*\n\nMy last post introduced [ctxfold](https://github.com/antrixy/ctxfold) — lossless, structure-aware compression for the bulky stuff we put in LLM prompts. Its benchmark table had one cell I wasn't proud of:\n\n```\n| CSV / TSV | char reduction | ~30–45%* |\n\n*CSV readability not yet validated against a model\n```\n\nJSON and logs had real numbers: a model answering lookup questions off the folded form, scored against exact ground truth, matching raw field-for-field. CSV had a character count and an asterisk.\n\nSo I wrote the same harness for CSV. Generate 400 records with realistic redundancy, fold them, ask GPT-4o-mini to look up specific records in both forms, score against ground truth.\n\nRaw CSV: **24/24**. Folded CSV: **0/24**.\n\nNot \"slightly worse.\" Zero.\n\nFirst hypothesis: capability threshold. Re-ran on GPT-4o.\n\nRaw: **24/24**. Folded: **6/24**. A later run: 9/24. So no — a stronger model helps a little and inconsistently. The format is the problem.\n\nThe failure had structure, which is what made it diagnosable. Asked about record `NW-1258`\n\n, the model reported a warehouse of `WH-2`\n\n(a half-applied prefix), a supplier of `DAL-2`\n\n(a value from the warehouse column), and a price from some other row entirely. In another run it answered qty `1238`\n\nfor sku `NW-1238`\n\n— it read the record's own identifier back as data. Two distinct failures, every time: it couldn't find the right row, and it couldn't reconstruct the values in it.\n\nctxfold's JSON and logs encoders fold **syntax**. A JSON array of objects repeats every key on every record; the encoder lifts keys, braces, and quotes into a one-time header, and every *value* stays verbatim in its row. Logs are the same story with templates: timestamps, levels, `reqId=`\n\nprefixes get lifted; the payload stays put. The model reads a plain labeled table containing exactly the values it needs.\n\nCSV has no syntax to remove. No keys, no braces — it's already a compact table. The only redundancy left is **inside the values**: shared prefixes like `NW-`\n\non every sku, `WH-`\n\non every warehouse, a constant `USD`\n\ncolumn. So the CSV encoder factors those out, and each row keeps only the varying middle. Lossless, byte-exact, verified on every encode.\n\nAnd unreadable. To answer a question, the model has to find a row by a *partial* key (the sku column only contains `258`\n\n, because `NW-1`\n\nwas factored out — all 400 skus shared it) and then reconstruct every value through the header's `prefix + middle`\n\nrules. It can't do either reliably. On GPT-4o-mini it essentially can't do it at all.\n\nI'd actually seen a milder version of this before. ctxfold's opt-in dictionary coding (low-cardinality values become small integers, mapped once in the header) pushes JSON savings from ~39% to ~46% — and in testing, models resolved the coded values slightly less reliably than plain ones. That's why it ships off by default. The CSV result is the same phenomenon at full strength:\n\n**Models read values, not reconstruction rules. Indirection through a header costs readability — and in CSV's case, the savings were the indirection.**\n\nI considered \"fixing\" the format — don't factor unique columns, keep identifiers whole. But that only patches row-finding; the prefix-reconstruction failure remains, and the savings shrink toward nothing. CSV is already near its readable minimum. That's *why* there was nothing safe to fold.\n\nSo the fix was documentation, not code. As of v0.1.4, CSV folding is **pipeline-mode**: fold it for lossless transit or storage, call `decompress()`\n\nbefore the prompt is built, and the model never sees the folded form. For direct model reading, send CSV raw. The benchmark table now says so, with the measured numbers where the asterisk used to be. JSON and logs remain validated direct-readable — their folded output keeps every value intact, and the scores show it.\n\nThe rule I keep coming back to: ctxfold's core contract is *lossless or no-op — never lossy*. It turns out the same discipline applies to claims. Either a readability claim has a measurement behind it, or it gets the asterisk. And when the measurement comes in against you, the asterisk becomes documentation.\n\nSitting with those numbers, the useful question turned out to be: *before* anyone folds anything — where do a prompt's tokens actually go, and what's safely foldable?\n\nv0.2.0 ships that as `ctxfold --profile`\n\n:\n\n``` bash\n$ ctxfold --profile users.json\n\n[ctxfold profile]\nformat      JSON array — 300 records × 7 fields\nsize        65,614 chars ≈ 16,404 tokens (estimated; pass a tokenizer for exact)\n\nwhere the characters go\n  keys         27%   repeated field names (with quotes)\n  syntax        7%   braces, brackets, commas, colons\n  values       36%   the data itself (with string quotes)\n  whitespace   30%   indentation and spacing\n\nfoldable (lossless, verified by round-trip)\n  fold             -66%   direct-readable — validated 24/24 vs raw\n  + --dictionary   -73%   readability tradeoff — off by default, see README\n\nverdict: fold it — 16,404 → ~5,595 tokens (~66% fewer)\n```\n\nThat's a real API response shape — wrapped array, pretty-printed. 64% of the file is structure and formatting, which is why the fold is fat. On a CSV file, the same command reports the fold as **pipeline-only** and tells you to send it raw or decompress first — the negative result is baked into the tool's own advice.\n\nThe profiler follows the rules the failure taught:\n\n`compress()`\n\ndelivers — that's an invariant with a test on it, not a policy.Try it with zero setup (no API key, deterministic output):\n\n```\ngit clone https://github.com/antrixy/ctxfold && cd ctxfold\nnode examples/profile-demo.js\n```\n\nOr on your own data:\n\n```\nnpm install -g ctxfold\nctxfold --profile your-file.json\n```\n\nIf you maintain a tool that makes claims — savings, accuracy, compatibility — the cheapest test you'll ever write is the one that checks your own table. Mine took an afternoon, cost a few cents of API calls, and found that a third of my format list didn't do what a reader would assume. The fix cost nothing but honesty, and the tool that fell out of it is the most useful thing in the package.\n\nThe full harness is in the repo (`examples/gpt-csv-equivalence.js`\n\n), along with the [v0.1.4](https://github.com/antrixy/ctxfold/releases/tag/v0.1.4) and [v0.2.0](https://github.com/antrixy/ctxfold/releases/tag/v0.2.0) release notes. If you push structured data into prompts and your payloads break my assumptions, I want to hear about it.\n\n**Repo & docs:** [https://github.com/antrixy/ctxfold](https://github.com/antrixy/ctxfold) · **npm:** `npm install ctxfold`\n\n· MIT licensed.", "url": "https://wpnews.pro/news/i-built-a-readability-test-for-my-own-compression-format-it-scored-0-24", "canonical_source": "https://dev.to/maverickyadav/i-built-a-readability-test-for-my-own-compression-format-it-scored-024-4epc", "published_at": "2026-07-18 14:34:09+00:00", "updated_at": "2026-07-18 14:58:32.182135+00:00", "lang": "en", "topics": ["developer-tools", "large-language-models", "ai-research"], "entities": ["ctxfold", "GPT-4o-mini", "GPT-4o"], "alternates": {"html": "https://wpnews.pro/news/i-built-a-readability-test-for-my-own-compression-format-it-scored-0-24", "markdown": "https://wpnews.pro/news/i-built-a-readability-test-for-my-own-compression-format-it-scored-0-24.md", "text": "https://wpnews.pro/news/i-built-a-readability-test-for-my-own-compression-format-it-scored-0-24.txt", "jsonld": "https://wpnews.pro/news/i-built-a-readability-test-for-my-own-compression-format-it-scored-0-24.jsonld"}}