{"slug": "a-deleted-file-isn-t-gone-it-s-waiting-to-be-overwritten", "title": "A deleted file isn't gone. It's waiting to be overwritten", "summary": "A developer built Palimpsest, an open-source tool that recovers deleted files from ext4 disk images and then uses locally run open-weight models to explain what the recovered artifacts were. The pipeline pairs classical forensics (fls, istat, icat) with Gemma 3 vision triage, nomic-embed-text embeddings clustered via union-find, and a hypothesis stage that labels unrecoverable files as dashed 'tombstones' rather than dropping them, with every generated sentence tied to the evidence ID it came from. The sample case is reproducible from a synthetic 128 MB image built without root or mounting, and renders to a static, JavaScript-free site.", "body_md": "*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine](https://dev.to/challenges/hacktoberfest-week4-2026-10-26)*\n\nThe letter is 208 bytes, and it is fictional. But the situation isn't.\n\nI built a 128 MB ext4 disk image, wrote the letter into it, and deleted it at 07:34 on a Tuesday morning. Then I tried to recover it.\n\nThe filesystem still knew the letter existed. The directory entry survived. The inode survived. Even the deletion timestamp survived. `icat` can copy the bytes out. Any classical recovery tool — `photorec`, `scalpel`, `foremost` — will hand the letter back, perfectly, and then stop.\n\nBecause the next question is not a filesystem question.\n\nWhat else went with it? Which of these eight photographs was in the box? What was the file whose name survived but whose blocks did not? A recovery report that lists only what it recovered is lying by omission.\n\nThat gap is **Palimpsest**. It recovers deleted files from a disk image and then explains what they were, using open-weight models running entirely on your own machine. It looks at the photographs, groups the files by meaning, writes a labelled hypothesis for the ones that cannot be recovered, and composes an epitaph for each artifact in which **every sentence carries the id of the evidence it came from**.\n\nIt is not a chatbot for your files. It is a witness statement with a chain of custody.\n\n**[The sample case, rendered](https://flameeecyber.github.io/palimpsest/demo/)** — nine recovered files, two hollow tombstones, and the evidence table behind every sentence. Everything on that page was generated on this laptop from a synthetic disk image, and the four commands below reproduce it:\n\n./scripts/make_sample_image.sh          # builds a 128 MB ext4 image, no root, no mount\n\npalimpsest run sample/ghost.img --out out\n\npalimpsest ablate --out out\n\npalimpsest verify --out out\n\n`out/site/index.html` is a static site: no server, no JavaScript, no CDN, no webfont. Copy it to a USB stick and it still opens.\n\n*The rendered case. The two dashed stones at the bottom are files that provably existed and did not come back.*\n\nRecovery first, because the story depends on it. No root, no loop device, no mount — the image is built with `mkfs.ext4` and `debugfs`, so it's reproducible in CI and can't damage the machine it runs on:\n\nfls -r -d -p image      # every deleted entry, recursively\n\nistat image      # the ext4 \"Deleted:\" line — when the file died\n\nicat image       # the bytes, if the blocks are still free\n\nOne detail matters: `istat` gives the **deletion time**, not the modification time — the difference between *when this was last written* and *when someone got rid of it*. And when `icat` returns zero bytes, the artifact is kept as **missing**, with its name, size, parent directory and deletion time. It is not dropped from the report.\n\nFrom there, five model stages:\n\n**1. Triage.** Gemma 3 looks at the pixels of a recovered image and returns subject, setting, people count, visible text, and a human label. A carved file has no name and no EXIF, so the pixels are the only evidence. On this CPU, one cold vision call takes 200–700 seconds, so triage is capped at the most promising images — a scheduling decision, not a filter that hides failure.\n\n**2. Link.** Every artifact is rendered as a line of text, embedded locally with `nomic-embed-text`, and clustered with union-find over a cosine graph. No random seeds, no hidden parameters. The model names each group the way a person labels a box of belongings — `C1: Summer Family Photos`, `C2: Sunset Landscapes June`.\n\n**3. Infer.** For a file whose blocks are gone, the model gets only facts: name, size, deletion time, directory, neighbours. It returns a one-sentence hypothesis. The UI renders it as a dashed tombstone labelled `hypothesis`, never as a recovered file. `photorec` reports an unrecoverable file as a failure. Palimpsest reports the shape of the hole.\n\n**4. Narrate.** Two to four sentences per artifact, each citing evidence ids. Then the validator runs.\n\n**5. Query.** `palimpsest ask` embeds your question, retrieves the ten nearest artifacts, and answers with artifact citations — then checks the answer the same way the epitaphs are checked.\n\nThe first version of the validator checked that every cited evidence id existed. That is the obvious check, and it is not enough.\n\n`gemma3:1b` wrote:\n\n\"The image depicts a sunset over the sea.\" — citing `E1: IMG_2019_0006.png`\n\nEvery word is plausible. None of it came from the evidence. A citation-existence check published it.\n\nSo the second check computes a **support score**: the fraction of the sentence's content words that actually appear in the cited values, after removing function words and generic visual nouns. Below `0.2`, the sentence is struck. Years get a stricter rule: a year in the sentence must appear verbatim in a cited value.\n\nStruck sentences are never published. They appear on the detail page crossed out, with the reason — because the failure is evidence too.\n\n*The same letter, narrated by `gemma3:1b` instead of `gemma3:4b`. One sentence survives with its source at 100%; the other quoted the letter but cited the wrong row, so it is struck on the page with the reason. A forensic report that silently deletes its failures is not a forensic report.*\n\nThen the loop feeds the rejection back to the model, once, with the exact strike reason. The best attempt wins. That retry loop is the difference between a demo and something that holds up when the model is having a bad day.\n\nI want to be precise about what this is: **lexical overlap is a tripwire, not entailment.** A fluent sentence can reuse the right nouns and still be wrong. The defence is layered — the prompt forbids invented relations, the support score catches the crude cases, `verify` re-checks later, and the UI labels speculation differently. It is not a proof. It is a documented, reproducible filter.\n\nThe obvious next feature was a chat box. `palimpsest ask` embeds the question, takes the cosine top-k over the case, and asks for two to four sentences with artifact ids attached.\n\nThe first answer it produced was this:\n\nThe recovered photographs [2f3dfa31df], [59125a46a8], [78294c9b10] and [cadaa85ef6] were all created on June 20th, 2019 with an iPhone 8.\n\nEvery id is real. Every id is a photograph from the same cluster. One of them was created on June 20th; the others were created on the 15th, the 18th and the 23rd. Retrieval did its job, the citations were correct, and the sentence was false anyway — because **a citation is not a check.**\n\nSo the answer now goes through the same machine as an epitaph. `check_answer()` splits it into sentences, requires each sentence to cite an artifact that exists, and looks for every timestamp, year, date, and quoted passage in the facts of the files that sentence cites.\n\nThe rule that catches the sentence above is the quantifier. If a sentence says *all*, *every*, *both*, *each*, *none* or *neither*, the fact has to hold for **every** artifact it cites; otherwise one is enough. *\"The file [2f3dfa31df] was created on June 20th, 2019\"* passes. The same date stretched over four files is struck, and the strike is fed back once, exactly like a rejected epitaph.\n\nStruck sentences never become the answer. They are printed underneath it, crossed out with the reason. The tool shows you what it refused to say.\n\nThe unrecoverable files are always in the context, marked `unrecovered`, because they have no embedding to retrieve and they are the most interesting thing in the case:\n\n$ palimpsest ask \"Which files never came back, and what do you think they were?\" --out out\n\nThe file [0343d28351] was unrecovered and probably a photograph. The file\n\n[6e82e87c54] was also unrecovered and likely taken on a mobile device.\n\nstruck (never shown to the user):\n\n  x  \"Both files have an unknown creation date.\"\n\n     -> cites no artifact\n\nThe retry is what the check buys you. Same question, same model, second attempt:\n\n$ palimpsest ask \"Which recovered photographs were taken on the same summer\n\ntrip, and what does the letter have to do with them?\" --out out\n\nThe file [c3d3c9aebb] IMG_2021_0002.png was created on 2019-06-16T18:24:01.\n\nThe file [fb343c5bd6] IMG_2020_0007.png was created on 2019-06-21T18:29:01.\n\nThe file [2f3dfa31df] IMG_2019_0006.png was created on 2019-06-20T18:28:01.\n\nThe file [cb023cb531] IMG_2021_0008.png was created on 2019-06-22T18:30:01.\n\nThe letter [b81c5c8508] letter.txt mentions a box of photographs from a\n\nsummer trip to the coast.\n\nFour true dates instead of one true date repeated four times. Fifteen minutes on this CPU for the two attempts — the honest price of a second opinion when the first one is wrong.\n\nThe claim \"the AI is load-bearing\" is easy to make and hard to check, so it is a command:\n\n$ palimpsest ablate --out out\n\nartifacts with a human label             0             2\n\ngroups found by meaning                  0             2\n\nunrecoverable files explained            0             2\n\nartifacts with an epitaph                9             9\n\nsentences written                       26            18\n\nsentences with a source                 26            18\n\nwords about the content                  0            17\n\nThe baseline is not a strawman. `stub_narrate()` was written before the model path and emits only what a `stat` call can prove: *\"letter.txt is a document of 208 bytes.\"* It writes more sentences and says less. That is the whole argument in one table.\n\nA forensic report is only worth something if someone else can check it later. `palimpsest run` writes `manifest.json` with the source image hash, the SHA-256 of every recovered artifact, the model names, the SHA-256 of every prompt file, and the version of every tool that touched the evidence.\n\n`palimpsest verify` re-reads the case and checks seven things: the artifact set, every file hash, every citation target, every published sentence's support score, prompt drift, code drift, and the source image hash. Exit code 1 on any failure, so it drops straight into CI.\n\nThis is the concrete form of the reproducibility argument. A report produced by a hosted model that was silently updated last Tuesday cannot be reproduced. A report produced by a pinned digest can.\n\n`palimpsest verify` can tell you whether the case still stands. You cannot do that with a closed model's system prompt, and you cannot do it at all if the model behind an API changes without a version number.\n`nomic-embed-text` via Ollama · no network required.", "url": "https://wpnews.pro/news/a-deleted-file-isn-t-gone-it-s-waiting-to-be-overwritten", "canonical_source": "https://dev.to/merouane_flame_56f/a-deleted-file-isnt-gone-its-waiting-to-be-overwritten-22dg", "published_at": "2026-10-27 18:09:11+00:00", "updated_at": "2026-10-27 18:12:18.911392+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "large-language-models", "ai-agents", "developer-tools"], "entities": ["Palimpsest", "Gemma 3", "nomic-embed-text", "ext4", "photorec", "Hacktoberfest"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/a-deleted-file-isn-t-gone-it-s-waiting-to-be-overwritten", "markdown": "https://wpnews.pro/news/a-deleted-file-isn-t-gone-it-s-waiting-to-be-overwritten.md", "text": "https://wpnews.pro/news/a-deleted-file-isn-t-gone-it-s-waiting-to-be-overwritten.txt", "jsonld": "https://wpnews.pro/news/a-deleted-file-isn-t-gone-it-s-waiting-to-be-overwritten.jsonld"}}