{"slug": "deduplicating-git-worktrees-on-ext4-with-jdupes", "title": "Deduplicating git worktrees on EXT4 with jdupes", "summary": "A developer found that ten git worktrees consumed 9.9 GB of disk on an EXT4 filesystem, mostly from duplicate node_modules and vendor directories. Using jdupes to hardlink byte-identical files reduced the footprint by roughly half, and the approach is safe because npm and Composer write new files on update, breaking the link only for changed files.", "body_md": "A git worktree workflow eats a lot more disk than you’d expect. On the [remote box I run my coding agents on](/remote-coding-environment-vps/)\n, ten worktrees took a 28 GB volume down to 687 MB free in about a day. Here’s one way to limit that.\n\n## Ten worktrees, 9.9 GB[#](#ten-worktrees-99-gb)\n\nEvery agent gets its own worktree, which is the whole point: two agents editing the same working directory trip over each other instantly. What I hadn’t thought through is what a worktree *costs*.\n\n``` bash\n$ du -xh -d2 ~/.t3/worktrees | sort -rh | head -4\n9.7G\t/home/dev/.t3/worktrees\n5.9G\t/home/dev/.t3/worktrees/ohdear.app\n3.7G\t/home/dev/.t3/worktrees/ma.ttias.be\n1.9G\t/home/dev/.t3/worktrees/ma.ttias.be/t3code-c207f00a\n```\n\nThe checkout itself is cheap. Git shares one `.git`\n\nacross all worktrees, so the source files are a rounding error. The expensive part is what my setup script does *after* the checkout: `composer install`\n\nand `npm ci`\n\n.\n\nOn the Oh Dear app that’s about 530 MB of `node_modules`\n\nand 350 MB of `vendor`\n\n. Call it 880 MB per worktree. Ten worktrees, nine gigabytes, and almost all of it byte-for-byte identical because most of those branches share a lockfile.\n\n## No reflinks on ext4[#](#no-reflinks-on-ext4)\n\nThe right answer to “I need ten near-identical copies of a directory” is a reflink. On a copy-on-write filesystem, `cp --reflink`\n\nmakes a clone that shares the underlying blocks with the original: nothing gets copied up front, and when one side writes, only the changed blocks get their own copy. That’s the exact shape of this problem.\n\nSo I tried it.\n\n``` bash\n$ findmnt -no FSTYPE,SOURCE /\next4 /dev/vda1\n\n$ cp --reflink=always a.bin b.bin\ncp: failed to clone 'b.bin' from 'a.bin': Operation not supported\n```\n\nBoth files are on the same filesystem. There’s no flag to fix this. **ext4 has no copy-on-write**, so it has no reflinks, and a standard Ubuntu cloud image gives you ext4 on the root volume. Reflinks need btrfs, XFS formatted with `reflink=1`\n\n, or ZFS.\n\nI could reformat the root filesystem of a live box for a dev-convenience win. I’m not going to.\n\n## Hardlinks, the ext4-shaped answer[#](#hardlinks-the-ext4-shaped-answer)\n\nWhat ext4 *does* have is hardlinks. Two names pointing at one inode, one copy of the data on disk. [jdupes](https://codeberg.org/jbruchon/jdupes)\nfinds byte-identical files and replaces the duplicates with hardlinks.\n\nThe whole idea, with two 8 MB files:\n\n``` bash\n$ ls -li a.bin b.bin\n761293 -rw-r--r-- 1 dev dev 8388608 Jul 24 08:08 a.bin\n761296 -rw-r--r-- 1 dev dev 8388608 Jul 24 08:08 b.bin\n$ du -sh .\n17M\t.\n\n$ jdupes -L -r .\n\n$ ls -li a.bin b.bin\n761293 -rw-r--r-- 2 dev dev 8388608 Jul 24 08:08 a.bin\n761293 -rw-r--r-- 2 dev dev 8388608 Jul 24 08:08 b.bin\n$ du -sh .\n8.1M\t.\n```\n\nSame inode, link count 2, half the disk. `-L`\n\ndoes the linking, `-r`\n\nrecurses. jdupes compares candidates byte for byte rather than trusting a hash, so it only links on real identity.\n\nThis isn’t copy-on-write. Hardlinked files don’t diverge on write: write into one and you’ve written into all of them. That would be a disaster for source files.\n\nIt’s fine for dependencies because of *how* package managers write. npm and Composer don’t edit files in place, they write a new file and rename it over the old one. A rename gives you a fresh inode, which breaks the link for that one file and leaves every other worktree’s copy untouched. This is the same trick pnpm uses on purpose with its global store; I’m just applying it after the fact.\n\n## Point it at node_modules and vendor[#](#point-it-at-node_modules-and-vendor)\n\nFor a Laravel app those two directories *are* the disk usage, so there’s nothing clever to do. Name the paths and let jdupes walk them:\n\n``` bash\n$ jdupes -r -L ~/projects/*/node_modules ~/projects/*/vendor \\\n             ~/.t3/worktrees/*/*/node_modules ~/.t3/worktrees/*/*/vendor\n```\n\nAcross ten worktrees that takes 25 seconds on this box. I run it nightly and again at the end of worktree setup, right after `npm ci`\n\nfinishes, because the duplicate exists from the moment the install does and waiting until 04:20 wastes it for a day.\n\nName the paths, don’t match on directory name anywhere in the tree. Laravel also ships `public/vendor`\n\nand `resources/views/vendor`\n\n, and both of those are **committed to the repo**. Hardlinking a tracked file across worktrees means editing a Blade view on one branch silently edits it on every other. A top-level `node_modules`\n\nand `vendor`\n\nare gitignored build artifacts, which is what makes this safe.\n\n## The disk space saved[#](#the-disk-space-saved)\n\nThe first sweep reclaimed **2833 MB**. Combined with deleting the worktrees whose branches were already merged, the disk went from 98% to 57%.\n\nIf you’re provisioning a box like this from scratch, put the worktrees on a filesystem that can do reflinks and skip all of this. Real copy-on-write beats dedup-after-the-fact, because it never writes the duplicate blocks at all. I’m keeping ext4 on this box and running jdupes instead, because reformatting a root filesystem is a much bigger job than a nightly cron.", "url": "https://wpnews.pro/news/deduplicating-git-worktrees-on-ext4-with-jdupes", "canonical_source": "https://ma.ttias.be/deduplicating-git-worktrees-ext4-jdupes/", "published_at": "2026-07-24 00:00:00+00:00", "updated_at": "2026-07-24 10:33:17.559605+00:00", "lang": "en", "topics": ["developer-tools"], "entities": ["jdupes", "EXT4", "npm", "Composer", "Oh Dear"], "alternates": {"html": "https://wpnews.pro/news/deduplicating-git-worktrees-on-ext4-with-jdupes", "markdown": "https://wpnews.pro/news/deduplicating-git-worktrees-on-ext4-with-jdupes.md", "text": "https://wpnews.pro/news/deduplicating-git-worktrees-on-ext4-with-jdupes.txt", "jsonld": "https://wpnews.pro/news/deduplicating-git-worktrees-on-ext4-with-jdupes.jsonld"}}