{"slug": "i-built-a-local-keyless-firecrawl-for-claude-code-here-s-why-published-false", "title": "I built a local, keyless Firecrawl for Claude Code — here's why published: false", "summary": "A developer built tearsheet, an MIT-licensed MCP server that provides local, keyless web scraping for Claude Code. The tool aims to reduce token waste by returning minimal clean content and avoids SaaS middlemen by running entirely on the user's machine. It emphasizes trustworthy extraction with loud failure modes for omitted content.", "body_md": "A language model pays for every token of nav, ads, and footer it reads.\n\nThat one sentence is the whole reason **tearsheet** exists. I do a lot of research work inside Claude Code, and a huge chunk of it is \"go read this page and tell me what it says.\" Firecrawl-style tools are genuinely great at that — but they're SaaS. API keys, quotas, a bill, and your URLs leaving your machine to get there. I wanted the same toolset, running entirely on my own laptop, feeding my agent clean content without any of that.\n\nSo I built it. It's called tearsheet, it's MIT-licensed, and this is the story of why.\n\ntearsheet (n.): a page torn from a publication and filed as proof it ran.\n\nThat definition is on the repo for a reason. The whole tool is built around the idea that what it hands back should be trustworthy enough to file as evidence. More on that below — it's the part I care about most.\n\nTwo things pushed me over the edge.\n\n**Tokens are money.** When you scrape a page and dump the raw HTML — or even a naive \"readable\" conversion — into a model's context, you're paying for the cookie banner, the mega-menu, the newsletter modal, and the six-deep footer. On a research run that fans out across dozens of pages, that waste compounds fast. I wanted a tool whose *first principle* was \"return the least text that fully answers the question.\"\n\n**I didn't want a middleman.** No API key to manage, no service to trust with my browsing, no telemetry, no rate limit that isn't mine. Just a thing on my machine that Claude Code can call. Everything tearsheet does — fetch, extract, cache, crawl — happens locally.\n\ntearsheet is an MCP server exposing five tools. If you've used Firecrawl, these will feel familiar:\n\n`scrape`\n\n`search`\n\n`map`\n\n`crawl`\n\n`extract`\n\nThe intended flow is deliberately cheap: ** map → pick the URLs that matter → scrape those**, or\n\n`crawl`\n\n→ read the files it wroteThree design choices do the token-saving work:\n\n`~/.tearsheet/pages/`\n\nand prints the path. The model reads the file if it needs more, instead of you re-scraping with a bigger cap.`crawl`\n\nnever dumps page content into the tool output — it writes files and hands back a compact index (filename, ~token count, title, path). The consuming model decides what to actually read.Under the hood it's `fastmcp`\n\nfor the server, `trafilatura`\n\nfor extraction, `ddgs`\n\nfor keyless search, `extruct`\n\n+ `lxml`\n\nfor structured data, and a lazy-loaded `playwright`\n\nChromium fallback for the JavaScript-only pages that a plain fetch can't see.\n\nHere's the thing that makes tearsheet different from a weekend scraper.\n\nEvery extractor fails sometimes. The question is *how*. tearsheet's failure mode is **omission, never fabrication** — and every guard in it exists to make omission **loud** instead of silent. A tool that quietly drops half a pricing table is far more dangerous than one that says \"I couldn't read this cleanly,\" because you'll cite the clean-looking-but-wrong version without a second thought.\n\nSo tearsheet is built to be noisy about its own blind spots:\n\n`warning:`\n\nlines in the header.`scrape`\n\nresult starts with a header (`url`\n\n, `title`\n\n, token count, warnings) before the content. A `warning:`\n\nline means the extraction is known-unreliable — don't quote figures from it, re-run with `raw=true`\n\n.`raw=true`\n\nescape hatch.`raw`\n\nskips trafilatura entirely and hands back the visible page text so you can see for yourself.The design goal — and the track record so far — is zero fabricated content. It omits, and it tells you when it did.\n\nFair warning: **this is early.** It's not on PyPI yet — that's deliberate, I'm still refining it and I'd rather ship it clean than ship it fast. For now it's git-clone-only. If you don't mind a little rough, here's the whole setup.\n\nClone and install (requires **Python ≥ 3.12**):\n\n```\ngit clone https://github.com/Wynelson94/tearsheet.git\ncd tearsheet\npython3 -m venv .venv\n.venv/bin/pip install -e \".[dev]\"\n```\n\nOptional, for JavaScript-rendered pages:\n\n```\nplaywright install chromium\n```\n\n(Without it, tearsheet degrades gracefully and tells you when a page needed rendering.)\n\nThen register it with Claude Code as an MCP server:\n\n```\nclaude mcp add --scope user tearsheet -- ~/Projects/tearsheet/.venv/bin/tearsheet-mcp\n```\n\nThat's it. Restart Claude Code and you've got `scrape`\n\n, `search`\n\n, `map`\n\n, `crawl`\n\n, and `extract`\n\navailable as tools.\n\nI'd rather tell you the weak spots than let you find them.\n\nCommercial and tabular pages are the hard case — the whole price-guard system exists because that's where extraction is fragile. And some figures are invisible to *any* non-interactive fetch: prices rendered client-side in a JavaScript payload, literal `\"null\"`\n\nplaceholders, or numbers drawn inside an image. The rule I follow is procedural, not optimistic: **any figure I'm going to quote gets raw or independent verification.**\n\nTo keep myself honest about all this, there's a falsifiable eval harness in the repo (`evals/`\n\n) — a corpus weighted toward the commercial/tabular pages that are hardest, scored against an independent oracle, with a verdict that refuses to exist if the corpus isn't reachable. Neuter the guards and the verdict goes red; restore them and it goes green. The pages that once broke it are pinned as offline fixtures, so every future change has to answer to the original failures forever. There are 240+ tests, and CI runs fully offline on Python 3.12 and 3.13.\n\nThat's tearsheet: a local, keyless, no-telemetry Firecrawl alternative built to feed Claude Code clean content cheaply — and built to be loud about the moments it can't.\n\nIf you try it, I genuinely want the feedback — issues and bug reports especially. The whole point of the guards is that failures make the tool better, so if it breaks on a page, that's a finding, not a bug I'm embarrassed by.\n\nGo tear some pages out of the internet.", "url": "https://wpnews.pro/news/i-built-a-local-keyless-firecrawl-for-claude-code-here-s-why-published-false", "canonical_source": "https://dev.to/wynelson94/i-built-a-local-keyless-firecrawl-for-claude-code-heres-whypublished-false-55ho", "published_at": "2026-07-19 03:50:31+00:00", "updated_at": "2026-07-19 04:27:37.050735+00:00", "lang": "en", "topics": ["developer-tools", "large-language-models", "ai-tools", "ai-agents", "ai-infrastructure"], "entities": ["tearsheet", "Claude Code", "Firecrawl", "Wynelson94", "GitHub", "trafilatura", "playwright", "fastmcp"], "alternates": {"html": "https://wpnews.pro/news/i-built-a-local-keyless-firecrawl-for-claude-code-here-s-why-published-false", "markdown": "https://wpnews.pro/news/i-built-a-local-keyless-firecrawl-for-claude-code-here-s-why-published-false.md", "text": "https://wpnews.pro/news/i-built-a-local-keyless-firecrawl-for-claude-code-here-s-why-published-false.txt", "jsonld": "https://wpnews.pro/news/i-built-a-local-keyless-firecrawl-for-claude-code-here-s-why-published-false.jsonld"}}