Fixed Deep research and web fetch in Claude Code : full open source

Adaptive web research for AI coding agents

91% of Deep Research quality · 5% of the cost · Works in every MCP client

A skill by GrapeRoot

Docs: 中文 · 日本語 · 한국어 · Español · हिन्दी · Français · Deutsch · Português · Русский

pip install webify-mcp
claude mcp add webify -- webify-mcp

That's it. Works in Claude Code, Cursor, VS Code, Windsurf, and Zed.

flowchart TB
    Agent[AI Agent] -->|"web_find('query')"| Webify
    Agent -->|"web_lookup(url, 'query')"| Webify
    Webify -->|"80–300 tokens"| Agent
    
    subgraph Webify[Webify MCP Server]
        Search[Search\nBrave / DDG] --> Graph[DOM Structural\nGraph Builder]
        Graph --> Retrieve[BM25 + BFS\nRetrieval]
        Retrieve --> Synthesize[Haiku\nSynthesis]
    end

    style Webify fill:#1a1a2e,stroke:#16213e,color:#fff
    style Agent fill:#0f3460,stroke:#16213e,color:#fff

Two tools for web research — both dramatically cheaper than reading full pages:

flowchart LR
    A[Query] --> B[Search\nBrave / DDG]
    B --> C1[Page 1]
    B --> C2[Page 2]
    B --> C3[Page 3–6]
    C1 --> D[DOM Graph\n+ BM25]
    C2 --> D
    C3 --> D
    D --> E[Multi-aspect\nextraction]
    E --> F[Haiku\nsynthesis]
    F --> G["Answer\n(~800 tokens)"]

Adapts depth to query complexity. Simple questions hit 3 sources. Multi-dimensional research scales to 6+ with independent sub-aspect retrieval. Call it multiple times with focused sub-queries for deep-research-level coverage.

flowchart LR
    A[URL + Query] --> B[Fetch page]
    B --> C[DOM structural\ngraph]
    C --> D[BM25 scoring]
    D --> E[BFS traversal]
    E --> F["Relevant subtree\n(80–300 tokens)"]

Scores nodes against your query, returns only the relevant subtree — 80–300 tokens instead of the 3,000–15,000 tokens of full page text WebFetch puts in context.

Blind A/B test against Claude's Deep Research — 15 unseen queries, randomized order, Sonnet judge scoring accuracy + completeness + specificity (1–5 each).

Webify finds correct information every time. The gap is always completeness — Deep Research reads more. For most queries that difference doesn't matter; for exhaustive research, call web_find

multiple times.

Per-query breakdown #

Once installed, the AI automatically uses Webify for web research instead of expensive built-in tools — no configuration needed. The preference policy is embedded in the package itself.

> What are the tradeoffs between Raft and Paxos consensus?
→ Claude calls web_find() — searches, builds graphs, synthesizes answer

> Look up rate limits in the GitHub API docs
→ Claude calls web_lookup() — fetches that page, returns relevant sections only
pip install webify-mcp
claude mcp add webify -- webify-mcp

Add to your MCP config:

{
  "mcpServers": {
    "webify": {
      "command": "webify-mcp"
    }
  }
}

Config file locations:

Cursor→~/.cursor/mcp.json

Windsurf→~/.windsurf/settings.json

VS Code / Continue→~/.continue/config.json

Zed→~/.config/zed/settings.json

Command:webify-mcp

Transport: stdio

pip install --upgrade webify-mcp

WEBIFY_CACHE_DIR

~/.cache/webify

Search: Brave API (if key set) → DuckDuckGo Lite (free fallback, no key needed)

macOS / Linux — add to ~/.zshrc

or ~/.bashrc

:

export ANTHROPIC_API_KEY="sk-ant-..."
export BRAVE_SEARCH_API_KEY="BSA..."

Windows (PowerShell):

[Environment]::SetEnvironmentVariable("ANTHROPIC_API_KEY", "sk-ant-...", "User")
[Environment]::SetEnvironmentVariable("BRAVE_SEARCH_API_KEY", "BSA...", "User")

In your MCP config (applies only to Webify):

{
  "mcpServers": {
    "webify": {
      "command": "webify-mcp",
      "env": {
        "ANTHROPIC_API_KEY": "sk-ant-...",
        "BRAVE_SEARCH_API_KEY": "BSA..."
      }
    }
  }
}

Get your keys:

• Anthropic → https://console.anthropic.com/settings/keys - Brave Search → https://brave.com/search/api/

python -m webify build https://docs.python.org/3/library/json.html

python -m webify lookup https://docs.python.org/3/library/json.html "parse JSON string"
python
import webify

result = webify.web_find("how does mTLS work in service meshes")
print(result["content"])   # synthesized answer
print(result["sources"])   # [{url, title, confidence, tokens}]

result = webify.smart_lookup("https://docs.python.org/3/library/json.html", "parse JSON")
print(result["content"])   # relevant sections only (~376 tokens)
webify-mcp                  # test server (Ctrl+C to exit)
ls ~/.cache/webify/         # check cache

→ Runwebify-mcp: command not found

pip install webify-mcp

Tool not showing up→ Restart your editor after adding to config→ Setweb_find

errorsANTHROPIC_API_KEY

→ DDG rate-limited; setweb_find

returns no resultsBRAVE_SEARCH_API_KEY