Large language models know a lot, but they don't know what changed on a website this morning. If you use Claude Code, Claude Desktop, Cursor or another MCP client, you can close that gap by connecting a web data server through the Model Context Protocol (MCP). This post explains what that gives you, how to set it up in a few minutes, and how to use it without wasting tokens.
What MCP adds
MCP is an open protocol that lets an AI client call external tools. A web scraping MCP server exposes tools such as "scrape this URL" or "search the web" that the assistant can call on its own when a task needs fresh information: reading the latest docs for a library, checking a changelog, comparing pricing pages, or collecting examples from a site.
Which tools you get
As an example, the XCrawl MCP server (npm package xcrawl-mcp, for the XCrawl web data API) exposes these tools:
Option 1: the hosted endpoint
The simplest setup uses the hosted MCP URL, so nothing runs on your machine. In Claude Desktop's claude_desktop_config.json:
{
"mcpServers": {
"xcrawl": {
"url": "https://mcp.xcrawl.com/YOUR_API_KEY/mcp"
}
}
}
In Claude Code you can add the same endpoint from the terminal:
claude mcp add --transport http xcrawl https://mcp.xcrawl.com/YOUR_API_KEY/mcp
The server also accepts the key in an Authorization: Bearer header if you'd rather keep it out of the URL.
Option 2: run it locally over stdio
If you prefer a local process (Node.js 18+):
XCRAWL_API_KEY=YOUR_API_KEY npx -y xcrawl-mcp
or register it with Claude Code:
claude mcp add xcrawl -e XCRAWL_API_KEY=YOUR_API_KEY -- npx -y xcrawl-mcp
Prompts that work well
Keeping it efficient
Wrapping up
Connecting a web data MCP server turns your assistant from "trained up to some date" into something that can check the live web when it matters. Setup is a single config entry, and new XCrawl accounts start with 1,000 free credits, which is plenty to try the workflow on your own projects. Docs are at https://docs.xcrawl.com/.