# How to Give ChatGPT Web Scraping with MCP Connectors (2026)

> Source: <https://dev.to/simon_crawlforge_dev/how-to-give-chatgpt-web-scraping-with-mcp-connectors-2026-4ba6>
> Published: 2026-06-16 20:20:51+00:00

ChatGPT can now call your own tools through custom MCP connectors — including web scraping. But there is a catch the marketing pages skip: connectors must be **remote** servers, so a local tool like CrawlForge cannot be pasted in directly. This is the honest version: what is actually possible, why a wrapper is needed, and the exact bridge to build.

ChatGPT supports **custom MCP connectors** — renamed **"apps"** in December 2025, so the UI now says "Apps & Connectors." Through **Developer mode**, you connect an external MCP server and ChatGPT calls its tools mid-conversation, asking you to confirm before any write action. Same Model Context Protocol that powers web scraping in Claude — different client. Developer mode is explicitly a **beta**.

Per OpenAI's plan table, adding a custom MCP connector is available on **Plus, Pro, Business, Enterprise, and Edu** — not Free or Go. Full write-action support is rolling out most broadly to Business, Enterprise, and Edu. If you only need ChatGPT to *read* scraped data, the read-only path below is enough.

This trips people up. **A ChatGPT connector must be a remote MCP server reachable over HTTPS** (SSE or Streamable HTTP transport). You paste a URL; you do not point it at a command on your machine. That rules out **local stdio servers** — the kind you install with `npx`

. To use one, host it publicly or tunnel a local server via ngrok or Cloudflare Tunnel.

There is also a naming rule: ChatGPT's **deep research / company-knowledge** paths require two read-only tools named `search`

and `fetch`

with a specific schema. Full **Developer mode** allows arbitrary tools, so that constraint applies only to the deep-research path.

CrawlForge ships as a **local stdio MCP server** (via `npx`

) plus a **REST API** at `https://www.crawlforge.dev/api/v1/tools/`

. Neither is a remote MCP URL, and its tools are named `search_web`

, `fetch_url`

, and `extract_content`

— not the `search`

/`fetch`

pair deep research expects. So you cannot paste CrawlForge straight into ChatGPT today. The practical path is a **thin remote MCP wrapper** — about 30 lines.

[FastMCP](https://gofastmcp.com/integrations/chatgpt) (Python) is the quickest way to stand up a remote MCP server exposing the `search`

+ `fetch`

tools ChatGPT wants. Each calls CrawlForge's REST API with your `cf_live_`

key in the `X-API-Key`

header:

``` python
import os
import httpx
from fastmcp import FastMCP

mcp = FastMCP("CrawlForge Bridge")
BASE = "https://www.crawlforge.dev/api/v1/tools"
HEADERS = {"X-API-Key": os.environ["CRAWLFORGE_API_KEY"]}

@mcp.tool()
async def search(query: str) -> list[dict]:
    """Search the web. Returns id/title/url results for ChatGPT."""
    async with httpx.AsyncClient(timeout=30) as client:
        r = await client.post(f"{BASE}/search_web", headers=HEADERS,
                              json={"query": query, "limit": 10})
    results = r.json().get("results", [])
    return [{"id": x["link"], "title": x["title"], "url": x["link"]} for x in results]

@mcp.tool()
async def fetch(id: str) -> dict:
    """Fetch full page content by id (the URL) for ChatGPT."""
    async with httpx.AsyncClient(timeout=30) as client:
        r = await client.post(f"{BASE}/extract_content", headers=HEADERS,
                              json={"url": id})
    data = r.json()
    return {"id": id, "title": data.get("title", id),
            "text": data.get("content", ""), "url": id}

if __name__ == "__main__":
    mcp.run(transport="http", host="0.0.0.0", port=8000)
```

Run it and expose it over HTTPS. For a quick test, tunnel your local port:

```
pip install fastmcp httpx
export CRAWLFORGE_API_KEY="cf_live_your_key_here"
python server.py
# in another terminal:
ngrok http 8000
```

For Developer mode you can skip the `search`

/`fetch`

naming and map tools one-to-one to CrawlForge — expose `scrape_structured`

, `stealth_mode`

, or `deep_research`

directly. Same pattern.

`/mcp`

), name it, choose an auth method.Your `search`

and `fetch`

tools appear. In a chat, select the connector and ask ChatGPT to research a topic — it calls `search`

, then `fetch`

es the best results through CrawlForge.

Connectors authenticate with **none** (public) or **OAuth** — there is no API-key-header option in the UI, which is why the wrapper holds your CrawlForge key server-side. ChatGPT confirms before write actions, and you can inspect each call before approving.

Take OpenAI's warnings seriously: **only connect servers you trust.** Custom connectors increase risk, including prompt injection, and a model mistake on a write action could destroy or leak data. A read-only scraping bridge is low-risk; lock it down with OAuth before sharing.

If you would rather not host anything, use CrawlForge from code with the OpenAI Agents SDK or Responses API — no remote server required. See [CrawlForge with the OpenAI Agents SDK](https://www.crawlforge.dev/blog/crawlforge-openai-agents-integration).