# Give your LLM live web search: query top results clean page Markdown (no search API key)

> Source: <https://dev.to/get_anything/give-your-llm-live-web-search-query-top-results-clean-page-markdown-no-search-api-key-3apj>
> Published: 2026-10-30 06:39:29+00:00

LLMs are stuck at their training cutoff. The usual fix is to give them a **web search step**: run a query, take the top results, read the pages, and put the relevant text in the prompt. Most tutorials stop at "get a search API key", which means a paid account with Google, Bing or Brave and their rate limits.

Here's a way to do the whole loop (**search → fetch pages → clean Markdown**) in one call, with no search API key.

Query: `best open source vector database 2026`, top 5 results, with page content. This is the real output (Markdown trimmed):

```
[
  {
    "query": "best open source vector database 2026",
    "engine": "duckduckgo",
    "position": 1,
    "title": "9 Best Open Source Vector Databases in 2026 (Tested) - Geekflare",
    "url": "https://geekflare.com/open-source/vector-database/",
    "domain": "geekflare.com",
    "contentSuccess": true,
    "tokensApprox": 200,
    "markdown": "| Qdrant | Yes — sparse and dense vectors fused in one query | ..."
  },
  {
    "position": 5,
    "title": "7 Best Open-Source Vector Databases in 2026",
    "domain": "krunalkanojiya.com",
    "markdown": "Qdrant is the best balanced open-source vector database for most teams starting a dedicated retrieval service in 2026. It combines ..."
  }
]
```

Every result has its **rank, title, URL and domain**. With `fetchPageContent` on, it also has the **page body as Markdown** with boilerplate stripped, plus a token estimate. That's exactly what you want to paste into a prompt with citations.

``` python
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

def web_search(query, n=5, chars=3000):
    run = client.actor("get_anything/ai-web-search-scraper").call(run_input={
        "queries": [query],
        "engine": "duckduckgo",
        "maxResults": n,
        "fetchPageContent": True,
        "maxCharsPerPage": chars,      # keep the prompt within budget
    })
    return [r for r in client.dataset(run["defaultDatasetId"]).iterate_items() if r.get("contentSuccess")]

def build_context(results):
    return "\n\n".join(
        f"[{r['position']}] {r['title']} ({r['url']})\n{r['markdown']}" for r in results
    )

results = web_search("best open source vector database 2026")
context = build_context(results)

prompt = f"""Answer using only the sources below and cite them like [1].

{context}

Question: Which open-source vector database should a small team start with in 2026, and why?"""
# send `prompt` to the model you use (Claude, GPT, a local Llama...)
```

A few tips from using it:

`maxCharsPerPage`.` position` lets the model cite `[1]`–`[5]`, and you can show the links in your UI.` queries` takes a list, so you can fan out a question into 3–4 phrasings in one run and de-duplicate by URL.
If you use Claude, ChatGPT, Cursor or another MCP client, add the scraper as a tool through the Apify MCP server:

```
{ "mcpServers": { "apify": { "url": "https://mcp.apify.com/?tools=get_anything/ai-web-search-scraper" } } }
```

Now you can ask *"Search the web for this week's news on X and summarise the top 5 sources with links"*, and the assistant runs the search and reads the pages itself. Setup guide: [Connect any Apify scraper to Claude or ChatGPT in 2 minutes](https://dev.to/get_anything/connect-any-apify-scraper-to-claude-or-chatgpt-in-2-minutes-mcp-37he).

If you don't need the search step (say you have a list of docs pages or articles), use the sister tool that only converts URLs to clean Markdown and can crawl a whole site: [LLM Web Scraper](https://apify.com/get_anything/llm-web-scraper).

Pay per result, no subscription. See the Pricing tab on the Actor page. The Apify free plan's monthly credit is enough to try it properly.

**Try it:** [AI Web Search Scraper on Apify](https://apify.com/get_anything/ai-web-search-scraper)

What are you using for grounding today, and where does it break? I'm curious which sites give people the most trouble.