{"slug": "how-to-fit-more-web-search-data-into-a-local-llm", "title": "How to Fit More Web Search Data into a Local LLM", "summary": "SerpApi's new Markdown output lets a local LLM fit more web search data into its context window, because the same results take fewer tokens in Markdown than in JSON, according to the company's benchmark of seven searches. The benchmark compared four outputs — raw JSON, JSON with the JSON Restrictor, Markdown with the restrictor, and raw Markdown — using OpenAI's tiktoken cl100k_base encoding, and found that for most searches the full response now fits with no restrictor at all. The accompanying open-source app, built with TanStack Start 1.0 and running models in LM Studio with a 32K context window, defaults to Markdown with the restrictor.", "body_md": "A local model can only use the web search results that fit in its [__context window__](https://www.ibm.com/think/topics/context-window). When we built the app for [__How to Connect Your Local LLM with Web Search Data__](https://serpapi.com/blog/how-to-connect-your-local-llm-with-web-search-data/), a single SerpApi response was often bigger than the whole context window available. So we trimmed every response with the [__JSON Restrictor__](https://serpapi.com/json-restrictor), which keeps only the fields the model needs and drops the rest on SerpApi’s servers before it reaches the LLM.\n\nSerpApi's new [__Markdown output__](https://serpapi.com/markdown-output) changes that budget. The same data takes fewer tokens in Markdown than in JSON, so the restrictor can keep more of it. For most searches, the full response now fits with no restrictor at all.\n\nThis blog will show you how to apply the JSON Restrictor to Markdown output, and when the full response is worth the wait. We'll use the same [__open-source app__](https://github.com/serpapi/local-llm-web-search) that gives a local model running in [__LM Studio__](https://lmstudio.ai) internet access. The app is built with [TanStack Start](https://tanstack.com/start), and we upgraded it to Start 1.0 along the way.\n\n## What’s New Since the First Blog\n\nSerpApi's Markdown output returns curated results as a Markdown document instead of JSON. You add `output=md` to a request, and the results come back as headings, lists and tables without the JSON keys, quotes and braces. It works across SerpApi's [__100+ APIs__](https://serpapi.com/search-engine-apis), and a Markdown search costs the same as a JSON search. The launch blog [__Introducing SerpApi’s New Markdown Output__](https://serpapi.com/blog/introducing-serpapi-markdown-output/) covers the format and its token savings across our engines.\n\nThe app now lets you choose what the model reads after each search. The benchmark compares four outputs, and a model only ever reads three of them.\n\n| Output | What the model reads | Parameters | \n|---|---|---|\n| JSON (Raw) | The full JSON response. The benchmark measures it, but no model reads it | No restrictor | \n| JSON (Restrictor) | JSON trimmed to the fields the model needs, as in the first blog | `json_restrictor` | \n| Markdown (Restrictor) | Markdown with more results and more fields, at about the same size. This is the app's default | `json_restrictor` and`output=md` | \n| Markdown (Raw) | The full Markdown document | `output=md` | \n\nThe setup changed too. The first blog ran Qwen3.5-9B on a MacBook Pro M4 Max with 36 GB of memory. This time I ran four of LM Studio's staff picks on a MacBook Pro M5 Pro with 48 GB, using [__LM Link__](https://lmstudio.ai/link) to connect both MacBooks. They are:\n\n## How We Tested\n\nEvery model ran with a 32K context window, the same size as in the first blog. LM Studio loads a model with an 8K window by default, which is too small once a chat carries search results. A 32K window is enough for a long conversation, and it fits a 27B model in memory on a 36 GB Mac.\n\n### What the Benchmark Measures\n\nThe app's Benchmark page measures two things, and this blog uses both.\n\n**Payload size:** The same seven searches run once for each of the four outputs, with no model involved. There's one search per tool, covering [__Search__](https://serpapi.com/search-api), [__Finance__](https://serpapi.com/google-finance-api), [__News__](https://serpapi.com/google-news-api), [__Maps__](https://serpapi.com/google-maps-api), [__Flights__](https://serpapi.com/google-flights-api), [__Hotels__](https://serpapi.com/google-hotels-api) and [__Shopping__](https://serpapi.com/google-shopping-api). This shows what each output costs before a model reads anything. These sizes come from OpenAI's [__tiktoken__](https://github.com/openai/tiktoken) library with its `cl100k_base` encoding, which splits text into tokens. Each local model splits text its own way, so treat these sizes as estimates for comparing formats.\n\n**End to end:** Thirteen questions run through the whole app on each model, once for each of the three outputs a model can read. Seven need live data from SerpApi, one search per tool. Two need two searches at once, such as comparing AAPL with TSLA. The other four need no search, such as \"*What is 15% of 240?*\", so they show how the model behaves on its own. We used it to compare how each model handles each output, how long it takes, and what it answers.\n\n## How Much Does Markdown Save?\n\nTo see what the format alone saves, we kept the restrictor strings from the first blog and sent the same seven searches as JSON and as Markdown. The seven results took 4,668 tokens in JSON and 3,412 in Markdown, 27% fewer. Here's how each tool changed:\n\n| Tool | JSON | Markdown | Change | \n|---|---|---|---|\n| Flights | 823 | 439 | −47% | \n| Finance | 521 | 324 | −38% | \n| News | 678 | 422 | −38% | \n| Search | 785 | 586 | −25% | \n| Hotels | 567 | 433 | −24% | \n| Maps | 497 | 449 | −10% | \n| Shopping | 797 | 759 | −5% | \n| **Total** | **4,668** | **3,412** | **−27%** | \n\nThe saving depends on how deeply the data is nested. A flight option holds legs, and each leg holds airports and times, so JSON repeats keys, quotes and braces at every level. Markdown turns lists of results into tables and writes each column name once. Shopping results are flat, with a title, a price and a store per product, so there is little to remove.\n\nSo instead of keeping that saving, we spend it. In Markdown, the same token budget can carry more fields and more results, so the Markdown output gets a looser restrictor.\n\n## Apply the JSON Restrictor to Markdown Output\n\nThere's no separate restrictor for Markdown. You send the same `json_restrictor` parameter together with `output=md`, and SerpApi filters the sections first, then renders what's left as Markdown.\n\nIn the app, each tool has two restrictor strings. JSON sends the original one, and Markdown sends a looser one with more results and more fields. The Markdown request is the same search with `output=md` added:\n\n```\n// Pick the restrictor for the output mode. Markdown (Raw) sends none.\nfunction restrictorFor(name: RestrictorTool, output: ToolOutput): string | null {\n  if (output === \"markdown-full\") return null\n  return output === \"markdown\" ? MARKDOWN_RESTRICTORS[name] : RESTRICTORS[name]\n}\n\n// Same parameters as the JSON request, plus output=md.\nasync function serpApiMarkdown(params: Record<string, unknown>): Promise<string> {\n  const url = new URL(\"https://serpapi.com/search\")\n  for (const [key, value] of Object.entries(params)) {\n    if (value !== undefined && value !== null) url.searchParams.set(key, String(value))\n  }\n  url.searchParams.set(\"output\", \"md\")\n  url.searchParams.set(\"api_key\", process.env.SERPAPI_API_KEY ?? \"\")\n\n  const res = await fetch(url)\n  return res.text()\n}\n```\n\nEach tool spends the saving on different extras:\n\n| Tool | JSON (Restrictor) keeps | Markdown (Restrictor) keeps | \n|---|---|---|\n| Search | Answer box, [AI Overview](https://serpapi.com/google-ai-overview-api) (6 blocks), 3 related questions, knowledge graph, 5 web results | 5 web results, answer box with its lists, AI Overview (8 blocks), knowledge graph, 3 top stories, 3 related questions | \n| Finance | Price, key stats, 3 news items | Price, key stats, company description, 5 news items | \n| News | 5 stories, 1 related article each | 7 stories, 2 related articles each | \n| Maps | 5 places | 6 places, with opening hours and a description | \n| Flights | 3 best and 3 other options | 4 best and 4 other options, with aircraft, legroom, overnight layovers and price insights | \n| Hotels | 5 hotels | 7 hotels, with a description and a location rating | \n| Shopping | 5 products, plus 3 inline results | 7 products, plus 3 inline results, with second-hand condition | \n\nFor example, here are both restrictor strings for Maps, from `restrictors.ts`:\n\n``` js\n// JSON (Restrictor): five places.\nexport const RESTRICTORS = {\n  google_maps_search: [\n    \"local_results[0:5].{title,rating,reviews,type,address,price,open_state,phone,website,gps_coordinates}\",\n    \"place_results.{title,rating,reviews,type,address,price,open_state,phone,website}\",\n    \"error\",\n    \"search_metadata.google_maps_url\",\n  ].join(\",\"),\n  // ...one restrictor per tool\n}\n\n// Markdown (Restrictor): six places, with opening hours and a description.\nexport const MARKDOWN_RESTRICTORS = {\n  google_maps_search: [\n    \"local_results[0:6].{title,rating,reviews,type,address,price,open_state,hours,phone,website,description}\",\n    \"place_results.{title,rating,reviews,type,address,price,open_state,hours,phone,website,description}\",\n    \"error\",\n    \"search_metadata.google_maps_url\",\n  ].join(\",\"),\n  // ...one restrictor per tool\n}\n```\n\nThe extra data shows up in the answers. Asked for the best barbecue in Austin, Qwen 3.8 27B listed five places with JSON and six with Markdown. The new one, Good BBQ Company, was one of the two highest rated. The Markdown answer also described Terry Black's as \"4th-generation pitmasters, old-school smoked meats,\" from the place descriptions only the Markdown restrictor keeps.\n\n## What Happens When You Skip the JSON Restrictor\n\nThe bigger change is the full response, which I called Raw in the app. In the first blog, sending a raw SerpApi response to a local model wasn't an option. The seven raw JSON responses added up to 326,949 tokens, almost 10 times the context window. The same seven responses in Markdown came to 71,244 tokens, 4.6 times smaller, and none of them was bigger than the window.\n\nFitting in the window doesn't make it fast. With Markdown (Raw), the models read two to three times as many tokens as with Markdown (Restrictor), and took up to 2.7 times as long to answer.\n\nTwo full pages in one question can still overflow the context window. On one of the four models, the prompt comparing AAPL and TSLA went over 32K tokens. LM Studio cut part of it without an error, and the answer got Apple’s daily change wrong.\n\n## Which Output Should You Use?\n\nMost local chats should start with Markdown using the restrictor, and the other outputs fit narrower cases.\n\n- **Markdown (Restrictor):** The default. Use it whenever a model reads the results.\n- **JSON (Restrictor):** Use it when your code reads the results, for example to sort by price.\n- **Markdown (Raw):** Use it when an answer needs the whole page and can wait up to three times longer. Avoid it for questions that need two big searches, such as comparing two stocks.\n- **JSON (Raw):** Don't send it to a local model.\n\nIf your hardware has the memory, you can raise the 32K window with the app’s `LMSTUDIO_CONTEXT_LENGTH` setting.\n\n## How the Four Models Compare\n\nAll four models finished every question. The times below are the average per search question in Markdown (Restrictor):\n\n- **Qwen 3.8 27B** is the app's default model when it's installed. In long test conversations, it turned follow-ups like \"around there\" and \"which of those\" into the right searches. It took about 42 seconds per question.\n- **Meta Muse Glimmer 30B** splits text into fewer tokens than the other three, so the same search results take less of its window. It was the slowest, about 62 seconds per question.\n- **Prism Bonsai 27B** is a 27B model that LM Studio lists at about 4 GB, and it took about 24 seconds per question.\n- **Gemma 4 12B QAT** is the smallest model here, at 12B parameters, and the fastest, at about 16 seconds per question, or about 43 with Markdown (Raw). In long conversations it had the most problems, such as giving a price that wasn’t in the search results.\n\nWith less memory than the 48 GB used here, the app may load a smaller window, so less search data fits. New open models also come out almost every week, so the best choice changes fast. The staff picks in [__LM Studio's model search__](https://lmstudio.ai/docs/app/basics/download-model) are a good place to start, because LM Studio matches them to your hardware.\n\n## When Search Results Contradict What the Model Knows\n\nEvery model stops learning at its training cutoff, so it doesn’t know what happened after that date. On September 1, 2026, John Ternus became Apple's CEO and Tim Cook became executive chairman, after 15 years as CEO. So \"*Who is the current CEO of Apple?*\" puts a model's training against the search results.\n\nHere's what Qwen 3.8 27B thought before its first search:\n\n*The user is asking who the current CEO of Apple is. This is a factual question. As of my knowledge, Tim Cook has been Apple's CEO since 2011. However, since this is a question about the current situation, I should verify with a search to make sure there haven't been any changes. Let me search.*\n\nThat part is right. The model knows its knowledge can be out of date, and it checks. The harder part comes when the results disagree with it. In an earlier run, the AI Overview in Qwen's first results already named John Ternus, and Qwen didn't accept it.\n\n*This is a significant claim - it suggests Tim Cook stepped down and John Ternus took over. Today's date is 2026-10-02, so this would be about a month ago. Let me verify this with a news search to make sure this is accurate and not a hallucination.*\n\nWe can read all of this because the models run locally. LM Studio writes each model's reasoning to its server log, word for word. Hosted reasoning models usually show you a summary instead. When OpenAI launched o1, it [__chose not to show the raw chain of thought__](https://openai.com/index/learning-to-reason-with-llms/#:~:text=0.976-,Hiding%20the%20Chains%20of%20Thought,-We%20believe%20that), and other AI labs have done the same recently. One competitive risk is [__knowledge distillation__](https://www.ibm.com/think/topics/knowledge-distillation), where a smaller model learns to copy a bigger model's outputs, and full reasoning is valuable material for that.\n\n## Conclusion\n\nIn this blog we added SerpApi’s Markdown output to the app and benchmarked it on four local models. For the same fields, Markdown took 27% fewer tokens than JSON, so the restrictor could keep more results for about the same budget. Even without a restrictor, all seven full Markdown responses fit in the 32K window.\n\nThe same approach works for any SerpApi engine. Add `output=md` to the request, keep one restrictor per engine, and spend the tokens Markdown saves on the fields your questions need. To try it, clone the repo and add your SerpApi API key. [__A free account__](https://serpapi.com/users/sign_up?plan=free) includes 250 searches a month.\n\nWhen the next model you want to try shows up in LM Studio, run it through the app's Benchmark page and compare it with these four.", "url": "https://wpnews.pro/news/how-to-fit-more-web-search-data-into-a-local-llm", "canonical_source": "https://serpapi.com/blog/local-llm-web-search-markdown-output/", "published_at": "2026-10-16 20:51:47+00:00", "updated_at": "2026-10-16 20:55:20.197356+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "ai-search", "developer-tools"], "entities": ["SerpApi", "JSON Restrictor", "LM Studio", "TanStack Start", "OpenAI", "tiktoken", "Qwen3.5-9B", "LM Link"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-to-fit-more-web-search-data-into-a-local-llm", "markdown": "https://wpnews.pro/news/how-to-fit-more-web-search-data-into-a-local-llm.md", "text": "https://wpnews.pro/news/how-to-fit-more-web-search-data-into-a-local-llm.txt", "jsonld": "https://wpnews.pro/news/how-to-fit-more-web-search-data-into-a-local-llm.jsonld"}}