How to Fit More Web Search Data into a Local LLM

SerpApi's new Markdown output lets a local LLM fit more web search data into its context window, because the same results take fewer tokens in Markdown than in JSON, according to the company's benchmark of seven searches. The benchmark compared four outputs — raw JSON, JSON with the JSON Restrictor, Markdown with the restrictor, and raw Markdown — using OpenAI's tiktoken cl100k_base encoding, and found that for most searches the full response now fits with no restrictor at all. The accompanying open-source app, built with TanStack Start 1.0 and running models in LM Studio with a 32K context window, defaults to Markdown with the restrictor.

A local model can only use the web search results that fit in its context window https://www.ibm.com/think/topics/context-window . When we built the app for How to Connect Your Local LLM with Web Search Data https://serpapi.com/blog/how-to-connect-your-local-llm-with-web-search-data/ , a single SerpApi response was often bigger than the whole context window available. So we trimmed every response with the JSON Restrictor https://serpapi.com/json-restrictor , which keeps only the fields the model needs and drops the rest on SerpApi’s servers before it reaches the LLM. SerpApi's new Markdown output https://serpapi.com/markdown-output changes that budget. The same data takes fewer tokens in Markdown than in JSON, so the restrictor can keep more of it. For most searches, the full response now fits with no restrictor at all. This blog will show you how to apply the JSON Restrictor to Markdown output, and when the full response is worth the wait. We'll use the same open-source app https://github.com/serpapi/local-llm-web-search that gives a local model running in LM Studio https://lmstudio.ai internet access. The app is built with TanStack Start https://tanstack.com/start , and we upgraded it to Start 1.0 along the way. What’s New Since the First Blog SerpApi's Markdown output returns curated results as a Markdown document instead of JSON. You add output=md to a request, and the results come back as headings, lists and tables without the JSON keys, quotes and braces. It works across SerpApi's 100+ APIs https://serpapi.com/search-engine-apis , and a Markdown search costs the same as a JSON search. The launch blog Introducing SerpApi’s New Markdown Output https://serpapi.com/blog/introducing-serpapi-markdown-output/ covers the format and its token savings across our engines. The app now lets you choose what the model reads after each search. The benchmark compares four outputs, and a model only ever reads three of them. | Output | What the model reads | Parameters | |---|---|---| | JSON Raw | The full JSON response. The benchmark measures it, but no model reads it | No restrictor | | JSON Restrictor | JSON trimmed to the fields the model needs, as in the first blog | json restrictor | | Markdown Restrictor | Markdown with more results and more fields, at about the same size. This is the app's default | json restrictor and output=md | | Markdown Raw | The full Markdown document | output=md | The setup changed too. The first blog ran Qwen3.5-9B on a MacBook Pro M4 Max with 36 GB of memory. This time I ran four of LM Studio's staff picks on a MacBook Pro M5 Pro with 48 GB, using LM Link https://lmstudio.ai/link to connect both MacBooks. They are: How We Tested Every model ran with a 32K context window, the same size as in the first blog. LM Studio loads a model with an 8K window by default, which is too small once a chat carries search results. A 32K window is enough for a long conversation, and it fits a 27B model in memory on a 36 GB Mac. What the Benchmark Measures The app's Benchmark page measures two things, and this blog uses both. Payload size: The same seven searches run once for each of the four outputs, with no model involved. There's one search per tool, covering Search https://serpapi.com/search-api , Finance https://serpapi.com/google-finance-api , News https://serpapi.com/google-news-api , Maps https://serpapi.com/google-maps-api , Flights https://serpapi.com/google-flights-api , Hotels https://serpapi.com/google-hotels-api and Shopping https://serpapi.com/google-shopping-api . This shows what each output costs before a model reads anything. These sizes come from OpenAI's tiktoken https://github.com/openai/tiktoken library with its cl100k base encoding, which splits text into tokens. Each local model splits text its own way, so treat these sizes as estimates for comparing formats. End to end: Thirteen questions run through the whole app on each model, once for each of the three outputs a model can read. Seven need live data from SerpApi, one search per tool. Two need two searches at once, such as comparing AAPL with TSLA. The other four need no search, such as " What is 15% of 240? ", so they show how the model behaves on its own. We used it to compare how each model handles each output, how long it takes, and what it answers. How Much Does Markdown Save? To see what the format alone saves, we kept the restrictor strings from the first blog and sent the same seven searches as JSON and as Markdown. The seven results took 4,668 tokens in JSON and 3,412 in Markdown, 27% fewer. Here's how each tool changed: | Tool | JSON | Markdown | Change | |---|---|---|---| | Flights | 823 | 439 | −47% | | Finance | 521 | 324 | −38% | | News | 678 | 422 | −38% | | Search | 785 | 586 | −25% | | Hotels | 567 | 433 | −24% | | Maps | 497 | 449 | −10% | | Shopping | 797 | 759 | −5% | | Total | 4,668 | 3,412 | −27% | The saving depends on how deeply the data is nested. A flight option holds legs, and each leg holds airports and times, so JSON repeats keys, quotes and braces at every level. Markdown turns lists of results into tables and writes each column name once. Shopping results are flat, with a title, a price and a store per product, so there is little to remove. So instead of keeping that saving, we spend it. In Markdown, the same token budget can carry more fields and more results, so the Markdown output gets a looser restrictor. Apply the JSON Restrictor to Markdown Output There's no separate restrictor for Markdown. You send the same json restrictor parameter together with output=md , and SerpApi filters the sections first, then renders what's left as Markdown. In the app, each tool has two restrictor strings. JSON sends the original one, and Markdown sends a looser one with more results and more fields. The Markdown request is the same search with output=md added: // Pick the restrictor for the output mode. Markdown Raw sends none. function restrictorFor name: RestrictorTool, output: ToolOutput : string | null { if output === "markdown-full" return null return output === "markdown" ? MARKDOWN RESTRICTORS name : RESTRICTORS name } // Same parameters as the JSON request, plus output=md. async function serpApiMarkdown params: Record<string, unknown : Promise<string { const url = new URL "https://serpapi.com/search" for const key, value of Object.entries params { if value == undefined && value == null url.searchParams.set key, String value } url.searchParams.set "output", "md" url.searchParams.set "api key", process.env.SERPAPI API KEY ?? "" const res = await fetch url return res.text } Each tool spends the saving on different extras: | Tool | JSON Restrictor keeps | Markdown Restrictor keeps | |---|---|---| | Search | Answer box, AI Overview https://serpapi.com/google-ai-overview-api 6 blocks , 3 related questions, knowledge graph, 5 web results | 5 web results, answer box with its lists, AI Overview 8 blocks , knowledge graph, 3 top stories, 3 related questions | | Finance | Price, key stats, 3 news items | Price, key stats, company description, 5 news items | | News | 5 stories, 1 related article each | 7 stories, 2 related articles each | | Maps | 5 places | 6 places, with opening hours and a description | | Flights | 3 best and 3 other options | 4 best and 4 other options, with aircraft, legroom, overnight layovers and price insights | | Hotels | 5 hotels | 7 hotels, with a description and a location rating | | Shopping | 5 products, plus 3 inline results | 7 products, plus 3 inline results, with second-hand condition | For example, here are both restrictor strings for Maps, from restrictors.ts : js // JSON Restrictor : five places. export const RESTRICTORS = { google maps search: "local results 0:5 .{title,rating,reviews,type,address,price,open state,phone,website,gps coordinates}", "place results.{title,rating,reviews,type,address,price,open state,phone,website}", "error", "search metadata.google maps url", .join "," , // ...one restrictor per tool } // Markdown Restrictor : six places, with opening hours and a description. export const MARKDOWN RESTRICTORS = { google maps search: "local results 0:6 .{title,rating,reviews,type,address,price,open state,hours,phone,website,description}", "place results.{title,rating,reviews,type,address,price,open state,hours,phone,website,description}", "error", "search metadata.google maps url", .join "," , // ...one restrictor per tool } The extra data shows up in the answers. Asked for the best barbecue in Austin, Qwen 3.8 27B listed five places with JSON and six with Markdown. The new one, Good BBQ Company, was one of the two highest rated. The Markdown answer also described Terry Black's as "4th-generation pitmasters, old-school smoked meats," from the place descriptions only the Markdown restrictor keeps. What Happens When You Skip the JSON Restrictor The bigger change is the full response, which I called Raw in the app. In the first blog, sending a raw SerpApi response to a local model wasn't an option. The seven raw JSON responses added up to 326,949 tokens, almost 10 times the context window. The same seven responses in Markdown came to 71,244 tokens, 4.6 times smaller, and none of them was bigger than the window. Fitting in the window doesn't make it fast. With Markdown Raw , the models read two to three times as many tokens as with Markdown Restrictor , and took up to 2.7 times as long to answer. Two full pages in one question can still overflow the context window. On one of the four models, the prompt comparing AAPL and TSLA went over 32K tokens. LM Studio cut part of it without an error, and the answer got Apple’s daily change wrong. Which Output Should You Use? Most local chats should start with Markdown using the restrictor, and the other outputs fit narrower cases. - Markdown Restrictor : The default. Use it whenever a model reads the results. - JSON Restrictor : Use it when your code reads the results, for example to sort by price. - Markdown Raw : Use it when an answer needs the whole page and can wait up to three times longer. Avoid it for questions that need two big searches, such as comparing two stocks. - JSON Raw : Don't send it to a local model. If your hardware has the memory, you can raise the 32K window with the app’s LMSTUDIO CONTEXT LENGTH setting. How the Four Models Compare All four models finished every question. The times below are the average per search question in Markdown Restrictor : - Qwen 3.8 27B is the app's default model when it's installed. In long test conversations, it turned follow-ups like "around there" and "which of those" into the right searches. It took about 42 seconds per question. - Meta Muse Glimmer 30B splits text into fewer tokens than the other three, so the same search results take less of its window. It was the slowest, about 62 seconds per question. - Prism Bonsai 27B is a 27B model that LM Studio lists at about 4 GB, and it took about 24 seconds per question. - Gemma 4 12B QAT is the smallest model here, at 12B parameters, and the fastest, at about 16 seconds per question, or about 43 with Markdown Raw . In long conversations it had the most problems, such as giving a price that wasn’t in the search results. With less memory than the 48 GB used here, the app may load a smaller window, so less search data fits. New open models also come out almost every week, so the best choice changes fast. The staff picks in LM Studio's model search https://lmstudio.ai/docs/app/basics/download-model are a good place to start, because LM Studio matches them to your hardware. When Search Results Contradict What the Model Knows Every model stops learning at its training cutoff, so it doesn’t know what happened after that date. On September 1, 2026, John Ternus became Apple's CEO and Tim Cook became executive chairman, after 15 years as CEO. So " Who is the current CEO of Apple? " puts a model's training against the search results. Here's what Qwen 3.8 27B thought before its first search: The user is asking who the current CEO of Apple is. This is a factual question. As of my knowledge, Tim Cook has been Apple's CEO since 2011. However, since this is a question about the current situation, I should verify with a search to make sure there haven't been any changes. Let me search. That part is right. The model knows its knowledge can be out of date, and it checks. The harder part comes when the results disagree with it. In an earlier run, the AI Overview in Qwen's first results already named John Ternus, and Qwen didn't accept it. This is a significant claim - it suggests Tim Cook stepped down and John Ternus took over. Today's date is 2026-10-02, so this would be about a month ago. Let me verify this with a news search to make sure this is accurate and not a hallucination. We can read all of this because the models run locally. LM Studio writes each model's reasoning to its server log, word for word. Hosted reasoning models usually show you a summary instead. When OpenAI launched o1, it chose not to show the raw chain of thought https://openai.com/index/learning-to-reason-with-llms/ :~:text=0.976-,Hiding%20the%20Chains%20of%20Thought,-We%20believe%20that , and other AI labs have done the same recently. One competitive risk is knowledge distillation https://www.ibm.com/think/topics/knowledge-distillation , where a smaller model learns to copy a bigger model's outputs, and full reasoning is valuable material for that. Conclusion In this blog we added SerpApi’s Markdown output to the app and benchmarked it on four local models. For the same fields, Markdown took 27% fewer tokens than JSON, so the restrictor could keep more results for about the same budget. Even without a restrictor, all seven full Markdown responses fit in the 32K window. The same approach works for any SerpApi engine. Add output=md to the request, keep one restrictor per engine, and spend the tokens Markdown saves on the fields your questions need. To try it, clone the repo and add your SerpApi API key. A free account https://serpapi.com/users/sign up?plan=free includes 250 searches a month. When the next model you want to try shows up in LM Studio, run it through the app's Benchmark page and compare it with these four.