{"slug": "my-local-llm-was-running-at-1-6-of-its-context-here-s-the-setting-that-fixed-it", "title": "My Local LLM Was Running at 1.6% of Its Context. Here's the Setting That Fixed It", "summary": "A developer discovered that their local LLM was running at only 1.6% of its context capacity due to a default setting in LM Studio, which capped the context length at 4096 tokens despite the model supporting 262,144. After increasing the context length to 260,000, the model's performance on long-document tasks improved dramatically, highlighting the importance of checking context settings over blaming model weights.", "body_md": "I run a content pipeline on a Mac mini (48GB unified memory) that splits long blog drafts into platform-specific short-form pieces. That job — read a 30-page document, hold the whole thing in mind, extract what matters for YouTube Shorts vs TikTok vs Reels — is exactly what long-context LLMs are supposed to be good at.\n\nMine wasn't. It kept \"forgetting\" the second half of every document, dropping key details, and producing shallow summaries no matter how I tuned the prompt.\n\nI did what you'd do. Simplified the prompt. Rewrote the template. Swapped models. Re-downloaded them, twice. Spent entire evenings after work on this, convinced the model was the problem — a Q4_K_M quantized 13B–20B model should handle long documents, right? The symptoms said otherwise: solid on the first pages, incoherent by the end.\n\nClassic context-window behavior. I just didn't see it yet.\n\nThen I actually read the LM Studio load log instead of scrolling past it:\n\n```\ncontext_length: 4096\n```\n\nThe model I was running supports **262,144 tokens of context**. It was loaded with **4,096**.\n\nThat's 1.6% of what the model can do. A 48-lane highway restricted to one lane — and every long document I fed it was quietly getting truncated into memory of just the opening section.\n\nLM Studio's just-in-time model loading picks a conservative default context length on first load. For chat and short Q&A, 4096 is plenty and keeps memory pressure low — a sensible default for most users. For document-scale work, it's a silent killer. Nothing errors out. Nothing warns you. The model just appears to have a bad memory.\n\nContext is the model's working memory. Cap it at 4k tokens and a 30-page brief becomes \"read the first two pages, forget the rest.\"\n\nTwo things:\n\n`Context Length: 260000`\n\n(whatever your model supports — check the model card, not the default), then reload. On 48GB of unified memory the larger KV cache is entirely affordable.Immediate, dramatic improvement. Full-document comprehension, per-platform extraction without drift, details intact end to end. Same model, same hardware, same prompt — one setting was capping ~98% of the model's effective utility for my workload.\n\nDefaults are tuned for the average case, and document-scale synthesis is not the average case. When a local LLM \"feels dumb,\" check what it was actually loaded with before blaming the weights:\n\n`context_length`\n\nis actually in effect?The most expensive performance bug I've shipped was a single default value.\n\n*This post is based on a first-hand work log, written with AI assistance.*", "url": "https://wpnews.pro/news/my-local-llm-was-running-at-1-6-of-its-context-here-s-the-setting-that-fixed-it", "canonical_source": "https://dev.to/devlog/my-local-llm-was-running-at-16-of-its-context-heres-the-setting-that-fixed-it-3i4j", "published_at": "2026-08-25 09:06:37+00:00", "updated_at": "2026-08-25 09:14:15.576794+00:00", "lang": "en", "topics": ["large-language-models", "developer-tools", "mlops"], "entities": ["LM Studio", "Mac mini"], "alternates": {"html": "https://wpnews.pro/news/my-local-llm-was-running-at-1-6-of-its-context-here-s-the-setting-that-fixed-it", "markdown": "https://wpnews.pro/news/my-local-llm-was-running-at-1-6-of-its-context-here-s-the-setting-that-fixed-it.md", "text": "https://wpnews.pro/news/my-local-llm-was-running-at-1-6-of-its-context-here-s-the-setting-that-fixed-it.txt", "jsonld": "https://wpnews.pro/news/my-local-llm-was-running-at-1-6-of-its-context-here-s-the-setting-that-fixed-it.jsonld"}}