# My Local LLM Was Running at 1.6% of Its Context. Here's the Setting That Fixed It

> Source: <https://dev.to/devlog/my-local-llm-was-running-at-16-of-its-context-heres-the-setting-that-fixed-it-3i4j>
> Published: 2026-08-25 09:06:37+00:00

I run a content pipeline on a Mac mini (48GB unified memory) that splits long blog drafts into platform-specific short-form pieces. That job — read a 30-page document, hold the whole thing in mind, extract what matters for YouTube Shorts vs TikTok vs Reels — is exactly what long-context LLMs are supposed to be good at.

Mine wasn't. It kept "forgetting" the second half of every document, dropping key details, and producing shallow summaries no matter how I tuned the prompt.

I did what you'd do. Simplified the prompt. Rewrote the template. Swapped models. Re-downloaded them, twice. Spent entire evenings after work on this, convinced the model was the problem — a Q4_K_M quantized 13B–20B model should handle long documents, right? The symptoms said otherwise: solid on the first pages, incoherent by the end.

Classic context-window behavior. I just didn't see it yet.

Then I actually read the LM Studio load log instead of scrolling past it:

```
context_length: 4096
```

The model I was running supports **262,144 tokens of context**. It was loaded with **4,096**.

That's 1.6% of what the model can do. A 48-lane highway restricted to one lane — and every long document I fed it was quietly getting truncated into memory of just the opening section.

LM Studio's just-in-time model loading picks a conservative default context length on first load. For chat and short Q&A, 4096 is plenty and keeps memory pressure low — a sensible default for most users. For document-scale work, it's a silent killer. Nothing errors out. Nothing warns you. The model just appears to have a bad memory.

Context is the model's working memory. Cap it at 4k tokens and a 30-page brief becomes "read the first two pages, forget the rest."

Two things:

`Context Length: 260000`

(whatever your model supports — check the model card, not the default), then reload. On 48GB of unified memory the larger KV cache is entirely affordable.Immediate, dramatic improvement. Full-document comprehension, per-platform extraction without drift, details intact end to end. Same model, same hardware, same prompt — one setting was capping ~98% of the model's effective utility for my workload.

Defaults are tuned for the average case, and document-scale synthesis is not the average case. When a local LLM "feels dumb," check what it was actually loaded with before blaming the weights:

`context_length`

is actually in effect?The most expensive performance bug I've shipped was a single default value.

*This post is based on a first-hand work log, written with AI assistance.*