# Qwen 27B with 128k Context on 24GB VRAM? > Source: > Published: 2026-07-25 20:01:48+00:00 # Qwen 27B with 128k Context on 24GB VRAM? Running a 27B model with a 128k context window on a single 24GB card sounds like a recipe for an OOM (Out of Memory) crash, but I'm seeing reports that it's actually working. The key seems to be a specific combo of Ollama environment variables that drastically cuts down the memory footprint of the KV cache. It's a decent deep dive into how much we can optimize the AI workflow by tweaking the backend rather than just hoping for a smaller model. If you've been capping your context at 32k because of VRAM limits, definitely test these flags. If you're trying to squeeze long-context LLM agents into limited VRAM, you need to look at these two flags: ``` OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q4_0 ``` The `q4_0` KV cache quantization is the real heavy lifter here. Instead of storing the cache in FP16, it compresses it, which is why someone is reporting 83% VRAM usage at 128k context on a 24GB card. Without this, the KV cache for a 27B model at that context length would easily blow past 24GB on its own, regardless of the model quantization. The performance hit is surprisingly minimal too—hitting around 30 tok/s. If it spills over into system RAM, you're looking at a drop to maybe 4 tok/s, which makes the local deployment basically unusable for real-time work. A few observations from this setup: **Q4 Quantization:** 128k context fits comfortably.**Q5 Quantization:** Hits a wall around 96k context. It's a decent deep dive into how much we can optimize the AI workflow by tweaking the backend rather than just hoping for a smaller model. If you've been capping your context at 32k because of VRAM limits, definitely test these flags. [Next DeepSeek-R1 Local Deployment: My Hardware Struggles →](/en/threads/3346/) ## All Replies （4） Q Flash attention 2 makes a huge difference here, though it's a pain to compile sometimes. 0 N KV cache quantization helps, but I still hit a wall around 64k on my 3090. 0 T Does the perplexity tank as you get closer to that limit, or does it hold up? 0 J That's the real question. I've noticed some "lost in the middle" issues with other models, wonder if Qwen does the same. 0