# Fix Local LLM Quality: Context Stacking & Rope Freq Tweaks > Source: > Published: 2026-08-23 04:33:14+00:00 This article was originally published on[BuildZn]. Everyone's running local LLMs now, which is great. But then they hit the wall: "Why does my 7B model on Ollama feel dumber than a cloud API?" You've got the tokens/second, but the *quality* sucks. Figured it out the hard way after pulling my hair out trying to get better local LLM quality improvement for agent tasks. I’ve shipped FarahGPT to 5,100+ users and built multi-agent systems like NexusOS. I know what it takes to get an LLM to think, not just parrot. When I started building out a 9-agent YouTube automation pipeline locally, the raw output from quantized models was... dismal. Lots of factual errors, incoherent steps, total garbage. You'd think a Qwen 7B Q4_K_M model would at least manage basic reasoning, but default settings often choke it. The problem isn't always the model itself or your hardware. It's how you talk to it and how you let it configure its own internal world. We’re pushing these models to run on consumer hardware, often with heavy quantization. **Expecting them to perform like a 70B cloud model out of the box is naive.** They need help to maximize their limited capacity. This is where specific `modelfile` tweaks and prompt engineering for better local LLM reasoning come in. Here’s the thing — most guides tell you to increase `num_ctx` . Yeah, sure, more context is usually better. But it’s a blunt instrument. You’re missing the finer controls that genuinely improve how the model *processes* that context, especially for complex, multi-step agent operations. I don't get why most people stop at `num_ctx` . To actually fix this, you need to hit it from two angles: how you structure your prompts (the input) and how the model internally handles positional embeddings (its understanding of that input). This isn't just "system, user" stuff. This is about giving the model a structured thinking process *within* the prompt, making it explicitly think step-by-step. I call it "context-stacking" because you're layering context and instructions in a way that guides its internal monologue. This significantly helps with quantized model prompt engineering. Here's the pattern that works for me: **Example Prompt Structure (for a coding agent task):** ``` SYSTEM: You are an expert Python developer assistant for building Flask APIs. Your goal is to generate a simple Flask REST API endpoint that returns JSON data. Here are the strict rules you MUST follow: - Only use standard Flask libraries. No third-party API frameworks unless explicitly requested. - Ensure the output is a complete, runnable Flask application. - Prioritize clear, concise code. - Do NOT generate any setup or installation instructions. - Do NOT include example `curl` commands. - The endpoint must handle GET requests. The desired endpoint should be `/api/users` and return a list of user dictionaries, each with `id` and `name`. USER: Generate the Flask API for `/api/users`. AGENT THOUGHT PROCESS: 1. **Analyze Request:** The user wants a Flask GET endpoint at `/api/users` returning a list of user dictionaries. 2. **Required Imports:** Need `Flask` and `jsonify`. 3. **App Setup:** Instantiate Flask app. 4. **Route Definition:** Define `@app.route('/api/users', methods=['GET'])`. 5. **Data Structure:** Create a sample list of user dictionaries. 6. **Response:** Use `jsonify` to return the list. 7. **Main Block:** Add `if __name__ == '__main__': app.run(debug=True)`. 8. **Review Constraints:** Check all rules: standard Flask, complete app, concise, no setup/curl. Looks good. FINAL RESPONSE: ``` python from flask import Flask, jsonify app = Flask(**name**) users = [ {"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"}, {"id": 3, "name": "Charlie"} ] @app.route('/api/users', methods=['GET']) def get_users(): """ Returns a list of all users. --- responses: 200: description: A list of users. """ return jsonify(users) if **name** == '**main**': app.run(debug=True) plaintext See that `AGENT THOUGHT PROCESS:` ? That's not just for show. You *tell* the model to output that. It forces it to allocate tokens to internal reasoning before spitting out a `FINAL RESPONSE:` . This dramatically improves task completion coherence. `Modelfile` Parameter Tweaks: RoPE Frequencies This is where things get interesting and where most developers miss the mark. Forget just `num_ctx` for a minute. The `rope_freq_base` and `rope_freq_scale` parameters in your Ollama `Modelfile` are critical for how the model understands the *position* of tokens within its context window. Changing these can impact how well it discerns relationships between widely separated tokens. It's a key part of Ollama quality configuration. Here’s the deal: many quantized models, especially smaller ones, struggle with long-range dependencies and complex reasoning because their default RoPE (Rotary Positional Embedding) settings might not be optimal for the reduced precision. **My Fix for Qwen 7B Q4_K_M:** I built a custom `Modelfile` for `qwen:7b-chat-q4_K_M` (downloaded from Ollama) and explicitly set these. **Here’s the Modelfile snippet:** ``` FROM qwen:7b-chat-q4_K_M # Set a larger context window, but this isn't the primary lever for quality here PARAMETER num_ctx 4096 # The magic sauce for improved local LLM quality improvement: # These values are specific to Qwen architecture and quantization. # Experimentation is key, but these are a good starting point for 7B Qwen. # rope_freq_base controls the base frequency for the RoPE embeddings. # A lower value can sometimes help with longer contexts by making positional # information "decay" slower, improving long-range coherence. PARAMETER rope_freq_base 50000 # rope_freq_scale applies a scaling factor to the RoPE frequencies. # Adjusting this can fine-tune how quickly positional information changes # across the sequence length, impacting the model's ability to locate tokens. # For quantized models, slight adjustments can stabilize context understanding. PARAMETER rope_freq_scale 0.8 ``` To use this, save it as `Modelfile` in a directory, then run: `ollama create my-qwen-smart -f ./Modelfile` Then you can use `ollama run my-qwen-smart` . **Why these values?** Default RoPE settings are often optimized for the full-precision, non-quantized model. When you quantize, you introduce noise and lose precision. Tweaking `rope_freq_base` and `rope_freq_scale` can essentially "re-tune" the positional encoding to be more robust to this noise, helping the model better understand token relationships across the context. It's like re-calibrating its internal compass. This is a subtle but powerful lever for better local LLM reasoning. **The Numbers (Real Talk):** After combining the **context-stacking prompt technique** with these `rope_freq_base` (set to `50000` from default `10000` ) and `rope_freq_scale` (set to `0.8` from default `1.0` ) `modelfile` parameters on my `Qwen 7B Q4_K_M` model running via Ollama 0.1.29 on an RTX 4090 (with 16 layers loaded onto VRAM, hitting about 12.4 tok/s for generation after a full context prompt), I observed: `modelfile` with only `num_ctx` increased.This isn't about raw speed (which remained consistent at ~12.4 tok/s when measuring over 100 runs for generating ~200 tokens). It's purely about output quality. The `llama.cpp smart tips` aren't always about speed. Initially, I just threw more `num_ctx` at the problem and tried longer, more verbose prompts. That helped a bit, but often made the output *more* convoluted. The model would just fill up the extra context with verbose, but often irrelevant, fluff. It was like giving a confused person more books; they just get more overwhelmed. Another mistake was blindly copying `Modelfile` settings for different models. A `rope_freq_base` that works for Llama 2 might completely screw up Mistral or Qwen. **The rope_freq_base and rope_freq_scale values are highly model-architecture dependent.** You I also hit a weird behavior with Ollama 0.1.28 where repeated multi-turn conversations would sometimes drop the *entire* `system` prompt context after 3-4 turns, leading to completely nonsensical replies, almost like it had amnesia. Upgrading to 0.1.29 resolved this, so keep your Ollama version updated, folks. Even with these tweaks, local LLMs still aren't god-tier. **The real secret is iterative refinement.** After the initial output using the context-stacking and `modelfile` tweaks, I often pipe that output back into the model with a "Critique and Refine" prompt. ``` SYSTEM: You are a meticulous code reviewer. Your task is to identify errors, suggest improvements for clarity, security, and efficiency, and then rewrite the provided code. USER: Critique the following Python Flask code. Focus on: - Adherence to best practices. - Potential security vulnerabilities. - Readability and maintainability. - Correctness of implementation. CODE: [Initial code generated by the agent] AGENT THOUGHT PROCESS: 1. **Review Code:** Read through the Flask code provided. 2. **Check Best Practices:** Is it idiomatic Flask? 3. **Security Scan:** Look for common Flask vulnerabilities (e.g., debug mode in production, unsanitized input, no CSRF protection, if applicable). 4. **Clarity/Maintainability:** Are variable names clear? Is the structure logical? Add docstrings where missing. 5. **Correctness:** Does it actually solve the problem? 6. **Formulate Feedback:** Write a concise critique. 7. **Generate Refined Code:** Provide the improved version. FINAL CRITIQUE: ... REFINED CODE: ... ``` This multi-step approach, where one agent generates and another critiques, is a game-changer for getting genuinely useful output from local models. It mimics how humans collaborate and self-correct. `rope_freq_base` impact LLM quality? `rope_freq_base` directly influences how the model's positional embeddings are calculated. By adjusting it, you can change how quickly positional information "decays" across the sequence, potentially improving the model's ability to track long-range dependencies and token relationships within a large context, especially for quantized models where precision is reduced. `Modelfile` tweaks for any local LLM? While the concept applies, the specific `rope_freq_base` and `rope_freq_scale` values are highly dependent on the model's architecture (e.g., Llama, Mistral, Qwen) and its quantization level. You'll need to experiment with different values for your specific model to find the optimal settings. Start with the defaults and make small, incremental changes. `num_ctx` and `rope_freq_base` for local LLM performance tips? `num_ctx` simply expands the *maximum length* of the context window the model can process, allowing more tokens in. `rope_freq_base` , on the other hand, tweaks *how* the model understands the *position* of those tokens within that context. While `num_ctx` provides the capacity, `rope_freq_base` refines the model's ability to interpret positional information, leading to better contextual understanding and reasoning quality, not just more tokens. The default settings on Ollama are good starting points, but they're not optimized for every model or every use case, especially when you're pushing quantized models for complex reasoning. If your local LLM feels dumb, it's probably because you haven't given it the right tools to think. Combine intelligent prompt engineering with targeted `modelfile` tweaks like `rope_freq_base` and `rope_freq_scale` . It's not a silver bullet, but it's the closest thing to a quality upgrade for your local setup that doesn't involve buying a new GPU. Get those models working smarter, not just faster.