My commit message said "You've hit your session limit"

A developer replaced Claude with a local Ollama model for generating git commit messages after hitting Claude's usage limit, which resulted in a commit message reading 'You've hit your session limit'. The developer chose the quantized qwen2.5-coder:1.5b model for its small size and low memory footprint, but found that Ollama's default context window of 4096 tokens caused the model to produce inaccurate commit messages based on partial diffs. By creating a Modelfile with an increased context window of 8192 tokens and a lower temperature of 0.2, the developer improved the model's performance for this task.

I had this one-liner that I was using. git commit -m "$ git diff --staged | claude -p "Provide a simple, one-line git commit message based on this diff following best practices. Output absolutely nothing else." " Pipe the staged diff to Claude, get a commit message back. Worked well until I hit my Claude usage limit mid commit. The shell captured the error instead of a commit message. So I had a commit in my repo that said: You've hit your session limit That's when it hit me Voila, ✨My use case for a Local Model.✨ Ollama https://ollama.com/ lets you run open source models locally. After installing it, you have a server running at http://localhost:11434 . ollama pull qwen2.5-coder:1.5b I picked qwen2.5-coder:1.5b because it's small and code-aware. Why 1.5b specifically? My laptop has 8GB RAM. That's not a lot when you're running a model locally. Here's the rough math these are estimates from my machine, yours may vary : Interestingly, despite being a 1.5 billion parameter model, qwen2.5-coder:1.5b only takes up about 1 GB of disk space. That's because it's a quantized model. Quantization https://huggingface.co/docs/optimum/en/concept guides/quantization means the model's weights https://www.ultralytics.com/glossary/model-weights model-weights-vs-biases are stored at lower precision, using 4-bit or 8-bit integers instead of the usual 16-bit or 32-bit floating point numbers. This significantly reduces the model size and memory footprint, although it may slightly impact accuracy. I tried larger models. My laptop became unusable. Fans spinning, apps freezing, the whole thing. So 1.5b it is. There's another quantized model I found that could work — gemma3:1b-it-qat . I plan to test it sometime and see how it compares in terms of performance and resource usage. I swapped Claude with Ollama in my one-liner: git commit -m "$ git diff --staged | ollama run qwen2.5-coder:1.5b "Provide a simple, one-line git commit message based on this diff following best practices. Output absolutely nothing else." " I ran it against a change where I had removed the tools section from some agent config front matter from 6 files. This Worked The commit message said it was a change to a README file. Despite qwen2.5-coder:1.5b's large native context window of 32,768 tokens, Ollama actually restricts the default context size when running without a Modelfile. I checked Ollama's logs and found this line: level=INFO source=routes.go:2073 msg="vram-based default context" total vram="5.3 GiB" default num ctx=4096 It shows that based on my machine's VRAM of 5.3 GiB, Ollama set a default num ctx of 4096 tokens. That's why the model only saw the beginning of the diff and guessed about the README file. I thought maybe I need a better prompt. So I ran it again with more instructions. This time it said the change was in code-reviewer.md . That was one of the 6 files, and it completely ignored the other 5. The important thing here is that the model did not complain. It did not say "I couldn't read the rest". It just gave me a confident answer based on partial input. At this point I understood tuning the prompt alone is insufficient and I need to tune the model too. This is something I just learned. A Modelfile is a config layer on top of a base model. You can change parameters and create a named model from it. FROM qwen2.5-coder:1.5b PARAMETER num ctx 8192 PARAMETER temperature 0.2 Two things I changed: num ctx 8192 — While qwen2.5-coder:1.5b can handle up to 32k tokens natively, Ollama defaults to a smaller context window when run without a Modelfile in my case, 4096 based on VRAM . I bumped it to 8k, and be memory-efficient on my 8GB machine. temperature 0.2 — lower temperature for more predictable output. For commit messages I don't want creative, I want consistent. ollama create qwen-commit -f ./Modelfile Now I have a model called qwen-commit that I can use for this specific task. By the way, a Modelfile is not the only way to set these. You can use the REST API directly, and pass an options object: curl http://localhost:11434/api/generate -d '{ "model": "qwen2.5-coder:1.5b", "prompt": "${YOUR PROMPT}", "options": { "temperature": 0.2, "num ctx": 8192 } }' For my use case the Modelfile made more sense because I just want to call ollama run qwen-commit and have everything pre-configured. With the bigger context window, the model could now see all 6 files. But it still described the change as "⠙ ⠹ ⠸ ⠼ ⠴ ⠦ ⠧ ⠇ ⠏ ⠋ diff feat .opencode/agent : update tool list for code-reviewer, frontend-enginee frontend-engineer, go-backend-engineer, project-lead, req requirements-analyst, solution-architect ". Better, Mouthful but wrong. The model was reading the full diff now but commit message was technically correct, but nothing like what we would write in a commit message. Look at how it had frontend-enginee frontend-engineer or req requirements-analyst So I changed the prompt. Instead of making the model figure it out, I just told it. bash affected files=$ git diff --staged --name-only | paste -sd, - Then added to the prompt: "Note that the changes are located in these files: $affected files " After this the commit messages got much better. The model didn't have to guess anymore. The commit messages were now accurate but the model kept wrapping them in weird formatting despite the prompt saying not to. Sometimes backticks. Sometimes it prefixed with "diff". Sometimes random quotes around the message. So I added a cleanup step to strip all of that out: bash msg=$ echo "$msg" | tr -d '\r' | sed -E \ -e 's/ diff ?//g' \ -e 's/^diff :space: +//I' \ -e 's/^ :space: +//;s/ :space: +$//' \ -e 's/^ "'\'' //' -e 's/ "'\'' $//' Not elegant but it catches most of the junk the model adds. Till the time I tune the prompt and model this stays I also switched from git diff --staged to git diff --staged --unified=0 . By default, git shows 3 lines of context around each change. For a commit message, the model doesn't need that surrounding context. It just needs to know what changed. --unified=0 strips all that out, which means fewer tokens sent to the model. On a small context window, every token counts. Tada 🎉 php b6f0abc HEAD - main, origin/main, origin/HEAD fix: update tool list for all agents Much bigger code related commit, you can see gradual improvements. php b13f344 HEAD - main fix inspection-workflow : add requirement for editing confirmed vess vessel profile 958053c sh fix app test.go, sqlite.go, sqlite test.go, tasks.md : add save and cancel behaviour tests for vessel profile editing 0f33259 sh fix: update vessel profile form and edit flow in App.svelte, add tests for editing workflow, and improve styles in styles.css, update model in go/mode go/models.ts After all the iterations, my Modelfile looks quite different from where I started: FROM qwen2.5-coder:1.5b PARAMETER num ctx 8192 PARAMETER temperature 0.2 PARAMETER top p 0.7 PARAMETER num predict 256 PARAMETER repeat penalty 1.2 PARAMETER stop "Changes to be committed:" PARAMETER stop "Note:" SYSTEM """ You are an expert developer's assistant. Your sole task is to generate a clean, concise one-line Git commit message based on the provided code diff. Rules: - Respond ONLY with the commit message text. - Do NOT include markdown code blocks, backticks, explanations, intro text, or outro text. - Use the Conventional Commits format e.g., feat scope : message, fix: message . - Keep the one line under 100 characters. - Use the imperative mood "Add feature", not "Added feature" or "Adds feature" . """ What each parameter does and why I added it: temperature 0.2 : controls randomness. Lower means more predictable. I don't want creative commit messages. top p 0.7 : works with temperature. It limits the model to only consider the top 70% most likely next words. Another way to keep the output focused and not wander off. num predict 256 : maximum number of tokens the model can output. A commit message is one line. I don't need the model writing an essay. This caps it. repeat penalty 1.2 : penalizes the model for repeating itself. Without this I was getting things like frontend-enginee frontend-engineer or req requirements-analyst . The model would stutter and repeat parts of words. stop "Changes to be committed:" and stop "Note:" — stop sequences. Sometimes the model would keep going after the commit message and start generating text that looked like git output. These tell the model to stop immediately if it starts outputting these strings. The SYSTEM block is the prompt baked into the model. Every time I run ollama run qwen-commit , this prompt is already there. I don't have to pass it every time. After all the iterations, here is what I ended up with. A custom shell function gac and an alias gacc . It defaults to the local model, but I can also use Claude when I want to. gac { 1. Check for staged changes if git diff --cached --quiet; then echo "❌ Error: No staged changes found. Run 'git add' first." return 1 fi local mode="${1:-qwen}" local msg="" local exit code=0 Gather file names for context local affected files affected files=$ git diff --staged --name-only | paste -sd, - --------------------------------------------------------- IMPROVED PROMPT: Strict rules for Conventional Commits --------------------------------------------------------- local system prompt="You are a strict code assistant. Write a single-line Conventional Commit message for the provided diff. Strict Rules: 1. Format must exactly match: type scope : description 2. Allowed types ONLY: feat, fix, docs, style, refactor, perf, test, chore. 3. The 'scope' must be a single, broad feature/module name e.g., vessel-profile, api . NEVER use file names. 4. The 'description' must summarize the high-level intent in the imperative mood e.g., 'add form validation' . 5. ABSOLUTELY DO NOT list specific file names, paths, or extensions in the commit message. 6. Output EXACTLY one line. No markdown blocks, no quotes, no explanations, and no stray prefixes like 'sh'. Context: The files modified are $affected files ." 2. Execution Routing if "$mode" = "claude" ; then msg=$ git diff --staged --unified=0 | claude -p "$system prompt" --output-format text 2 &1 exit code=$? else if curl -s --max-time 2 http://localhost:11434 /dev/null; then echo "❌ Error: Local Ollama server is not running on port 11434." return 1 fi msg=$ git diff --staged --unified=0 | ollama run qwen-commit "$system prompt" 2 /dev/null exit code=$? fi 3. Robust Error Validation if $exit code -ne 0 || -z "$msg" ; then echo "❌ Error: Failed to generate a response via $mode." echo "Details received: $msg" return 1 fi 4. Strict Text Cleaning Pipeline msg=$ echo "$msg" | tr -d '\r' | sed -E -e 's/ diff ?//g' -e 's/^ :space: +//;s/ :space: +$//' -e 's/^ "'\'' //' -e 's/ "'\'' $//' 5. Run git commit cleanly git commit -m "$msg" } Alias to explicitly force Claude alias gacc="gac claude" No. It still sometimes misses the point of a change. It takes time on larger commits. There is room for improvement. Why not just use Claude directly? That's the easiest thing to do, but it still costs me tokens. And I wanted to learn how local models work. How context windows affect output. How to tune a model for a specific job. That was the whole point for me. It works offline, costs nothing 💰, and I understand every piece because I broke it and fixed it. I find the best way to learn is to find a real use case, however trivial. It helps you understand concepts one thing at a time. Next up: My learnings building a green field product with OpenSpec meant for Brown field projects I welcome all constructive feedback and comments