{"slug": "one-real-docstring-a-night-a-local-open-weight-model-that-keeps-a-streak-honest", "title": "One Real Docstring a Night: A Local Open-Weight Model That Keeps a Streak Honest", "summary": "A developer built streakkeeper, a nightly GitHub Action that uses a local open-weight model (gemma4:e4b via Ollama) to draft one small, real documentation improvement per night and a second model to verify the draft before committing. The pipeline scans a repository's Python syntax tree for missing or empty docstrings and README sections, rejects drafts with unsupported claims, and refuses to commit if nothing passes review, so no empty or whitespace commits are made. The author reports the tool has committed genuine one-line docstring changes, such as a docstring on a placeholder CLI's main function.", "body_md": "*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady](https://dev.to/challenges/hacktoberfest-week3-2026-10-19)*\n\n**streakkeeper** is a nightly GitHub Action that makes one small, real documentation improvement to one of my repositories, then stops.\n\nIt is scheduled for 22:30 every night. It wakes up, asks GitHub whether I have already contributed today, picks a repository, shallow-clones it, and walks the Python syntax tree for a docstring or README section that is missing or says nothing. A local open-weight model drafts the text. A second model reads the same code and decides whether the draft is true. If it is, the script proves the code did not change, commits one file, and goes back to sleep.\n\nThe effort it saves is not typing. It is the ten minutes every evening when I think \"I should commit something\", open a repo, stare at it, and close it again. That loop is gone, and what replaces it is a change a reader would actually want.\n\nWhat it will not do is manufacture a contribution. No empty commits, no whitespace, no generated files. If nothing passes review, the run ends and nothing is committed. The tool is allowed to do nothing. It is not allowed to invent work.\n\nThe demo is the schedule itself: [the workflow](https://github.com/SammySN-car/streakkeeper/actions/workflows/nightly.yml), and [a commit it made on its own](https://github.com/SammySN-car/jev-deliberation-judge/commit/bda52ac) on a placeholder CLI.\n\n```\nLooking at SammySN-car/jev-deliberation-judge\n  review rejected cli: The proposed text asserts that the implementation was completed, whereas the code only states it is implemented in M5 without indicating completion status.\nCommitted src/deliberation_judge/cli.py to SammySN-car/jev-deliberation-judge\n```\n\nThree lines, the whole loop: it looked, the reviewer threw out the first draft, it tried the next place, and it committed a docstring.\n\nThe commit is one line added and nothing else:\n\n``` php\n def main() -> None:\n+    \"\"\"Raises a SystemExit upon execution. The exit message indicates that the judge CLI is not implemented yet. This message also references milestone M5.\"\"\"\n     raise SystemExit(\"judge CLI not implemented yet (milestone M5)\")\n```\n\nA dry run, from a separate dispatch, stops one step earlier and shows the text before it is written:\n\n```\nLooking at SammySN-car/devpulse\nWould update SammySN-car/devpulse/src/devpulse/__init__.py:1\n  reviewed: The proposed text accurately describes the __version__ assignment without introducing unsupported claims, markdown formatting, or speculative language.\n  --- new text ---\n  Defines the module's version number. The variable __version__ is assigned the string value \"0.1.0\". This variable provides a specific version identifier for the package.\n```\n\nThe part worth reading is the reviewer's sentence. It rules out \"markdown formatting\" and \"speculative language\" by name, because those were the two ways the first drafts went wrong: backticks inside docstrings, and hedged verbs like \"likely\".\n\nMakes one small, real documentation improvement to one of your repositories every night, so a contribution streak survives the days you are too busy to commit.\n\nA placeholder commit keeps the streak alive and makes the history worse. This does the opposite: it finds a docstring or a README section that is missing or says nothing, writes it properly, proves the code did not change, and commits that one file.\n\nEight steps, and only three of them involve a model:\n\n`day.toordinal() % len(repos)`), so no single project takes all the attention.`{\"ok\": true|false, \"reason\": \"...\"}`.` ast.dump`. If anything else differs, refuse.\nThe split is the design. The scan decides *where* to look, so a model can never invent a place to write. The model writes, and a different model judges. Nothing in the pipeline is both author and approver.\n\n**The models.** Ranking and writing run on `gemma4:e4b` through [Ollama](https://ollama.com), inside the GitHub runner: 5.5 GB of model cache, restored in about forty seconds, in a job that takes about five minutes end to end. Review runs on `gemini-3.5-flash` with thinking disabled, falling back to `gemini-3.1-flash-lite`, because a busy reviewer should not be able to block the night. An unclear verdict counts as a rejection, never as approval.\n\n**The proof.** `verify.code_unchanged()` parses both versions, removes every docstring, and compares the syntax trees. Comments are not in the tree, so a comment-only edit passes. A docstring edit passes. Anything else fails and the run exits non-zero. The code cannot change by construction, not by good behaviour.\n\nSeventy-one tests cover the scanner, the replacement logic, the parser for each model's answer, the review gate and the GitHub helpers. None of them need a network or a model.\n\nBecause the part that does the work is open, and I needed to see inside it.\n\nThe writer is an open-weight model running on the same machine as the tool. That is not a purity badge; it is what made the thing debuggable. When `gemma4:e4b` kept wrapping text in backticks that do not belong in a Python docstring, I could read the raw JSON, tighten the prompt, and then strip backticks in code instead of hoping the next prompt would behave. When it hedged - \"likely\", \"suggests\" - I could watch it hedge again on the same input. A closed writer would have handed me a wrong docstring and no way to see why.\n\nIt also costs nothing per night, runs on a free runner, and can be pointed at different models with two variables:\n\n```\nOLLAMA_MODEL=gemma4:e4b                                 # the writer\nREVIEW_MODELS=gemini-3.5-flash,gemini-3.1-flash-lite    # the auditor\n```\n\nThe reviewer is the one closed component, and I want to be straight about that. It is a second opinion, not the worker, and it is replaceable. If no reviewer answers, the run stops and nothing is committed. The writing - the part that makes the project work - is open and local.\n\nAfter setup the tool asks nothing of me: one file, one commit, one push, about five minutes of compute, zero minutes of attention. Of those five minutes, roughly two and a half are the tool itself; the rest is installing Ollama and restoring the model cache.\n\nThe honest accounting is that it has run for real twice. One commit landed. One was reverted, and that is the better story.\n\n**The night it deleted a code block.** The first real run added a docstring to a README section and, in doing so, removed a fenced `bash` block containing `ollama pull llama3.2`. The cause was one line in the scanner:\n\n```\n# roughly how the scanner decided which lines were headings\nheadings = [line for line in lines if line.lstrip().startswith(\"#\")]\n```\n\nA shell comment inside a fenced block - `# Runs on localhost:11434 by default` - is not a heading, but that line said it was. The section was measured as ending at the comment, so its body looked twenty-eight characters long, and the writer replaced the body, fence included. The reviewer approved it, because the text accurately described the command that was about to disappear.\n\nI reverted the commit within minutes. The fix is a fence-aware scan, plus a refusal to replace any section containing a code block, with tests for both. That is the lesson of the week: a tool that does nothing is better than a tool that does damage, and only a real run could have found this. Every dry run was clean.\n\n**The commit that would not have counted.** The first version committed as `streakkeeper <streakkeeper@users.noreply.github.com>`. GitHub credits a commit to your streak only when its author email is associated with your account, so the tool would have worked perfectly and kept no streak at all. A dry run cannot catch that either, because a dry run never commits. It now derives the account's no-reply address from the API and commits as me.\n\n`gemma4:e4b` ranks the candidates and writes the documentation, locally through Ollama. Without it there is no text and no commit.", "url": "https://wpnews.pro/news/one-real-docstring-a-night-a-local-open-weight-model-that-keeps-a-streak-honest", "canonical_source": "https://dev.to/sammysncar/one-real-docstring-a-night-a-local-open-weight-model-that-keeps-a-streak-honest-2j5a", "published_at": "2026-10-25 02:07:16+00:00", "updated_at": "2026-10-25 02:12:10.389116+00:00", "lang": "en", "topics": ["ai-tools", "ai-agents", "large-language-models", "developer-tools"], "entities": ["streakkeeper", "Ollama", "gemma4:e4b", "GitHub Actions", "SammySN-car"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/one-real-docstring-a-night-a-local-open-weight-model-that-keeps-a-streak-honest", "markdown": "https://wpnews.pro/news/one-real-docstring-a-night-a-local-open-weight-model-that-keeps-a-streak-honest.md", "text": "https://wpnews.pro/news/one-real-docstring-a-night-a-local-open-weight-model-that-keeps-a-streak-honest.txt", "jsonld": "https://wpnews.pro/news/one-real-docstring-a-night-a-local-open-weight-model-that-keeps-a-streak-honest.jsonld"}}