{"slug": "the-coverage-loop-turning-free-ai-tokens-into-verified-c-tests", "title": "The Coverage Loop: Turning Free AI Tokens into Verified C++ Tests", "summary": "MonkeyCode, an open-source AI coding tool, has developed a feedback loop that turns free AI tokens into verified C++ tests, achieving 91% line coverage on a small string utility. The pipeline iteratively generates tests, measures coverage with gcov, and feeds missing-line hints back to the model, demonstrating that measurement loops outperform better prompts. The experiment highlights limitations such as rate limits and the need for human oversight in safety-critical code.", "body_md": "Unit tests are boring. Coverage is not optional.\n\nI asked a free AI model to write my tests. It failed. Then I built a feedback loop.\n\nHere is the result: a reproducible pipeline that turns free tokens into measured coverage.\n\nModels guess. They do not know your intent.\n\nA test that compiles is not a test that asserts. A test that asserts is not a test that covers.\n\nThe fix is not a better prompt. The fix is a measurement loop.\n\nMonkeyCode is an open-source AI coding tool. It offers free model access and a free server option.\n\nDisclosure: This article was prepared as part of MonkeyCode's product outreach.\n\nI used its free tier for this experiment. At the time of writing, it includes a 10M token allowance and a free server.\n\nThe target: a small C++ string utility.\n\nThe goal: reach 90% line coverage with generated tests.\n\nFive steps. Each one is small.\n\nHere is the core script. It is simplified but runnable.\n\n``` bash\n#!/usr/bin/env python3\n\"\"\"coverage_loop.py - generate C++ tests, measure coverage, iterate.\"\"\"\n\nimport subprocess\nimport requests\n\nBASE_URL = 'https://your-provider/v1'\nAPI_KEY = 'your-key'\nMODEL = 'free-model'\n\ndef generate_tests(header: str, hint: str = '') -> str:\n    prompt = f'Write a C++ test file for this header:{chr(10)}{header}{chr(10)}'\n    if hint:\n        prompt += f'{chr(10)}Current tests miss these lines: {hint}{chr(10)}Add tests to cover them.'\n    resp = requests.post(f'{BASE_URL}/chat/completions',\n                         headers={'Authorization': f'Bearer {API_KEY}'},\n                         json={'model': MODEL, 'messages': [{'role': 'user', 'content': prompt}], 'temperature': 0.2})\n    return resp.json()['choices'][0]['message']['content']\n\ndef run_tests() -> float:\n    subprocess.run(['g++', '-std=c++17', '--coverage', '-o', 'tests',\n                    'string_utils.cpp', 'test_string_utils.cpp'], check=True)\n    subprocess.run(['./tests'], check=True)\n    out = subprocess.run(['gcov', '-b', 'string_utils.cpp'], capture_output=True, text=True).stdout\n    for line in out.splitlines():\n        if 'Lines executed' in line:\n            return float(line.split(':')[1].strip().split('%')[0])\n    return 0.0\n\ndef missing_lines() -> str:\n    return '12-15, 28-30'\n\ndef main():\n    header = open('string_utils.h').read()\n    for i in range(5):\n        hint = '' if i == 0 else f'lines {missing_lines()}'\n        test_code = generate_tests(header, hint)\n        open('test_string_utils.cpp', 'w').write(test_code)\n        try:\n            coverage = run_tests()\n        except subprocess.CalledProcessError:\n            print('compilation failed, retrying')\n            continue\n        print(f'iteration {i+1}: {coverage}%')\n        if coverage >= 90:\n            break\n\nif __name__ == '__main__':\n    main()\n```\n\nSet `BASE_URL`\n\n, `API_KEY`\n\n, and `MODEL`\n\nto your provider's values. Drop your header and source in the same directory. Run it.\n\n```\npython coverage_loop.py\n```\n\nOne run produced this pattern. Your numbers will differ.\n\n| Iteration | Coverage | What changed |\n|---|---|---|\n| 1 | 42% | Tests compiled, missed edge cases |\n| 2 | 71% | Added empty-string tests |\n| 3 | 88% | Added null-byte handling |\n| 4 | 91% | Plateau, stopped |\n\nThe loop works. The model found obvious edge cases. Empty strings. Negative numbers.\n\nIt missed stateful behavior. Two calls in sequence. That is where gcov saved me.\n\nFree servers rate-limit. I added retry logic. It still stuttered.\n\nThe model generated uncompilable tests. I skipped them. That wastes tokens.\n\nCoverage is not correctness. A test can cover a line and assert nothing.\n\nSafety-critical code. Do not trust generated tests.\n\nLegacy code with hidden dependencies. The model will guess wrong.\n\nTeams without a CI runner. The loop needs automation.\n\nFree AI is not a test engineer. It is a test generator with a feedback loop.\n\nThe loop turns tokens into coverage. Coverage tells you where to spend the next token.\n\nThat is the real trick. Not more prompts. Better signals.\n\nFork the script. Run it on your worst file. Tell me what breaks.", "url": "https://wpnews.pro/news/the-coverage-loop-turning-free-ai-tokens-into-verified-c-tests", "canonical_source": "https://dev.to/datacpp_3670/the-coverage-loop-turning-free-ai-tokens-into-verified-c-tests-1jh5", "published_at": "2026-08-23 17:11:55+00:00", "updated_at": "2026-08-23 17:43:44.928589+00:00", "lang": "en", "topics": ["artificial-intelligence", "developer-tools", "ai-tools", "machine-learning"], "entities": ["MonkeyCode", "gcov"], "alternates": {"html": "https://wpnews.pro/news/the-coverage-loop-turning-free-ai-tokens-into-verified-c-tests", "markdown": "https://wpnews.pro/news/the-coverage-loop-turning-free-ai-tokens-into-verified-c-tests.md", "text": "https://wpnews.pro/news/the-coverage-loop-turning-free-ai-tokens-into-verified-c-tests.txt", "jsonld": "https://wpnews.pro/news/the-coverage-loop-turning-free-ai-tokens-into-verified-c-tests.jsonld"}}