{"slug": "build-a-token-ledger-before-you-burn-through-a-free-model-tier", "title": "Build a Token Ledger Before You Burn Through a Free Model Tier", "summary": "A developer built a stateful token budget guard to prevent free model endpoints from exhausting their allowance during retry loops. The tool checks projected costs before API calls, records actual usage afterward, and refuses to send requests that would exceed the budget. It is designed as a disposable first pass for testing OpenAI-style chat completions on free endpoints like MonkeyCode's.", "body_md": "Disclosure: This article was prepared as part of MonkeyCode's product outreach.\n\nWhy this is worth reading: a free model endpoint with a large token allowance is a good place to validate a new CLI workflow, but it can burn through the allowance in a single retry loop before you notice. I built a small stateful budget guard that checks the projected cost before the call, records actual usage after the call, and refuses to touch the ledger when the endpoint sends an unexpected response. It works as a disposable first pass on a free endpoint and leaves you a clean exit when the shape changes.\n\nMonkeyCode's outreach describes an open-source project with a free model route and a free hosted server. I do not treat either as a permanent dependency. I treat them as a test target: an endpoint I can call without a contract while I am still changing prompts, timeouts, and schemas. The tool below is independent of MonkeyCode's exact model list; it assumes only an OpenAI-style chat completion path and usage accounting in the response. Swap one function if the free server does not follow that shape.\n\nMost model dashboards report aggregate usage after the fact. That is enough for casual work, but it is not enough when you wire an endpoint into a loop. I have seen two avoidable failures in my own drafts. A retry-on-timeout wrapper restarted a slow request four times before the first response arrived, multiplying total token spend. A long context buffer kept sending the same 6k-token history on every turn because I forgot to trim old messages. The dashboard showed the total drop, but not which call caused it.\n\nA local ledger fixes that by refusing to send the request when the projected total exceeds the budget. It does not replace the provider dashboard. It makes the decision before the endpoint gets a chance to consume tokens.\n\nThe script below does three jobs:\n\nPreflight is deliberately rough: prompt bytes divided by four, plus the requested max response tokens, plus a 15 percent margin. That is not tokenizer-accurate for non-English text or code-heavy prompts, but it is intentionally conservative because the goal is to stop accidental waste, not to replace metering. If you need precise preflight numbers, add a local tokenizer for the model you are calling.\n\n```\npython -m venv .venv && source .venv/bin/activate\npip install httpx\n# .env\nMONKEYCODE_BASE_URL=https://your-free-server.example.com/v1\nMONKEYCODE_API_KEY=your-key\nMODEL_NAME=the-current-free-model\nTOKEN_BUDGET=30000000\nMAX_RESPONSE_TOKENS=256\nTIMEOUT_S=30\nLEDGER_PATH=token_ledger.json\n```\n\nThen source it and call:\n\n```\nset -a; source .env; set +a\npython budgeted_call.py 'Summarize this connection error in one sentence.'\n```\n\n`budgeted_call.py`\n\n:\n\n``` python\n#!/usr/bin/env python3\nimport json\nimport os\nimport sys\nimport time\nfrom pathlib import Path\n\nimport httpx\n\nBASE_URL = os.getenv('MONKEYCODE_BASE_URL', '').rstrip('/')\nAPI_KEY = os.getenv('MONKEYCODE_API_KEY', '')\nMODEL = os.getenv('MODEL_NAME', '')\nBUDGET = int(os.getenv('TOKEN_BUDGET', '30000000'))\nLEDGER = Path(os.getenv('LEDGER_PATH', 'token_ledger.json'))\nMAX_TOKENS = int(os.getenv('MAX_RESPONSE_TOKENS', '256'))\nTIMEOUT_S = float(os.getenv('TIMEOUT_S', '30'))\n\ndef load_ledger():\n    if LEDGER.exists():\n        data = json.loads(LEDGER.read_text())\n        return int(data.get('used', 0))\n    return 0\n\ndef save_ledger(used):\n    tmp = LEDGER.with_suffix('.tmp')\n    tmp.write_text(json.dumps({'used': used, 'updated': int(time.time())}, indent=2))\n    tmp.replace(LEDGER)\n\ndef preflight_estimate(prompt, max_tokens):\n    prompt_tokens = len(prompt.encode('utf-8')) // 4\n    return prompt_tokens + max_tokens\n\ndef run(prompt):\n    if not BASE_URL or not API_KEY or not MODEL:\n        sys.exit('Set MONKEYCODE_BASE_URL, MONKEYCODE_API_KEY, and MODEL_NAME first.')\n\n    used = load_ledger()\n    estimate = preflight_estimate(prompt, MAX_TOKENS)\n    margin = int(estimate * 0.15)\n    projected = used + estimate + margin\n\n    if projected > BUDGET:\n        sys.exit(f'Blocked: projected={projected} used={used} budget={BUDGET}. Shorten the prompt or raise the budget.')\n\n    response = httpx.post(\n        f'{BASE_URL}/chat/completions',\n        headers={'Authorization': f'Bearer {API_KEY}'},\n        json={\n            'model': MODEL,\n            'messages': [\n                {'role': 'system', 'content': 'Answer concisely. Return JSON only when asked.'},\n                {'role': 'user', 'content': prompt},\n            ],\n            'max_tokens': MAX_TOKENS,\n        },\n        timeout=TIMEOUT_S,\n    )\n    response.raise_for_status()\n    payload = response.json()\n\n    usage = payload.get('usage') or {}\n    total = usage.get('total_tokens')\n    if total is None:\n        total = int(usage.get('prompt_tokens', 0)) + int(usage.get('completion_tokens', 0))\n\n    if total <= 0:\n        sys.exit('Endpoint returned no usable token count; ledger was not updated.')\n\n    used += total\n    save_ledger(used)\n\n    content = payload['choices'][0]['message']['content']\n    print(json.dumps({\n        'text': content,\n        'total_tokens': total,\n        'used': used,\n        'remaining': BUDGET - used,\n    }, indent=2))\n\nif __name__ == '__main__':\n    run(sys.argv[1] if len(sys.argv) > 1 else 'Reply with the word pong.')\n```\n\nUse a failure fixture that does not hit the real endpoint. The expected result is a non-zero exit and an unchanged ledger.\n\n```\nMONKEYCODE_BASE_URL=http://127.0.0.1:9 python budgeted_call.py 'ping'\n```\n\nIf you want a decision table for a canary suite, keep the checks tiny:\n\n| Scenario | Expected exit | Ledger change |\n|---|---|---|\n| Missing base URL or model | non-zero | none |\n| Unreachable endpoint or timeout | non-zero | none |\n| Projected spend over budget | non-zero | none |\nValid response with `usage.total_tokens`\n|\nzero |\n`used` increases |\n| Response without a usable token count | non-zero | none |\n\nI run this once before I allow any larger script to call the endpoint. A failed run tells me which part of the integration changed instead of leaving me to guess from a balance chart.\n\nFor a solo build, the free server is most useful as a canary target, not as a permanent backend. I point this script at the free route first, keep the model name in an environment variable, and store all results in the local ledger. If the endpoint changes one day, the only change is a URL or model name. If the endpoint reports different usage fields, the script stops instead of silently undercounting.\n\nI also set a hard mental exit: if the free endpoint is slow enough that I need a timeout above 30 seconds, it is not ready for the actual CLI. The ledger cannot fix latency; it only prevents it from getting expensive while I measure.\n\nThe reference I was given describes a free tier with a 30,000,000-token allocation and a free server option. I do not verify quota pages as part of a code article, so I keep the number as `TOKEN_BUDGET`\n\nin an environment variable rather than hard-coding it. Check the current page before you rely on that number; if the allocation is different today, the script does not need to change.\n\nDo not use this local ledger if you need concurrent workers sharing one budget, hard SLOs on latency, audit trails, or compliance review for private data. A single JSON file is not concurrency-safe, and a free endpoint is the wrong home for sensitive prompts. Use the ledger as a canary, not as your production accounting system.\n\nIf you run this against a free server, tell me which usage fields the response actually returned. That determines whether the missing-usage guard is protecting you or getting in your way: `total_tokens`\n\nonly, split `prompt_tokens`\n\nand `completion_tokens`\n\n, or something else entirely.\n\nIf you have a MonkeyCode free server route, plug it into this script first; if not, the same budget guard works with any endpoint that returns usage.", "url": "https://wpnews.pro/news/build-a-token-ledger-before-you-burn-through-a-free-model-tier", "canonical_source": "https://dev.to/rivera123/build-a-token-ledger-before-you-burn-through-a-free-model-tier-1dk", "published_at": "2026-08-15 00:35:52+00:00", "updated_at": "2026-08-15 01:10:49.345849+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools", "artificial-intelligence"], "entities": ["MonkeyCode", "OpenAI"], "alternates": {"html": "https://wpnews.pro/news/build-a-token-ledger-before-you-burn-through-a-free-model-tier", "markdown": "https://wpnews.pro/news/build-a-token-ledger-before-you-burn-through-a-free-model-tier.md", "text": "https://wpnews.pro/news/build-a-token-ledger-before-you-burn-through-a-free-model-tier.txt", "jsonld": "https://wpnews.pro/news/build-a-token-ledger-before-you-burn-through-a-free-model-tier.jsonld"}}