{"slug": "build-a-token-budgeted-llm-service-on-a-free-server-a-step-by-step-tutorial", "title": "Build a Token-Budgeted LLM Service on a Free Server: A Step-by-Step Tutorial", "summary": "A developer has published a step-by-step tutorial for building a token-budgeted LLM service that runs on a free server. The service, built with Python and the httpx library, tracks token usage and fails loudly when the budget is exceeded, addressing the common problem of unmeasured API usage. The tutorial uses MonkeyCode's free tier, which currently includes 10 million tokens.", "body_md": "Last month, a side project died at the API checkout. The code worked. The credit card did not.\n\nThe fix is not a bigger budget. The fix is a smaller one.\n\nThis tutorial builds a working LLM endpoint from zero. Every step ends with a verification command. You need a terminal, Python 3.11+, and about thirty minutes.\n\nLLM prices keep dropping. My API bills did not. The reason: I never measured usage before adding features.\n\nEveryone is talking about agent memory right now. Token accounting is the boring sibling nobody writes about. This tutorial closes that gap.\n\nThe service you build summarizes incoming text under a hard token budget. It tracks every token it spends. It fails loudly when the budget is exceeded.\n\n`POST /summarize`\n\n`TokenBudget`\n\nclass that estimates, truncates, and tracksFree tokens still have limits. The budget class makes those limits visible.\n\nMonkeyCode offers free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.\n\nAt the time of writing, the free tier includes 10 million tokens. Quotas change. Verify the current numbers in the dashboard before you build on them.\n\nYou need three values:\n\nExport them as environment variables:\n\n```\nexport MONKEYCODE_API_KEY=\"...\"\nexport MONKEYCODE_BASE_URL=\"...\"\nexport MONKEYCODE_MODEL=\"...\"\n```\n\nVerification:\n\n```\ncurl -s \"$MONKEYCODE_BASE_URL/models\" \\\n  -H \"Authorization: Bearer $MONKEYCODE_API_KEY\"\n```\n\nThe exact path lives in the current docs. If the response lists models, you are ready.\n\n```\nmkdir budget-llm && cd budget-llm\npython3 -m venv .venv\nsource .venv/bin/activate\npip install httpx\n```\n\nVerification:\n\n``` python\npython -c \"import httpx; print(httpx.__version__)\"\n```\n\nOne dependency. That keeps the free server deployment boring. Boring is what you want in production.\n\nCreate `budget.py`\n\n. The example assumes an OpenAI-compatible chat endpoint. Confirm the request shape in the current docs before running.\n\n``` python\n# budget.py\nimport os\nimport httpx\n\nclass TokenBudget:\n    def __init__(self, limit: int, base_url: str = \"\", api_key: str = \"\", model: str = \"\"):\n        self.limit = limit\n        self.spent = 0\n        self._base_url = base_url or os.environ[\"MONKEYCODE_BASE_URL\"]\n        self._api_key = api_key or os.environ[\"MONKEYCODE_API_KEY\"]\n        self._model = model or os.environ[\"MONKEYCODE_MODEL\"]\n        self._client = None\n\n    def _get_client(self) -> httpx.Client:\n        if self._client is None:\n            self._client = httpx.Client(\n                base_url=self._base_url,\n                headers={\"Authorization\": f\"Bearer {self._api_key}\"},\n                timeout=30.0,\n            )\n        return self._client\n\n    @staticmethod\n    def estimate(text: str) -> int:\n        # Heuristic: about four characters per token.\n        return max(1, len(text) // 4)\n\n    def fit(self, text: str) -> str:\n        budget = self.limit - self.spent - 100  # reserve room for the reply\n        if budget <= 0:\n            raise RuntimeError(\"Token budget exhausted\")\n        while self.estimate(text) > budget:\n            text = text[: len(text) // 2]\n        return text\n\n    def summarize(self, text: str) -> str:\n        prompt = self.fit(text)\n        response = self._get_client().post(\n            \"/chat/completions\",\n            json={\n                \"model\": self._model,\n                \"messages\": [\n                    {\"role\": \"system\", \"content\": \"Summarize in three sentences.\"},\n                    {\"role\": \"user\", \"content\": prompt},\n                ],\n            },\n        )\n        response.raise_for_status()\n        data = response.json()\n        usage = data.get(\"usage\", {})\n        self.spent += usage.get(\"total_tokens\", self.estimate(prompt))\n        return data[\"choices\"][0][\"message\"][\"content\"]\n```\n\nThe `fit`\n\nmethod is the safety valve. It halves the text until it fits. It never guesses about the reply size.\n\n``` python\npython - <<'PY'\nfrom budget import TokenBudget\n\ntb = TokenBudget(limit=2000)\ntext = open(\"README.md\").read() * 10\nprint(tb.summarize(text))\nprint(\"spent:\", tb.spent)\nPY\n```\n\nVerification: the output is three sentences. The spent value is below 2000. Use a real article for the first real run, not a README.\n\nIf the script raises \"Token budget exhausted\", the truncation path is working. That is a pass, not a failure. A budget you cannot hit is not a budget.\n\nCreate `server.py`\n\nwith the standard library only. No FastAPI. No uvicorn. No extra install step.\n\n``` python\n# server.py\nimport json\nimport os\nfrom http.server import BaseHTTPRequestHandler, HTTPServer\n\nfrom budget import TokenBudget\n\ntb = TokenBudget(limit=int(os.environ.get(\"TOKEN_LIMIT\", \"2000\")))\n\nclass Handler(BaseHTTPRequestHandler):\n    def do_POST(self):\n        length = int(self.headers.get(\"Content-Length\", 0))\n        payload = json.loads(self.rfile.read(length))\n        try:\n            summary = tb.summarize(payload[\"text\"])\n            self.send_response(200)\n            self.end_headers()\n            self.wfile.write(json.dumps(\n                {\"summary\": summary, \"spent\": tb.spent}\n            ).encode())\n        except Exception as exc:\n            self.send_response(429)\n            self.end_headers()\n            self.wfile.write(json.dumps({\"error\": str(exc)}).encode())\n\n    def log_message(self, *args):\n        pass\n\nHTTPServer((\"0.0.0.0\", 8000), Handler).serve_forever()\n```\n\nPush the folder to the free server. The exact deploy command is in the current dashboard. The pattern is always the same: upload the folder, set the environment variables, run `python server.py`\n\n.\n\nVerification:\n\n```\ncurl -s -X POST https://<your-free-server>/ \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"text\": \"Paste a long article here and watch the summary appear.\"}'\n```\n\nExpect JSON with a summary and a spent value. If you get a 429 with \"Token budget exhausted\", the endpoint is alive and honest. The response includes `spent`\n\non purpose. You can graph it later or ignore it now.\n\n``` python\n# verify_budget.py\nfrom budget import TokenBudget\n\ntb = TokenBudget(limit=100)\nlong_text = \"word \" * 10_000\nfitted = tb.fit(long_text)\nassert tb.estimate(fitted) <= 100, \"budget not enforced\"\nassert tb.estimate(fitted) > 0, \"empty prompt\"\nprint(\"budget check passed:\", tb.estimate(fitted), \"tokens\")\n```\n\nRun it:\n\n```\npython verify_budget.py\n```\n\nAdd this file to your repo. Future you will thank present you. This check runs without any network call.\n\n| Situation | Free tier | Paid tier |\n|---|---|---|\n| Weekend prototype | Yes | No |\n| Internal tool, low traffic | Yes | Maybe |\n| Production traffic | No | Yes |\n| Strict data residency | Check first | Check first |\n\nThe free tier is a starting line. It is not a finish line.\n\nFree tiers do not offer SLAs. Plan accordingly.\n\nIf the budget check fails, the tutorial is working as intended. The point is not the free stuff. The point is a repeatable path from idea to deployed endpoint.", "url": "https://wpnews.pro/news/build-a-token-budgeted-llm-service-on-a-free-server-a-step-by-step-tutorial", "canonical_source": "https://dev.to/apprs_6334/build-a-token-budgeted-llm-service-on-a-free-server-a-step-by-step-tutorial-1oo9", "published_at": "2026-08-23 16:46:32+00:00", "updated_at": "2026-08-23 17:13:57.902389+00:00", "lang": "en", "topics": ["large-language-models", "developer-tools", "ai-infrastructure"], "entities": ["MonkeyCode", "Python", "httpx"], "alternates": {"html": "https://wpnews.pro/news/build-a-token-budgeted-llm-service-on-a-free-server-a-step-by-step-tutorial", "markdown": "https://wpnews.pro/news/build-a-token-budgeted-llm-service-on-a-free-server-a-step-by-step-tutorial.md", "text": "https://wpnews.pro/news/build-a-token-budgeted-llm-service-on-a-free-server-a-step-by-step-tutorial.txt", "jsonld": "https://wpnews.pro/news/build-a-token-budgeted-llm-service-on-a-free-server-a-step-by-step-tutorial.jsonld"}}