Ten million tokens sounds infinite. It is not. I burned through a large chunk of it in one weekend. The result? A legacy Python codebase with type annotations on every function. No manual edits. No coffee-fueled marathon. Just a batch script and patience.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
A friend inherited a Django project. Eight years old. Forty-three modules. Zero type hints. The team wanted to add mypy to CI. That requires annotations everywhere. Doing it by hand would take days. Hiring someone would cost real money.
I saw a different path. MonkeyCode offers free model access and a free server. The free tier includes a 10-million-token allowance as of this writing. Check the README for current numbers. I decided to use that allowance for a batch job instead of interactive chat.
Interactive chat burns tokens on conversation overhead. Every back-and-forth repeats context. Batch processing is the opposite. You send one prompt per file. You get one response. No chit-chat. No wasted tokens.
The plan was simple. Walk the repository. Find every Python file. Send it to the model with a strict prompt. Save the annotated version. Run the test suite. Review the diff.
Here is the core of what I ran. It is deliberately simple. No frameworks. No queues. Just a loop and a rate limiter.
#!/usr/bin/env python3
import os
import time
import json
import urllib.request
from pathlib import Path
API_KEY = os.environ["API_KEY"]
ENDPOINT = os.environ["ENDPOINT"]
MODEL = os.environ["MODEL"]
PROMPT_TEMPLATE = """
Add type annotations to this Python file.
Keep the logic identical.
Only add annotations to function signatures and variables.
Do not change behavior.
Return the complete file.
python
{code}
"""
def annotate(code: str) -> str:
payload = json.dumps({
"model": MODEL,
"messages": [{"role": "user", "content": PROMPT_TEMPLATE.format(code=code)}],
"temperature": 0,
}).encode()
req = urllib.request.Request(ENDPOINT, data=payload, headers={
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
})
with urllib.request.urlopen(req, timeout=120) as resp:
data = json.loads(resp.read())
return data["choices"][0]["message"]["content"]
def main():
root = Path("project")
files = sorted(root.rglob("*.py"))
files = [f for f in files if "migrations" not in str(f)]
for i, path in enumerate(files):
print(f"[{i+1}/{len(files)}] {path}")
code = path.read_text()
for attempt in range(5):
try:
result = annotate(code)
break
except Exception as e:
print(f" attempt {attempt+1} failed: {e}")
time.sleep(2 ** attempt)
else:
print(f" SKIPPED after 5 attempts")
continue
if result.startswith("```
python"):
result = result.split("
``` python", 1)[1].rsplit("```
", 1)[0]
out_path = path.with_suffix(".annotated.py")
out_path.write_text(result)
time.sleep(1) # polite rate limiting
if __name__ == "__main__":
main()
``` shell
Run it like this:
bash
export API_KEY="your-key"
export ENDPOINT="https://api.monkeycode.example/v1/chat"
export MODEL="model-from-dashboard"
python3 annotate_batch.py
## The Verification Step
A batch job without verification is garbage collection. I did three checks.
First, syntax check every output file.
bash
for f in project/**/*.annotated.py; do
python3 -m py_compile "$f" || echo "FAIL: $f"
done
Second, diff the original against the annotated version. The logic should be identical except for annotations.
bash
diff <(sed 's/: [^=,)]//g' original.py) <(sed 's/: [^=,)]//g' annotated.py)
That sed strips annotations. A clean diff means the model did not change behavior.
Third, run the test suite against the annotated files. I swapped them in one by one and ran pytest after each swap.
## The Numbers
Forty-three files. About 12,000 lines of code. The batch took two hours and eleven minutes. Most of that was rate limiting and retries. The actual API time was under thirty minutes.
Token usage was roughly 1.8 million. That is under a fifth of the free allowance. The cost was zero dollars. My friend estimated the manual work at three full days. The batch job cost me a Saturday morning.
## What Went Wrong
Three files came back with broken syntax. The model added annotations that referenced undefined types. One file had a circular import that mypy would reject. The fix was manual. It took fifteen minutes.
Two files were too large for a single request. The prompt exceeded the context window. I split them by function. That worked.
One model response was truncated mid-file. The retry logic caught it. The second attempt returned the full file.
## When Batch AI Makes Sense
Batch processing shines when the task is mechanical and the output is verifiable. Type annotations. Test generation. Error message rewrites. Docstring extraction. These have clear success criteria. You can check the output automatically.
It fails when the task needs judgment. Refactoring architecture. Renaming concepts. Changing behavior. Do not batch those. You will spend more time reviewing than you saved.
## Limitations
The free server is shared. Do not send proprietary code. The model list changes. Quotas change. Check the README before planning a large run. And remember that a 5% failure rate on 43 files means two or three manual fixes. Budget for that.
## Who Should Skip This
If your codebase is under ten files, just annotate by hand. If your code is confidential, use a local model. If you cannot run a test suite after the batch, do not start. Verification is not optional.
## The Takeaway
Free token allowances are usually wasted on chat. The real value is in batch jobs. A mechanical task that would take days can take hours. The math is simple. Try it on a small module first. See if the output passes your tests. Then scale up.