Free AI servers are dangerously easy to trust. The docs promise zero cost, the setup takes minutes, and your first curl returns a perfect 200. Then your second user shows up, and the server starts answering with 503s or, worse, hangs forever. I've been burned by this pattern enough times that I finally wrote a 30-line load tester to check a free tier before I build anything on it. Here's the script, how to run it, and how to read the numbers without fooling yourself.
The idea is simple: fire a fixed number of concurrent HTTP requests at a health endpoint, record status codes and latencies, and let the results speak. I used asyncio
and aiohttp
because they handle concurrency without spawning a thread per request, which would skew the test on a small machine.
import asyncio
import time
import aiohttp
async def hit(session, url, results):
start = time.perf_counter()
try:
async with session.get(url) as resp:
await resp.read()
status = resp.status
except Exception as exc:
status = type(exc).__name__
results.append((status, time.perf_counter() - start))
async def main(url, concurrency, total):
results = []
async with aiohttp.ClientSession() as session:
tasks = []
for _ in range(total):
tasks.append(hit(session, url, results))
if len(tasks) >= concurrency:
await asyncio.gather(*tasks)
tasks = []
if tasks:
await asyncio.gather(*tasks)
return results
if __name__ == "__main__":
import sys
url = sys.argv[1] if len(sys.argv) > 1 else "https://example.com/health"
concurrency = int(sys.argv[2]) if len(sys.argv) > 2 else 20
total = int(sys.argv[3]) if len(sys.argv) > 3 else 200
results = asyncio.run(main(url, concurrency, total))
ok = [r for r in results if r[0] == 200]
errors = [r for r in results if r[0] != 200]
latencies = [r[1] for r in ok]
print(f"Total: {len(results)}")
print(f"OK: {len(ok)}")
print(f"Errors: {len(errors)}")
if latencies:
print(f"Avg latency: {sum(latencies) / len(latencies):.3f}s")
print(f"Max latency: {max(latencies):.3f}s")
if errors:
print(f"First error type: {errors[0][0]}")
Save it as loadtest.py
, install aiohttp
with pip install aiohttp
, and point it at your endpoint. The script sends requests in batches, so it won't flood the server with 200 simultaneous sockets on a tiny free instance.
Start with a gentle baseline: one request, then ten, then fifty. I usually run this sequence:
python loadtest.py https://your-free-server.example.com/health 1 10
python loadtest.py https://your-free-server.example.com/health 10 100
python loadtest.py https://your-free-server.example.com/health 50 500
The first run tells you the raw latency. The second reveals connection limits. The third shows how the server behaves under sustained pressure. Three numbers matter most:
MonkeyCode is an open-source project that caught my attention because it bundles free model access with a free server option, which is exactly the combination that needs this kind of testing. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I ran the load tester against a small service deployed on their free server, and the process was genuinely useful: it gave me concrete numbers before I wrote any application code, and it exposed a cold-start delay on the first request that I would have missed otherwise.
The free tier is not a production SLA, and I don't treat it as one. But for a prototype or a weekend project, knowing that the server can handle 20 concurrent requests with a 400ms average latency is enough to move forward. The tester doesn't validate model quality or token limits, so I check the current docs for those numbers before committing.
This script only measures HTTP behavior, not model correctness, token throughput, or data privacy. If you're building something that handles sensitive information, a shared free server is the wrong choice regardless of load test results. Also, a single endpoint test won't catch every failure mode: database connections, background workers, and cold starts all hide behind the health check. Run the tester against the actual routes your users will hit, not just /health
.
If you're already on a paid platform with a guaranteed SLA, you don't need this playbook. But if you're evaluating a free tier for a side project, or you're tired of discovering limits after your users do, spend ten minutes with this script first. The numbers will tell you more than any README promise.