UnlimitedNIM – Proxy that makes Nvidia Nim's 40 RPM feel unlimited

Make NVIDIA's 40 RPM free tier feel unlimited. Smart priority queue proxy for coding agents.

Works with Cline · Roo Code · OpenCode · Any OpenAI-compatible agent

NVIDIA's free NIM tier limits you to 40 requests per minute. Coding agents like Cline, Roo Code, and OpenCode generate bursts of traffic — an agent loop firing tool calls, context reads, and model invocations in parallel will slam straight into that wall. The result: a flood of 429 Rate Limit Exceeded

errors, retried work, frustrated agents, and stalls in the middle of a task.

You don't get more throughput from NVIDIA — but you can stop wasting the window you have.

UnlimitedNIM is a thin, streaming reverse proxy that sits between your coding agent and NVIDIA's API. It enforces NVIDIA's 40 RPM limit for you — in a way that's invisible to your agent:

Sliding-window rate limiter— never fires more than 40 requests per minute, so NVIDIA never sees a burst and never returns 429.** Priority queue**— when the window is full, requests wait in line instead of failing:** priority 1**— user messages (you typed it, you're waiting)** priority 3**— agent loop tool calls (the bulk of traffic)** priority 5**— debug dumps and huge stack traces (fire when there's room)

Full-duplex streaming— SSE responses stream straight through, chunk by chunk.** Starvation protection**— a low-priority request stuck longer than the TTL is promoted so it eventually fires.** Backpressure**— a bounded queue rejects with503 + Retry-After

instead of leaking memory or flooding NVIDIA.

260+ real NVIDIA requests driven through the proxy — zero leaked 429s, zero rejections.

Wave 1 (40 concurrent) fired instantly. Wave 2 (60 more, fired while wave 1 was still in flight) queued and streamed in across the next window — every single request completed, and not one rate-limit error reached a client.

Coding agent ──POST /v1/chat/completions──▶ UnlimitedNIM ──▶ NVIDIA API (40 RPM cap)
                      ▲                          │
                      └──── 200 + SSE stream ◄────┘
                        (or waits in priority queue)

• A request arrives with an X-Priority

header (1/3/5). - If the current 60s window has room, it fires immediately.

• Otherwise it enters the min-heap priority queue — highest priority (lowest number) leaves first; same priority is FIFO.
• As the window rolls over (or requests finish), the queue drains up to 40/min, never exceeding the limit.
• Streaming responses relay byte-for-byte; client disconnects abort the upstream call and free the slot (the window slot is never refunded — NVIDIA already counted it, so we never re-fire).

export NVIDIA_API_KEY=your-key-here

pip install -r requirements.txt

python main.py

Verify it's alive:

curl http://localhost:8000/status

All options come from environment variables or a .env

file (auto-loaded).

Add to opencode.json

(or ~/.config/opencode/opencode.json

):

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "nvidia": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "UnlimitedNIM",
      "options": {
        "baseURL": "http://localhost:8000/v1",
        "apiKey": "anything-works"
      },
      "models": {
        "nvidia/nemotron-3.5-lightning-30b-a3b": {
          "name": "Nemotron 3.5 Lightning 30B"
        }
      }
    }
  }
}

• Settings → API Provider → OpenAI Compatible - Base URL: http://localhost:8000/v1

• API Key: anything (the proxy uses its own NVIDIA key)

• Model: nvidia/nemotron-3.5-lightning-30b-a3b

• Settings → API Configuration → Provider → OpenAI Compatible - Base URL: http://localhost:8000/v1

• API Key: anything

• Model: nvidia/nemotron-3.5-lightning-30b-a3b

load_test.py

simulates heavy agentic traffic (100 concurrent requests, mixed priorities, two waves) and reports status counts, leaked 429s, and queue wait times:

python load_test.py                       # full run (40 + 60 waves)
python load_test.py --total 12 --wave1 5 --wave2 7   # quick smoke

For safe local runs without touching NVIDIA, start the included mock upstream and point the proxy at it:

python mock_upstream.py                    # terminal 1
NVIDIA_BASE_URL=http://localhost:9001/v1 python main.py   # terminal 2
app/
  config.py        # env-driven settings
  rate_limiter.py  # sliding-window limiter + priority queue + in-flight tracking
  proxy.py         # FastAPI app, streaming upstream, 429 retry, validation
main.py            # entry point
load_test.py       # agentic traffic simulator
mock_upstream.py   # fake NVIDIA upstream for local testing
tests/             # 43 tests covering all 34 test-case specs

The suite implements the full 34-case spec — window resets, previous-minute in-flight tracking, priority ordering, starvation promotion, clock skew, upstream 429/500 handling, client disconnects, and the compliance layer (/v1/models

, /status

).

python -m pytest -q   # 43 passed