Pick a model, pick a GPU, hit deploy. LLMPanel provisions the container, exposes an OpenAI-compatible endpoint, and streams every GPU metric back to one dashboard — on your own hardware or any cloud.
MIT licensed · no GPU lock-in · works with vLLM and Ollama
-
Model
-
meta-llama/Llama-3.3-70B-Instruct
-
Provider
-
RunPod
-
GPU
-
2× H100 80GB
-
Cost
-
$4.38/hr
-
Endpoint
-
llama-70b.gw.llmpanel.io/v1
-
RunPod
-
Vast.ai
-
Lambda
-
CoreWeave
-
Nebius
-
Crusoe
-
Hyperstack
-
FluidStack
-
Paperspace
-
TensorDock
-
RunPod
-
Vast.ai
-
Lambda
-
CoreWeave
-
Nebius
-
Crusoe
-
Hyperstack
-
FluidStack
-
Paperspace
-
TensorDock
Running your own LLMs shouldn't need a platform team #
Self-hosting inference today means stitching together clouds, containers, keys and dashboards. Every piece works — none of it works together.
Scattered tooling
- A different dashboard for every GPU cloud
- Hand-written docker run commands nobody remembers
- One API key shared by every app and teammate
- nvidia-smi in a tmux pane is the monitoring
- No idea what a model actually costs per hour
One control plane for inference
- Docker hosts, RunPod and Vast.ai in one fleet view
- Pick a model, pick a GPU, deploy — vLLM or Ollama
- Scoped keys with rpm / tpm / daily caps and revocation
- Per-GPU utilisation, VRAM and temperature, live
- Cost per hour and uptime on every deployment
One-click deployments
Pick a model from the catalog, choose a GPU, hit deploy. The container, weights and endpoint are handled in minutes.
OpenAI-compatible gateway
Every deployment sits behind one API with managed keys, rate limits and routing. Swap models without touching your app.
Real-time GPU monitoring
Utilisation, VRAM, temperature and tokens per second streamed live to your dashboard for every card.
Multi-provider fleet
Run on your own Docker hosts or burst to RunPod and Vast.ai with your API keys — the same workflow either way.
One endpoint, scoped keys, no surprises
Every deployment sits behind one OpenAI-compatible base URL. Hand each app or environment its own key with rpm, tpm and daily caps — and revoke it in one click when it leaks.
See what every GPU is doing, right now
Utilisation, VRAM, temperature and power draw for every card across Docker hosts and cloud providers, plus container CPU and RAM — live and historical, no exporters to wire up.
Drill into any request
Status, latency, prompt and completion token split, and which key made the call. Filter by status, model, key or time range and open the full payload inline.
Ground a model in your own docs
Attach an agent to any deployment with its own system prompt and knowledge base. Upload files, get chunking and embeddings for free, and test it in the built-in chat before wiring it to Telegram or Discord.
Which GPU do I need for Llama 3.3 70B?
Llama 3.3 70B in FP8 needs roughly 70 GB of VRAM for weights, so a single H100 80GB works for short contexts. For 32k context, deploy on 2× H100 with tensor parallelism — the catalog preset does this for you.
source · docs/sizing-guide.md## Everything between a checkpoint and production
LLMPanel covers the full life of a model in production — so your team stops stitching tools together.
Deploy
Ship any Hugging Face model on vLLM or Ollama to a Docker host or a cloud GPU in one flow.
Route
One OpenAI-compatible base URL in front of every deployment. Point your SDK at it and go.
Monitor
Per-GPU metrics, request logs and usage analytics — live and historical, no exporters to wire up.
Scale
Add a second host or burst to RunPod when demand spikes; the gateway keeps the same URL.
Secure
Scoped API keys with rpm, tpm and daily caps, expiry, per-key usage and one-click revocation.
Back up
Snapshot panel data and config on a schedule, and restore from any point when you need to.
Live in an afternoon #
No Kubernetes, no YAML. Three steps from a bare server to your first OpenAI-compatible endpoint.
Install the panel
One command on any Linux box with Docker. It brings up the API, the dashboard and the gateway, and prints the login URL.
curl -fsSL https://raw.githubusercontent.com/zf0x00/openpanel-oss/master/install.sh | bash
Connect your GPUs
Add a local Docker host, or paste a RunPod / Vast.ai API key. Every card shows up in the fleet view with live metrics.
Deploy and call it
Pick a model, pick a GPU, deploy. Point any OpenAI SDK at the gateway URL with a scoped key — nothing else changes.
from openai import OpenAI
client = OpenAI(
base_url="https://gw.llmpanel.io/v1",
api_key="lp_live_…",
)
client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "user", "content": "hi"}],
)
Works with your stack #
Bring the GPUs, engines and SDKs you already use. LLMPanel sits in the middle without asking you to migrate anything.
- DockerLocal hosts
- RunPodCloud GPUs
- Vast.aiCloud GPUs
- LambdaCloud GPUs
- HyperstackCloud GPUs
- vLLMInference engine
- OllamaInference engine
- Hugging FaceModel catalog
- OpenAI SDKAny language
- TelegramChannel
- DiscordChannel
- WebhooksEvents & alerts
Self-host free, or let us run it #
LLMPanel is open source and free to self-host, forever. Pick a cloud plan when you'd rather not babysit the panel itself.
Free forever. Run the full platform on your own hardware or cloud.
Open source, no strings attached
- Full platform, every feature included
- Unlimited deployments & models
- Bring your own GPUs or cloud keys
- OpenAI-compatible gateway & API keys
- Community support on GitHub
For one developer running a handful of models. We host the panel.
First 50 customers: $9/mo, forever
- 4 deployments included
- $8/mo per extra deployment
- Single seat
- Hosted dashboard — no panel to babysit
- OpenAI-compatible gateway & API keys
For teams sharing a GPU fleet across one workspace.
Extra deployments just $5/mo each
- Everything in Solo
- 10 deployments included
- $5/mo per extra deployment
- Unlimited team members
- Shared workspace with roles & invites
Every plan includes the OpenAI-compatible gateway and per-GPU metrics.
Put your models on your own GPUs today #
Install in five minutes, deploy in one click, and call it from any OpenAI SDK. Free to self-host, forever.