LLMPanel Deploy vLLM to RunPod or Vast.ai Without Kubernetes

LLMPanel has launched an open-source platform that deploys large language models on any GPU cloud, including RunPod and Vast.ai, without requiring Kubernetes. The tool provisions containers, exposes OpenAI-compatible endpoints, and streams GPU metrics to a single dashboard, supporting models like meta-llama/Llama-3.3-70B-Instruct on 2× H100 80GB at $4.38/hr. It integrates with vLLM and Ollama and offers features such as scoped API keys, real-time monitoring, and multi-provider fleet management.

Deploy LLMs on any GPU, anywhere Pick a model, pick a GPU, hit deploy. LLMPanel provisions the container, exposes an OpenAI-compatible endpoint, and streams every GPU metric back to one dashboard — on your own hardware or any cloud. MIT licensed · no GPU lock-in · works with vLLM and Ollama - Model - meta-llama/Llama-3.3-70B-Instruct - Provider - RunPod - GPU - 2× H100 80GB - Cost - $4.38/hr - Endpoint - llama-70b.gw.llmpanel.io/v1 - RunPod - Vast.ai - Lambda - CoreWeave - Nebius - Crusoe - Hyperstack - FluidStack - Paperspace - TensorDock - RunPod - Vast.ai - Lambda - CoreWeave - Nebius - Crusoe - Hyperstack - FluidStack - Paperspace - TensorDock Running your own LLMs shouldn't need a platform team Self-hosting inference today means stitching together clouds, containers, keys and dashboards. Every piece works — none of it works together. Scattered tooling - A different dashboard for every GPU cloud - Hand-written docker run commands nobody remembers - One API key shared by every app and teammate - nvidia-smi in a tmux pane is the monitoring - No idea what a model actually costs per hour One control plane for inference - Docker hosts, RunPod and Vast.ai in one fleet view - Pick a model, pick a GPU, deploy — vLLM or Ollama - Scoped keys with rpm / tpm / daily caps and revocation - Per-GPU utilisation, VRAM and temperature, live - Cost per hour and uptime on every deployment One-click deployments Pick a model from the catalog, choose a GPU, hit deploy. The container, weights and endpoint are handled in minutes. OpenAI-compatible gateway Every deployment sits behind one API with managed keys, rate limits and routing. Swap models without touching your app. Real-time GPU monitoring Utilisation, VRAM, temperature and tokens per second streamed live to your dashboard for every card. Multi-provider fleet Run on your own Docker hosts or burst to RunPod and Vast.ai with your API keys — the same workflow either way. One endpoint, scoped keys, no surprises Every deployment sits behind one OpenAI-compatible base URL. Hand each app or environment its own key with rpm, tpm and daily caps — and revoke it in one click when it leaks. Learn more /api-reference See what every GPU is doing, right now Utilisation, VRAM, temperature and power draw for every card across Docker hosts and cloud providers, plus container CPU and RAM — live and historical, no exporters to wire up. Learn more /features Drill into any request Status, latency, prompt and completion token split, and which key made the call. Filter by status, model, key or time range and open the full payload inline. Learn more /features Ground a model in your own docs Attach an agent to any deployment with its own system prompt and knowledge base. Upload files, get chunking and embeddings for free, and test it in the built-in chat before wiring it to Telegram or Discord. Learn more /integrations Which GPU do I need for Llama 3.3 70B? Llama 3.3 70B in FP8 needs roughly 70 GB of VRAM for weights, so a single H100 80GB works for short contexts. For 32k context, deploy on 2× H100 with tensor parallelism — the catalog preset does this for you. source · docs/sizing-guide.md Everything between a checkpoint and production LLMPanel covers the full life of a model in production — so your team stops stitching tools together. Deploy Ship any Hugging Face model on vLLM or Ollama to a Docker host or a cloud GPU in one flow. Route One OpenAI-compatible base URL in front of every deployment. Point your SDK at it and go. Monitor Per-GPU metrics, request logs and usage analytics — live and historical, no exporters to wire up. Scale Add a second host or burst to RunPod when demand spikes; the gateway keeps the same URL. Secure Scoped API keys with rpm, tpm and daily caps, expiry, per-key usage and one-click revocation. Back up Snapshot panel data and config on a schedule, and restore from any point when you need to. Live in an afternoon No Kubernetes, no YAML. Three steps from a bare server to your first OpenAI-compatible endpoint. Install the panel One command on any Linux box with Docker. It brings up the API, the dashboard and the gateway, and prints the login URL. curl -fsSL https://raw.githubusercontent.com/zf0x00/openpanel-oss/master/install.sh | bash Connect your GPUs Add a local Docker host, or paste a RunPod / Vast.ai API key. Every card shows up in the fleet view with live metrics. Deploy and call it Pick a model, pick a GPU, deploy. Point any OpenAI SDK at the gateway URL with a scoped key — nothing else changes. python from openai import OpenAI client = OpenAI base url="https://gw.llmpanel.io/v1", api key="lp live …", client.chat.completions.create model="llama-3.3-70b", messages= {"role": "user", "content": "hi"} , Works with your stack Bring the GPUs, engines and SDKs you already use. LLMPanel sits in the middle without asking you to migrate anything. - DockerLocal hosts - RunPodCloud GPUs - Vast.aiCloud GPUs - LambdaCloud GPUs - HyperstackCloud GPUs - vLLMInference engine - OllamaInference engine - Hugging FaceModel catalog - OpenAI SDKAny language - TelegramChannel - DiscordChannel - WebhooksEvents & alerts Self-host free, or let us run it LLMPanel is open source and free to self-host, forever. Pick a cloud plan when you'd rather not babysit the panel itself. Free forever. Run the full platform on your own hardware or cloud. Open source, no strings attached Deploy yourself https://github.com/zf0x00/openpanel-oss - Full platform, every feature included - Unlimited deployments & models - Bring your own GPUs or cloud keys - OpenAI-compatible gateway & API keys - Community support on GitHub For one developer running a handful of models. We host the panel. First 50 customers: $9/mo, forever Start with Solo https://app.llmpanel.io/login - 4 deployments included - $8/mo per extra deployment - Single seat - Hosted dashboard — no panel to babysit - OpenAI-compatible gateway & API keys For teams sharing a GPU fleet across one workspace. Extra deployments just $5/mo each Start with Team https://app.llmpanel.io/login - Everything in Solo - 10 deployments included - $5/mo per extra deployment - Unlimited team members - Shared workspace with roles & invites Every plan includes the OpenAI-compatible gateway and per-GPU metrics. Put your models on your own GPUs today Install in five minutes, deploy in one click, and call it from any OpenAI SDK. Free to self-host, forever.