# Show HN: Inference Stack – Everything you need to own your inference

> Source: <https://www.inferencestack.io/>
> Published: 2026-11-22 02:02:13+00:00

**Key takeaway:** Inference Stack v0.1.2 bundles vLLM serving, a LiteLLM gateway with per-request replica routing based on cache and queue data, and Prometheus dashboards into a single Docker Compose install that runs on one machine with NVIDIA GPUs.

### vLLM

Search Hugging Face, check GPU requirements, and deploy from the launcher. vLLM runs the model on your GPUs.

[vLLM docs](https://docs.vllm.ai)

Everything you need to own your inference.

Deploy with one command.

For one machine. Requires Docker with Compose, curl, and OpenSSL. [Setup details](https://www.inferencestack.io/docs#docker-compose)

```
curl -fsSL https://github.com/BerriAI/inference-stack/releases/download/v0.1.2/install.sh | bash -s -- docker
```

Open [localhost:8080](http://localhost:8080) and sign in as `admin` with the generated password. Choose a model in **vLLM**, deploy it, then open **LiteLLM → Playground** to send a prompt.

Real models need NVIDIA GPUs. No GPU? Enable simulation in the model’s Settings for canned responses.

Send requests through one API. Create keys, manage teams and budgets, or try a model in the Playground.

Selects a vLLM replica for each request using cache and queue data. Routing works by default; adjust the weights when you need to.

See model health, request traffic, and response times in the included dashboards. Metrics come from Prometheus.
