BareMetalRT – TensorRT-LLM running natively on Windows (no WSL)

BareMetalRT launches TensorRT-LLM natively on Windows without WSL, enabling heterogeneous tensor parallelism across consumer GPUs over standard networking. Users can run HuggingFace models locally via PyTorch backend, supporting Llama, Mistral, Mixtral, Qwen 2/3, Gemma 2/3, Phi-3, and DeepSeek.

Linked GPUs Detecting GPU... Loading stats... For the first time ever Heterogeneous tensor parallelism across consumer GPUs over standard networking Network RANK 0 -- Waiting... No model loaded OFFLINE RANK 1 -- Waiting... No model loaded OFFLINE Loading... Bring your own model Paste a HuggingFace model link. It runs on your GPU via the PyTorch backend — no cloud, no engine build. Supported families: Llama, Mistral, Mixtral, Qwen 2/3 incl. MoE , Gemma 2/3, Phi‑3, DeepSeek . Weights must be a HuggingFace checkpoint safetensors in FP16/BF16 or 4‑bit AWQ — GGUF is not supported . Other architectures are rejected when you paste them.Request an integration Missing a tool your business or workflow needs? Tell us which one — we'll build the connector and add it, usually within one business day .