LocalAI – Run any model – LLMs, vision, voice, image, video – on any hardware LocalAI, the open-source AI engine created by Ettore Di Giacinto and maintained by the LocalAI team, now runs any model—LLMs, vision, voice, image, and video—on any hardware without requiring a GPU. The platform offers drop-in API compatibility with OpenAI, Anthropic, and ElevenLabs, supports NVIDIA, AMD, Intel, Apple Silicon, Vulkan, and CPU-only systems, and includes built-in AI agents with tool use, RAG, MCP, and skills. LocalAI is designed to be composable, with backends pulled on demand, and privacy-first, ensuring data never leaves user infrastructure. Deutsch https://zdoc.app/de/mudler/LocalAI | Español https://zdoc.app/es/mudler/LocalAI | français https://zdoc.app/fr/mudler/LocalAI | 日本語 https://zdoc.app/ja/mudler/LocalAI | 한국어 https://zdoc.app/ko/mudler/LocalAI | Português https://zdoc.app/pt/mudler/LocalAI | Русский https://zdoc.app/ru/mudler/LocalAI | 中文 https://zdoc.app/zh/mudler/LocalAI LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required. A small core, not a bundle. Each backend wraps a best-in-class engine llama.cpp, vLLM, whisper.cpp, stable-diffusion, MLX... in its own image, pulled only when a model needs it. You install nothing you don't use. Composable by design : backends are separate and pulled on demand, so you install only what your model needs Open and extensible : load any model, or build your own backend in any language against an open interface Drop-in API compatibility : OpenAI, Anthropic, and ElevenLabs APIs across every backend Any model, any modality : LLMs, vision, voice, image, and video behind one API Any hardware : NVIDIA, AMD, Intel, Apple Silicon, Vulkan, or CPU-only Multi-user ready : API key auth, user quotas, role-based access Built-in AI agents : autonomous agents with tool use, RAG, MCP, and skills Privacy-first : your data never leaves your infrastructure Created by Ettore Di Giacinto https://github.com/mudler and maintained by the LocalAI team team . 📖 Documentation | 💬 Discord | 💻 Quickstart | 🖼️ Models | ❓FAQ model-fit-canvas-mode.mp4 Click to see more usersquota-1775167475876.mp4 agents.mp4 usage.mp4 quantize-fine-tune.mp4 talk.mp4 Note:The DMG is not signed by Apple. After installing, run: sudo xattr -d com.apple.quarantine /Applications/LocalAI.app . See 6268 for details. Already ran LocalAI before? Use docker start -i local-ai to restart an existing container. docker run -ti --name local-ai -p 8080:8080 localai/localai:latest CUDA 13 docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-13 CUDA 12 docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12 NVIDIA Jetson ARM64 CUDA 12, for AGX Orin and similar docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64 NVIDIA Jetson ARM64 CUDA 13, for DGX Spark docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64-cuda-13 docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblas docker run -ti --name local-ai -p 8080:8080 --device=/dev/dri/card1 --device=/dev/dri/renderD128 localai/localai:latest-gpu-intel docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-vulkan From the model gallery see available models with local-ai models list or at https://models.localai.io local-ai run llama-3.2-1b-instruct:q4 k m From Huggingface local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8 0.gguf From the Ollama OCI registry local-ai run ollama://gemma:2b From a YAML config local-ai run https://gist.githubusercontent.com/.../phi-2.yaml From a standard OCI registry e.g., Docker Hub local-ai run oci://localai/phi-2:latest To work with a running LocalAI server from the terminal, start the built-in agent from another shell. It answers questions, reads your files and runs commands on your machine, asking you to approve anything that changes state. Inside a session, /models lists installed models and /model