Deutsch | Español | français | 日本語 | 한국어 | Português | Русский | 中文
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.
A small core, not a bundle. Each backend wraps a best-in-class engine (llama.cpp, vLLM, whisper.cpp, stable-diffusion, MLX...) in its own image, pulled only when a model needs it. You install nothing you don't use.
Composable by design: backends are separate and pulled on demand, so you install only what your model needs** Open and extensible**: load any model, or build your own backend in any language against an open interface** Drop-in API compatibility**: OpenAI, Anthropic, and ElevenLabs APIs across every backend** Any model, any modality**: LLMs, vision, voice, image, and video behind one API** Any hardware**: NVIDIA, AMD, Intel, Apple Silicon, Vulkan, or CPU-only** Multi-user ready**: API key auth, user quotas, role-based access** Built-in AI agents**: autonomous agents with tool use, RAG, MCP, and skills** Privacy-first**: your data never leaves your infrastructure
Created by Ettore Di Giacinto and maintained by the LocalAI team.
[📖 Documentation]|[💬 Discord]|[💻 Quickstart]|[🖼️ Models]|[❓FAQ]
model-fit-canvas-mode.mp4 #
Click to see more! #
usersquota-1775167475876.mp4 #
agents.mp4 #
usage.mp4 #
quantize-fine-tune.mp4 #
talk.mp4 #
Note:The DMG is not signed by Apple. After installing, run:sudo xattr -d com.apple.quarantine /Applications/LocalAI.app
. See[#6268]for details.
Already ran LocalAI before? Use
docker start -i local-ai
to restart an existing container.
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-13
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64-cuda-13
docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblas
docker run -ti --name local-ai -p 8080:8080 --device=/dev/dri/card1 --device=/dev/dri/renderD128 localai/localai:latest-gpu-intel
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-vulkan
local-ai run llama-3.2-1b-instruct:q4_k_m
local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
local-ai run ollama://gemma:2b
local-ai run https://gist.githubusercontent.com/.../phi-2.yaml
local-ai run oci://localai/phi-2:latest
To work with a running LocalAI server from the terminal, start the built-in agent from another shell. It answers questions, reads your files and runs commands on your machine, asking you to approve anything that changes state. Inside a session, /models
lists installed models and /model <name>
switches between them. See the Terminal agent docs.
local-ai run llama-3.2-1b-instruct:q4_k_m
local-ai chat --model llama-3.2-1b-instruct:q4_k_m
Automatic Backend Detection: LocalAI automatically detects your GPU capabilities and downloads the appropriate backend. For advanced options, see[GPU Acceleration].
For more details, see the Getting Started guide.
June 2026: New native biometric backends from the LocalAI team:voice-detect.cppfor speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion) andface-detect.cppfor face detection, recognition, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace). Both are from-scratch C++/ggml engines with no Python or onnxruntime at inference, self-contained GGUF weights, bit-exact parity with the reference, and GPU cuDNN parity, replacing the heavier Pythoninsightface
andspeaker-recognition
backends (PR #10441).June 2026: Newrealtime voice assistant demo(a tiny Go client for the Realtime API with a full talk-back voice loop and tool calling), plusstreaming of the realtime LLM / TTS / transcription pipeline stagesandconfigurable WebRTC ICE candidates.June 2026: Big speech push: theparakeet.cppASR engine gainsNeMo-faithful segment timestamps, amultilingual streaming Nemotron-3.5 model,dynamic batching for concurrent transcriptionandCUDA graphs; the newCrispASR backendadds multi-architecture ASR + TTS, and60 Piper TTS voices across 42 languagesland in the gallery (plusper-request TTS instructions and params).June 2026: New backends and models:locate-anything.cppfor open-vocabulary object detection via ggml,Ideogram4 image generationin stablediffusion-ggml,llama.cpp video input, and theGemma 4 QAT family with MTP speculative-decoding pairs. Plus aninteractive CLI chat modeandRAG source citations in agent responses.June 2026: Distributed mode hardening:prefix-cache-aware routing, aproduction-ready request router with auto-sized embedding/rerank batches,ds4 layer-split distributed inference,NATS JWT auth + TLS/mTLS, andresumable file uploads.May 2026:** LocalAI 4.3.0**-llama.cpp
prompt cache on by default(repeated system prompts collapse from minutes to seconds),keyless cosign signing of backend OCI images,per-API-key + per-user usage attribution, Distributed v3 withper-request replica routing.Release notesMay 2026:** LocalAI 4.2.0**- LocalAI sees and hears:voice recognition,face recognition + antispoofing liveness, speaker diarization. Plusdrop-in Ollama API,video generation, redesigned UI with i18n + admin-configurable branding, vLLM at feature parity with llama.cpp, and 11 new backends.Release notesApril 2026:** LocalAI 4.1.0**- LocalAI becomes a control tower: distributed cluster mode with VRAM-aware smart routing + autoscaling, multi-user platform with OIDC and API keys, per-user quotas with predictive analytics, in-UI fine-tuning with TRL (auto-export to GGUF), on-the-fly quantization backend, visual pipeline editor.Release notesMarch 2026:** LocalAI 4.0.0**- native agentic orchestration with the newAgenthubcommunity hub, full React UI rewrite with Canvas mode,MCP Apps + client-sidewith tool streaming,WebRTC realtime audio,MLX-distributed.Release notesFebruary 2026:Realtime API for audio-to-audio with tool calling,ACE-Step 1.5 supportJanuary 2026:** LocalAI 3.10.0**— Anthropic API support, Open Responses API, video & image generation (LTX-2), unified GPU backends, tool streaming, Moonshine, Pocket-TTS.Release notesDecember 2025:Dynamic Memory Resource reclaimer,Automatic multi-GPU model fitting (llama.cpp),Vibevoice backendNovember 2025:Import models via URL,Multiple chats and historyOctober 2025:Model Context Protocol (MCP)support for agentic capabilities** September 2025**: New Launcher for macOS and Linux, extended backend support for Mac and Nvidia L4T, MLX-Audio, WAN 2.2** August 2025**: MLX, MLX-VLM, Diffusers, llama.cpp now supported on Apple Silicon** July 2025**: All backends migrated outside the main binary —lightweight, modular architecture
For older news and full release notes, see GitHub Releases and the blog.
Text generation(llama.cpp
,transformers
,vllm
...and more)Text to AudioAudio to TextImage generationOpenAI-compatible tools APIRealtime API(Speech-to-speech)Embeddings generationConstrained grammarsDownload models from HuggingfaceVision APIObject DetectionReranker APIP2P InferencingDistributed Mode— Horizontal scaling with PostgreSQL + NATSModel Context Protocol (MCP)Built-in Agents— Autonomous AI agents with tool use, RAG, skills, SSE streaming, andAgent HubBackend Gallery— Install/remove backends on the fly via OCI images- Voice Activity Detection (Silero-VAD)
- Integrated WebUI
LocalAI supports 60+ backends including llama.cpp, vLLM, SGLang, transformers, whisper.cpp, diffusers, MLX, MLX-VLM, and many more. Hardware acceleration is available for NVIDIA (CUDA 12/13), AMD (ROCm), Intel (oneAPI/SYCL), Apple Silicon (Metal), Vulkan, and NVIDIA Jetson (L4T). All backends can be installed on-the-fly from the Backend Gallery.
See the full Backend & Model Compatibility Table and GPU Acceleration guide.
Most backends wrap a best-in-class upstream engine. A handful of them are native C/C++/GGML engines (no Python at inference) developed and maintained by the LocalAI project itself:
| Backend | What it does |
|---|---|
parakeet.cppmoss-transcribe.cppmoss-tts.cppmagpie-tts.cppced.cppvoice-detect.cppvoxtral-tts.cvibevoice.cpprf-detr.cpplocate-anything.cppdepth-anything.cppface-detect.cppfree-splatter.cpptrellis2.cppprivacy-filter.cppLocalVQElocal-storeWe also maintain apex-quant, a per-tensor, per-layer quantization recipe for Mixture-of-Experts models that exploits their structural sparsity to produce GGUFs matching or beating Q8_0 quality - and they run out of the box on stock llama.cpp.
DocumentationLLM fine-tuning guideBuild from sourceKubernetes installationIntegrations & community projectsInstallation video walkthroughBlog: release write-ups, benchmarks and engineering notesExamples— including therealtime voice assistant demo(Go client for the Realtime API with tool calling)
LocalAI is maintained by a small team of humans, together with the wider community of contributors.
— original author and project leadEttore Di Giacinto— maintainerRichard Palethorpe
A huge thank you to everyone who contributes code, reviews PRs, files issues, and helps users in Discord — LocalAI is a community-driven project and wouldn't exist without you. See the full contributors list.
If you utilize this repository, data in a downstream project, please consider citing it with:
@misc{localai,
author = {Ettore Di Giacinto},
title = {LocalAI: The free, Open source OpenAI alternative},
year = {2023},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/go-skynet/LocalAI}},
Do you find LocalAI useful?
Support the project by becoming a backer or sponsor. Your logo will show up here with a link to your website.
A huge thank you to our generous sponsors who support this project covering CI expenses, and our Sponsor list:
A special thanks to individual sponsors, a full list is on GitHub and buymeacoffee. Special shout out to drikster80 for being generous. Thank you everyone!
LocalAI is a community-driven project created by Ettore Di Giacinto and maintained by the LocalAI team.
MIT - Author Ettore Di Giacinto mudler@localai.io
LocalAI couldn't have been built without the help of great software already available from the community. Thank you!
llama.cpphttps://github.com/tatsu-lab/stanford_alpacahttps://github.com/cornelk/llama-gofor the initial ideashttps://github.com/antimatter15/alpaca.cpphttps://github.com/EdVince/Stable-Diffusion-NCNNhttps://github.com/ggerganov/whisper.cpphttps://github.com/rhasspy/piperexofor the MLX distributed auto-parallel sharding implementation
This is a community project, a special thanks to our contributors!