A self-hosted, OpenAI-compatible AI gateway for private RAG, natural-language data access, and tool-calling agents — run it in your own environment across 37+ model providers.
Quick Start •
•
Demos•
Features•
Tutorial
Docs ORBIT is a self-hosted AI gateway that lets you chat with private data through one OpenAI-compatible endpoint. Put it in front of local or cloud models, connect files, SQL, NoSQL, vector stores, APIs, and tools, then operate everything from an admin panel with keys, quotas, prompts, metrics, and audit logs.
Reach for ORBIT when you need to:
- 💬 Query SQL, NoSQL, vector stores, REST/GraphQL APIs, Elasticsearch, and files in plain English
- 🔀 Route one API contract across local (Ollama, llama.cpp, vLLM) and cloud models
- 🛠️ Give models scoped access to MCP tools and multi-step agent workflows
- 🔒 Ship private RAG with auth, quotas, moderation, metrics, and admin controls
- 📨 Drive inference asynchronously over a message queue for decoupled, batch-style workloads
- 🧪 Prototype locally, then keep the same architecture in production
Comparisons:
[ORBIT vs. Open WebUI]·[ORBIT vs. LiteLLM]
multimodal.mp4 #
Upload private files, ask questions in chat, and keep the whole workflow behind your own gateway.
Install the latest stable release, then start ORBIT locally.
curl -LO https://github.com/schmitech/orbit/releases/download/v2.9.1/orbit-2.9.1.tar.gz
tar -xzf orbit-2.9.1.tar.gz && cd orbit-2.9.1
./install/setup.sh # add --wizard for interactive setup
./bin/orbit.sh start
Open the admin panel:
| URL | Login |
|---|---|
admin
/ admin123
Verify the gateway:
curl -X POST http://localhost:3000/v1/chat \
-H 'Content-Type: application/json' \
-H 'X-API-Key: default-key' \
-H 'X-Session-ID: local-test' \
-d '{"messages": [{"role": "user", "content": "Summarize ORBIT in one sentence."}], "stream": false}'
Open a browser chat client:
ORBIT_ADAPTER_KEYS='{"simple-chat":"default-key"}' npx orbitchat --open
What you should see: the server responds through the OpenAI-compatible /v1/chat
endpoint, the admin panel shows live health and adapters, and OrbitChat opens a local chat UI.
For a Docker-based local demo with Ollama, use the development compose stack:
git clone https://github.com/schmitech/orbit.git
cd orbit/docker
docker compose up -d
The first Docker run can take a few minutes while the local model is downloaded.
Warning
Docker from main
is for local evaluation and development. For production installs, use the latest stable release tarball from GitHub Releases.
More setup paths: Docker Guide · Tutorial · Windows native install
If ORBIT saves you setup time, a GitHub star helps other developers find it.
Pick the path closest to what you want to build.
| Demo | What it shows | Run it |
|---|---|---|
| Chat with private files | ||
| Upload PDFs, spreadsheets, and images, then query them in one thread. | ||
Ask SQL questions in EnglishTutorial** Operate the gateway**/admin
.First chatChat with private files
multimodal.mp4 #
Analysts stop hunting across files: upload PDFs, spreadsheets, and images, then query them together in one thread with context cached across the conversation.
Ask SQL questions in English
db-query.mp4 #
No SQL and no ticket queue: ask in plain English, and ORBIT generates the query, runs it against your database, and charts the result in chat.
Operate the gateway from the admin panel
orbit-admin.mp4 #
Monitor health, latency, tokens, sessions, adapters, and logs from one dashboard behind API keys, quotas, and rate limits.
| Capability | What you get |
|---|---|
| OpenAI-compatible gateway | |
One /v1/chat interface across local, self-hosted, and cloud providers. |
|
| Model routing (37+ providers) | |
| Local: Ollama, llama.cpp, vLLM, TensorRT-LLM, Transformers, LM Studio, BitNet. Cloud: OpenAI, Anthropic, Gemini, Bedrock, Vertex, Azure, Groq, Mistral, DeepSeek, xAI, and | |
Natural-language data accessVector RAGFile & multimodal RAGPluggable file storageFile encryption at rest****Cloud secrets management.env
— selected with one config key. Guide.Web searchMCP tool agentsA2A peer protocolGoogle Agent-to-Agentsupport — discovery via/.well-known/agent.json
and task delegation over JSON-RPC. Guide.Message-queue (async) surfaceGuide.** Media generation****Voice (STT/TTS)Production controlsConfig-first**📚 Deep dive: Docs index · Adapter guide · Configuration
Same gateway, different jobs. ORBIT is useful anywhere a team needs private data access, controlled model routing, and operational visibility behind one API.
| Business outcome | Who feels the pain | What it replaces |
|---|---|---|
| Answers from scattered documents | ||
| Insurance, legal, finance, research | Hours spent manually cross-referencing PDFs, spreadsheets, and scans | |
| Self-service data access | ||
| Retail, operations, finance, SaaS | BI ticket backlogs and waiting on the data team to write SQL | |
| Faster incident response | ||
| SRE, DevOps, security ops | Bottlenecks where only a few specialists can write query DSL | |
| Automated internal workflows | ||
| IT ops, support, back-office | Manual glue work across databases, Slack, files, and tickets | |
| Conversational tool use, zero orchestration code | ||
| Product, support, SaaS builders | Bolt-on agent frameworks (LangChain, AutoGen) just to get tool calling | |
| AI on regulated data | ||
| Healthcare, government, legal, banking | Compliance blocks on sending sensitive data to cloud LLMs | |
| Governed AI rollout | ||
| Any regulated or enterprise org | Shadow AI with no audit trail, cost control, or visibility | |
| In-flow content generation | ||
| Marketing, e-learning, communications | Slow, fragmented media-production tooling |
Cut incident response time by searching logs in plain English
es-logs.mp4 #
Investigations no longer stall on DSL expertise: ask operational questions naturally, and ORBIT compiles the Elasticsearch Query DSL and returns the answer.
Automate multi-step workflows across your internal tools
mcp-tool-demo.mp4 #
Replace manual glue work: give models scoped, server-side access to MCP tools — filesystem, Slack, Postgres, GitHub, Jira — with bounded agent loops.
Let the model call tools on its own, like ChatGPT or Claude
Beyond explicit, client-requested tool skills, any conversational adapter can
opt into opportunistic MCP tool calling: the model decides,
turn by turn, whether an external tool is needed and calls it inline — no
skill
field, no adapter swap, same thread throughout. Multi-step chains, self-correction from tool errors, and mixing several tools in one turn all work out of the box, powered by providers' native function calling with no LangChain, AutoGen, or CrewAI dependency — just a small, bounded, fully-owned server-side loop. See the opportunistic mode guide.
Keep regulated data in-house by running AI fully offline
sensitive-data.mp4 #
Meet data-residency and compliance rules: run local llama.cpp/Ollama models so sensitive PII never leaves your environment.
Generate on-brand media inside the same chat flow
puppy.mp4 #
Produce content without leaving the conversation: generate images and video as cross-adapter skills that carry conversation context.
More capabilities
svg-rendering.mp4 #
Render dynamic LLM-generated SVGs inline.
second-opinion.mp4 #
Switch inference models mid-conversation without breaking chat history.
business-analytics-demo.mp4 #
Sub-conversation threading and document caching for faster retrieval.
| Client | Description |
|---|---|
| ORBIT Chat |
ORBIT_ADAPTER_KEYS='{"simple-chat":"default-key"}' npx orbitchat
Node.js SDK****Python client| Topic | Start here | |---|---| Getting started | |
ConfigurationConfiguration guide·Adapter configAdapters & RAGAdapters overview·File adapterNL data accessSQL retriever architecture·Intent SQL RAGMCP agentsMCP agent guide** Production ops**Rate limiting·Fault toleranceAuth & SSOAuthentication guide— built-in users, Entra ID & Auth0Contributions are welcome — new retrievers, adapters, and provider integrations; better examples and deployment guides; tests, bug fixes, and docs. Start with CONTRIBUTING.md, open an issue, or send a PR. Roadmap and active work live in GitHub Issues.
Maintained by Remsy Schmilinsky.
ORBIT is licensed under the Apache 2.0 License. See LICENSE for details.