# Orbit, an Open-Source Toolkit for Retrieval-Based Inference

> Source: <https://github.com/schmitech/orbit>
> Published: 2026-07-09 22:34:15+00:00

**A self-hosted, OpenAI-compatible AI gateway for private RAG, natural-language data access, and tool-calling agents — run it in your own environment across 37+ model providers.**

[ Quick Start](#-quick-start)
•

[•](#-demos)

**Demos**[•](#-features)

**Features**[•](/schmitech/orbit/blob/main/docs/tutorial.md)

**Tutorial**

**Docs** ORBIT is a self-hosted AI gateway that lets you chat with private data through one OpenAI-compatible endpoint. Put it in front of local or cloud models, connect files, SQL, NoSQL, vector stores, APIs, and tools, then operate everything from an admin panel with keys, quotas, prompts, metrics, and audit logs.

**Reach for ORBIT when you need to:**

- 💬 Query SQL, NoSQL, vector stores, REST/GraphQL APIs, Elasticsearch, and files in plain English
- 🔀 Route one API contract across local (Ollama, llama.cpp, vLLM) and cloud models
- 🛠️ Give models scoped access to MCP tools and multi-step agent workflows
- 🔒 Ship private RAG with auth, quotas, moderation, metrics, and admin controls
- 📨 Drive inference asynchronously over a message queue for decoupled, batch-style workloads
- 🧪 Prototype locally, then keep the same architecture in production

Comparisons:

[ORBIT vs. Open WebUI]·[ORBIT vs. LiteLLM]

## multimodal.mp4

*Upload private files, ask questions in chat, and keep the whole workflow behind your own gateway.*

Install the latest stable release, then start ORBIT locally.

```
curl -LO https://github.com/schmitech/orbit/releases/download/v2.9.1/orbit-2.9.1.tar.gz
tar -xzf orbit-2.9.1.tar.gz && cd orbit-2.9.1
./install/setup.sh        # add --wizard for interactive setup
./bin/orbit.sh start
```

Open the admin panel:

| URL | Login |
|---|---|
|

`admin`

/ `admin123`

Verify the gateway:

```
curl -X POST http://localhost:3000/v1/chat \
  -H 'Content-Type: application/json' \
  -H 'X-API-Key: default-key' \
  -H 'X-Session-ID: local-test' \
  -d '{"messages": [{"role": "user", "content": "Summarize ORBIT in one sentence."}], "stream": false}'
```

Open a browser chat client:

```
ORBIT_ADAPTER_KEYS='{"simple-chat":"default-key"}' npx orbitchat --open
```

What you should see: the server responds through the OpenAI-compatible `/v1/chat`

endpoint, the admin panel shows live health and adapters, and OrbitChat opens a local chat UI.

For a Docker-based local demo with Ollama, use the development compose stack:

```
git clone https://github.com/schmitech/orbit.git
cd orbit/docker
docker compose up -d
```

The first Docker run can take a few minutes while the local model is downloaded.

Warning

Docker from `main`

is for local evaluation and development. For production installs, use the latest stable release tarball from [GitHub Releases](https://github.com/schmitech/orbit/releases).

More setup paths: [Docker Guide](/schmitech/orbit/blob/main/docker/README.md) · [Tutorial](/schmitech/orbit/blob/main/docs/tutorial.md) · [Windows native install](/schmitech/orbit/blob/main/install/windows.md)

If ORBIT saves you setup time, a GitHub star helps other developers find it.

Pick the path closest to what you want to build.

| Demo | What it shows | Run it |
|---|---|---|
Chat with private files |
Upload PDFs, spreadsheets, and images, then query them in one thread. |
|

**Ask SQL questions in English**[Tutorial](/schmitech/orbit/blob/main/docs/tutorial/sql-database-sqlite.md)** Operate the gateway**`/admin`

.[First chat](/schmitech/orbit/blob/main/docs/tutorial/first-chat.md)**Chat with private files**

## multimodal.mp4

*Analysts stop hunting across files: upload PDFs, spreadsheets, and images, then query them together in one thread with context cached across the conversation.*

**Ask SQL questions in English**

## db-query.mp4

*No SQL and no ticket queue: ask in plain English, and ORBIT generates the query, runs it against your database, and charts the result in chat.*

**Operate the gateway from the admin panel**

## orbit-admin.mp4

*Monitor health, latency, tokens, sessions, adapters, and logs from one dashboard behind API keys, quotas, and rate limits.*

| Capability | What you get |
|---|---|
OpenAI-compatible gateway |
One `/v1/chat` interface across local, self-hosted, and cloud providers. |
Model routing (37+ providers) |
Local: Ollama, llama.cpp, vLLM, TensorRT-LLM, Transformers, LM Studio, BitNet. Cloud: OpenAI, Anthropic, Gemini, Bedrock, Vertex, Azure, Groq, Mistral, DeepSeek, xAI, and
|

**Natural-language data access****Vector RAG****File & multimodal RAG****Pluggable file storage****File encryption at rest****Cloud secrets management**`.env`

— selected with one config key. [Guide](/schmitech/orbit/blob/main/docs/security/secrets-management-setup.md).**Web search****MCP tool agents****A2A peer protocol**[Google Agent-to-Agent](https://google.github.io/A2A/)support — discovery via`/.well-known/agent.json`

and task delegation over JSON-RPC. [Guide](/schmitech/orbit/blob/main/docs/a2a-protocol.md).**Message-queue (async) surface**[Guide](/schmitech/orbit/blob/main/docs/server.md#message-queue-async-protocol).** Media generation****Voice (STT/TTS)****Production controls****Config-first**📚 Deep dive: [Docs index](/schmitech/orbit/blob/main/docs/README.md) · [Adapter guide](/schmitech/orbit/blob/main/docs/adapters/adapters.md) · [Configuration](/schmitech/orbit/blob/main/docs/configuration.md)

Same gateway, different jobs. ORBIT is useful anywhere a team needs private data access, controlled model routing, and operational visibility behind one API.

| Business outcome | Who feels the pain | What it replaces |
|---|---|---|
Answers from scattered documents |
Insurance, legal, finance, research | Hours spent manually cross-referencing PDFs, spreadsheets, and scans |
Self-service data access |
Retail, operations, finance, SaaS | BI ticket backlogs and waiting on the data team to write SQL |
Faster incident response |
SRE, DevOps, security ops | Bottlenecks where only a few specialists can write query DSL |
Automated internal workflows |
IT ops, support, back-office | Manual glue work across databases, Slack, files, and tickets |
Conversational tool use, zero orchestration code |
Product, support, SaaS builders | Bolt-on agent frameworks (LangChain, AutoGen) just to get tool calling |
AI on regulated data |
Healthcare, government, legal, banking | Compliance blocks on sending sensitive data to cloud LLMs |
Governed AI rollout |
Any regulated or enterprise org | Shadow AI with no audit trail, cost control, or visibility |
In-flow content generation |
Marketing, e-learning, communications | Slow, fragmented media-production tooling |

**Cut incident response time by searching logs in plain English**

## es-logs.mp4

*Investigations no longer stall on DSL expertise: ask operational questions naturally, and ORBIT compiles the Elasticsearch Query DSL and returns the answer.*

**Automate multi-step workflows across your internal tools**

## mcp-tool-demo.mp4

*Replace manual glue work: give models scoped, server-side access to MCP tools — filesystem, Slack, Postgres, GitHub, Jira — with bounded agent loops.*

**Let the model call tools on its own, like ChatGPT or Claude**

Beyond explicit, client-requested tool skills, any conversational adapter can
opt into **opportunistic MCP tool calling**: the model decides,
turn by turn, whether an external tool is needed and calls it inline — no
`skill`

field, no adapter swap, same thread throughout. Multi-step
chains, self-correction from tool errors, and mixing several tools in one turn
all work out of the box, powered by providers' native function calling with
**no LangChain, AutoGen, or CrewAI dependency** — just a small,
bounded, fully-owned server-side loop. See the
[opportunistic mode guide](/schmitech/orbit/blob/main/docs/adapters/mcp-agent.md#opportunistic-mode-mcp_tools-capability).

**Keep regulated data in-house by running AI fully offline**

## sensitive-data.mp4

*Meet data-residency and compliance rules: run local llama.cpp/Ollama models so sensitive PII never leaves your environment.*

**Generate on-brand media inside the same chat flow**

## puppy.mp4

*Produce content without leaving the conversation: generate images and video as cross-adapter skills that carry conversation context.*

**More capabilities**

## svg-rendering.mp4

*Render dynamic LLM-generated SVGs inline.*

## second-opinion.mp4

*Switch inference models mid-conversation without breaking chat history.*

## business-analytics-demo.mp4

*Sub-conversation threading and document caching for faster retrieval.*

| Client | Description |
|---|---|
ORBIT Chat |

`ORBIT_ADAPTER_KEYS='{"simple-chat":"default-key"}' npx orbitchat`

**Node.js SDK****Python client**| Topic | Start here |
|---|---|
Getting started |
|

**Configuration**[Configuration guide](/schmitech/orbit/blob/main/docs/configuration.md)·[Adapter config](/schmitech/orbit/blob/main/docs/adapters/adapter-configuration.md)**Adapters & RAG**[Adapters overview](/schmitech/orbit/blob/main/docs/adapters/adapters.md)·[File adapter](/schmitech/orbit/blob/main/docs/adapters/file-adapter-guide.md)**NL data access**[SQL retriever architecture](/schmitech/orbit/blob/main/docs/sql-retriever-architecture.md)·[Intent SQL RAG](/schmitech/orbit/blob/main/docs/intent-sql-rag-system.md)**MCP agents**[MCP agent guide](/schmitech/orbit/blob/main/docs/adapters/mcp-agent.md)** Production ops**[Rate limiting](/schmitech/orbit/blob/main/docs/rate-limiting-architecture.md)·[Fault tolerance](/schmitech/orbit/blob/main/docs/fault-tolerance/fault-tolerance-architecture.md)**Auth & SSO**[Authentication guide](/schmitech/orbit/blob/main/docs/authentication.md)— built-in users, Entra ID & Auth0Contributions are welcome — new retrievers, adapters, and provider integrations; better examples and deployment guides; tests, bug fixes, and docs. Start with [CONTRIBUTING.md](/schmitech/orbit/blob/main/CONTRIBUTING.md), open an [issue](https://github.com/schmitech/orbit/issues), or send a PR. Roadmap and active work live in [GitHub Issues](https://github.com/schmitech/orbit/issues).

Maintained by [ Remsy Schmilinsky](https://www.linkedin.com/in/remsy/).

ORBIT is licensed under the Apache 2.0 License. See [LICENSE](/schmitech/orbit/blob/main/LICENSE) for details.