Orbit, an Open-Source Toolkit for Retrieval-Based Inference

Orbit, an open-source AI gateway for retrieval-based inference, has been released, enabling self-hosted private RAG, natural-language data access, and tool-calling agents across 37+ model providers. The toolkit supports querying SQL, NoSQL, vector stores, APIs, and files in plain English, with admin controls for keys, quotas, prompts, metrics, and audit logs.

A self-hosted, OpenAI-compatible AI gateway for private RAG, natural-language data access, and tool-calling agents — run it in your own environment across 37+ model providers. Quick Start -quick-start • • -demos Demos • -features Features • /schmitech/orbit/blob/main/docs/tutorial.md Tutorial Docs ORBIT is a self-hosted AI gateway that lets you chat with private data through one OpenAI-compatible endpoint. Put it in front of local or cloud models, connect files, SQL, NoSQL, vector stores, APIs, and tools, then operate everything from an admin panel with keys, quotas, prompts, metrics, and audit logs. Reach for ORBIT when you need to: - 💬 Query SQL, NoSQL, vector stores, REST/GraphQL APIs, Elasticsearch, and files in plain English - 🔀 Route one API contract across local Ollama, llama.cpp, vLLM and cloud models - 🛠️ Give models scoped access to MCP tools and multi-step agent workflows - 🔒 Ship private RAG with auth, quotas, moderation, metrics, and admin controls - 📨 Drive inference asynchronously over a message queue for decoupled, batch-style workloads - 🧪 Prototype locally, then keep the same architecture in production Comparisons: ORBIT vs. Open WebUI · ORBIT vs. LiteLLM multimodal.mp4 Upload private files, ask questions in chat, and keep the whole workflow behind your own gateway. Install the latest stable release, then start ORBIT locally. curl -LO https://github.com/schmitech/orbit/releases/download/v2.9.1/orbit-2.9.1.tar.gz tar -xzf orbit-2.9.1.tar.gz && cd orbit-2.9.1 ./install/setup.sh add --wizard for interactive setup ./bin/orbit.sh start Open the admin panel: | URL | Login | |---|---| | admin / admin123 Verify the gateway: curl -X POST http://localhost:3000/v1/chat \ -H 'Content-Type: application/json' \ -H 'X-API-Key: default-key' \ -H 'X-Session-ID: local-test' \ -d '{"messages": {"role": "user", "content": "Summarize ORBIT in one sentence."} , "stream": false}' Open a browser chat client: ORBIT ADAPTER KEYS='{"simple-chat":"default-key"}' npx orbitchat --open What you should see: the server responds through the OpenAI-compatible /v1/chat endpoint, the admin panel shows live health and adapters, and OrbitChat opens a local chat UI. For a Docker-based local demo with Ollama, use the development compose stack: git clone https://github.com/schmitech/orbit.git cd orbit/docker docker compose up -d The first Docker run can take a few minutes while the local model is downloaded. Warning Docker from main is for local evaluation and development. For production installs, use the latest stable release tarball from GitHub Releases https://github.com/schmitech/orbit/releases . More setup paths: Docker Guide /schmitech/orbit/blob/main/docker/README.md · Tutorial /schmitech/orbit/blob/main/docs/tutorial.md · Windows native install /schmitech/orbit/blob/main/install/windows.md If ORBIT saves you setup time, a GitHub star helps other developers find it. Pick the path closest to what you want to build. | Demo | What it shows | Run it | |---|---|---| Chat with private files | Upload PDFs, spreadsheets, and images, then query them in one thread. | | Ask SQL questions in English Tutorial /schmitech/orbit/blob/main/docs/tutorial/sql-database-sqlite.md Operate the gateway /admin . First chat /schmitech/orbit/blob/main/docs/tutorial/first-chat.md Chat with private files multimodal.mp4 Analysts stop hunting across files: upload PDFs, spreadsheets, and images, then query them together in one thread with context cached across the conversation. Ask SQL questions in English db-query.mp4 No SQL and no ticket queue: ask in plain English, and ORBIT generates the query, runs it against your database, and charts the result in chat. Operate the gateway from the admin panel orbit-admin.mp4 Monitor health, latency, tokens, sessions, adapters, and logs from one dashboard behind API keys, quotas, and rate limits. | Capability | What you get | |---|---| OpenAI-compatible gateway | One /v1/chat interface across local, self-hosted, and cloud providers. | Model routing 37+ providers | Local: Ollama, llama.cpp, vLLM, TensorRT-LLM, Transformers, LM Studio, BitNet. Cloud: OpenAI, Anthropic, Gemini, Bedrock, Vertex, Azure, Groq, Mistral, DeepSeek, xAI, and | Natural-language data access Vector RAG File & multimodal RAG Pluggable file storage File encryption at rest Cloud secrets management .env — selected with one config key. Guide /schmitech/orbit/blob/main/docs/security/secrets-management-setup.md . Web search MCP tool agents A2A peer protocol Google Agent-to-Agent https://google.github.io/A2A/ support — discovery via /.well-known/agent.json and task delegation over JSON-RPC. Guide /schmitech/orbit/blob/main/docs/a2a-protocol.md . Message-queue async surface Guide /schmitech/orbit/blob/main/docs/server.md message-queue-async-protocol . Media generation Voice STT/TTS Production controls Config-first 📚 Deep dive: Docs index /schmitech/orbit/blob/main/docs/README.md · Adapter guide /schmitech/orbit/blob/main/docs/adapters/adapters.md · Configuration /schmitech/orbit/blob/main/docs/configuration.md Same gateway, different jobs. ORBIT is useful anywhere a team needs private data access, controlled model routing, and operational visibility behind one API. | Business outcome | Who feels the pain | What it replaces | |---|---|---| Answers from scattered documents | Insurance, legal, finance, research | Hours spent manually cross-referencing PDFs, spreadsheets, and scans | Self-service data access | Retail, operations, finance, SaaS | BI ticket backlogs and waiting on the data team to write SQL | Faster incident response | SRE, DevOps, security ops | Bottlenecks where only a few specialists can write query DSL | Automated internal workflows | IT ops, support, back-office | Manual glue work across databases, Slack, files, and tickets | Conversational tool use, zero orchestration code | Product, support, SaaS builders | Bolt-on agent frameworks LangChain, AutoGen just to get tool calling | AI on regulated data | Healthcare, government, legal, banking | Compliance blocks on sending sensitive data to cloud LLMs | Governed AI rollout | Any regulated or enterprise org | Shadow AI with no audit trail, cost control, or visibility | In-flow content generation | Marketing, e-learning, communications | Slow, fragmented media-production tooling | Cut incident response time by searching logs in plain English es-logs.mp4 Investigations no longer stall on DSL expertise: ask operational questions naturally, and ORBIT compiles the Elasticsearch Query DSL and returns the answer. Automate multi-step workflows across your internal tools mcp-tool-demo.mp4 Replace manual glue work: give models scoped, server-side access to MCP tools — filesystem, Slack, Postgres, GitHub, Jira — with bounded agent loops. Let the model call tools on its own, like ChatGPT or Claude Beyond explicit, client-requested tool skills, any conversational adapter can opt into opportunistic MCP tool calling : the model decides, turn by turn, whether an external tool is needed and calls it inline — no skill field, no adapter swap, same thread throughout. Multi-step chains, self-correction from tool errors, and mixing several tools in one turn all work out of the box, powered by providers' native function calling with no LangChain, AutoGen, or CrewAI dependency — just a small, bounded, fully-owned server-side loop. See the opportunistic mode guide /schmitech/orbit/blob/main/docs/adapters/mcp-agent.md opportunistic-mode-mcp tools-capability . Keep regulated data in-house by running AI fully offline sensitive-data.mp4 Meet data-residency and compliance rules: run local llama.cpp/Ollama models so sensitive PII never leaves your environment. Generate on-brand media inside the same chat flow puppy.mp4 Produce content without leaving the conversation: generate images and video as cross-adapter skills that carry conversation context. More capabilities svg-rendering.mp4 Render dynamic LLM-generated SVGs inline. second-opinion.mp4 Switch inference models mid-conversation without breaking chat history. business-analytics-demo.mp4 Sub-conversation threading and document caching for faster retrieval. | Client | Description | |---|---| ORBIT Chat | ORBIT ADAPTER KEYS='{"simple-chat":"default-key"}' npx orbitchat Node.js SDK Python client | Topic | Start here | |---|---| Getting started | | Configuration Configuration guide /schmitech/orbit/blob/main/docs/configuration.md · Adapter config /schmitech/orbit/blob/main/docs/adapters/adapter-configuration.md Adapters & RAG Adapters overview /schmitech/orbit/blob/main/docs/adapters/adapters.md · File adapter /schmitech/orbit/blob/main/docs/adapters/file-adapter-guide.md NL data access SQL retriever architecture /schmitech/orbit/blob/main/docs/sql-retriever-architecture.md · Intent SQL RAG /schmitech/orbit/blob/main/docs/intent-sql-rag-system.md MCP agents MCP agent guide /schmitech/orbit/blob/main/docs/adapters/mcp-agent.md Production ops Rate limiting /schmitech/orbit/blob/main/docs/rate-limiting-architecture.md · Fault tolerance /schmitech/orbit/blob/main/docs/fault-tolerance/fault-tolerance-architecture.md Auth & SSO Authentication guide /schmitech/orbit/blob/main/docs/authentication.md — built-in users, Entra ID & Auth0Contributions are welcome — new retrievers, adapters, and provider integrations; better examples and deployment guides; tests, bug fixes, and docs. Start with CONTRIBUTING.md /schmitech/orbit/blob/main/CONTRIBUTING.md , open an issue https://github.com/schmitech/orbit/issues , or send a PR. Roadmap and active work live in GitHub Issues https://github.com/schmitech/orbit/issues . Maintained by Remsy Schmilinsky https://www.linkedin.com/in/remsy/ . ORBIT is licensed under the Apache 2.0 License. See LICENSE /schmitech/orbit/blob/main/LICENSE for details.