Talk to Jarvis. Work with your agents. Bring your tools, models, and knowledge together.
An open-source workspace for conversations and the work that follows.
Personal Jarvis is an open-source AI ecosystem that runs on your own computer. At its center is Jarvis, a voice orchestrator that connects a conversation to agents, coding sessions, desktop actions, and the services you choose to connect. Speak naturally or type a message: Jarvis can answer, use a tool, or delegate work while you follow the conversation and inspect what happens. The desktop app brings those conversations, your team, and their results into one workspace on Windows, macOS, and Linux.
The workspace extends beyond the conversation. Build persistent specialists with their own instructions and recurring routines. Use Jarvis Voice to dictate into other apps. Connect plugins, skills, and MCP servers, work alongside coding CLIs, keep knowledge in a local Markdown wiki, and open generated reports, pages, and files in Artifacts. These are connected parts of the same application: a request can start with your voice, continue with an agent, and leave behind something you can read, use, or edit.
Choose the models and services that fit your work. Jarvis supports hosted providers, local models, and mixed setups. Your selected provider supplies the intelligence; Jarvis manages application state, tool access, approvals, and execution. Local speech and model options can keep supported work on your hardware. Cloud models and connected services receive the content needed for their requests; running the app locally does not make every integration offline. See privacy and local data and how the system fits together.
Website · Getting started · Documentation · GitHub · YouTube · Discord · X · Instagram
Windows — PowerShell
irm https://raw.githubusercontent.com/PersonalJarvis/PersonalJarvis/main/install/install.ps1 | iex
macOS and Linux
curl -fsSL https://raw.githubusercontent.com/PersonalJarvis/PersonalJarvis/main/install/install.sh | bash
The installer checks Python 3.11+ and Git, offers to install missing prerequisites through the host package manager, installs the applicable desktop components, registers the desktop launcher, and opens the app. Language, wake phrase, and provider setup happen in the app. OS permissions and hardware capabilities affect voice and desktop control. Re-running the installer updates an existing installation.
Personal Jarvis is free and open source. Hosted models, coding subscriptions, and optional services are billed by their respective providers. Supported local models do not require a cloud model account.
Full installation, platform requirements, and uninstall instructions.
Start a Voice Chat, tap the voice bar, or use your configured wake phrase. Your speech and Jarvis's replies appear in the conversation. Start a normal Chat when you prefer a keyboard.
Jarvis brings the workspace into reach: ask an agent to research a topic, work with a coding session, find something in memory, or use a connected tool. Available actions depend on your providers, installed tools, and permissions. Computer use needs a desktop and the required OS permissions.
Choose your voice and model access in the app. The voice path can use realtime audio or a speech-recognition, model, and speech-output pipeline. Your selected provider determines the available capabilities; execution and approvals remain under Jarvis's control.
<sub>“Hey George” → listening → conversation. The desktop window and navigation stay in view.</sub>
The demos on this page are Remotion recreations of the interface with illustrative conversations, not live recordings or response-time benchmarks. The wake phrase and listening transition above are animated to explain the interaction. Click a GIF for its sharper 60 fps video, or use the still previews and reproducible source.
Voice conversations · Wake phrase and audio · Models and providers
- Open Personal Jarvis. The installer opens the desktop app for you. Later, findPersonal Jarvis in Windows Search, macOS Spotlight, or the Linux application menu. You do not need to keep a terminal open.
- Complete the in-app setup. Choose your language, review device permissions, and choose a wake phrase or a keyboard shortcut. The assistant name follows your chosen phrase:Hey George is the example above, not a required name.
- Connect model access. OpenAPI Keys & Providers to connect and test a supported provider, or configure a local model. Chat needs a ready model; voice additionally needs a working realtime connection or speech pipeline.
- Start a conversation. UseNew chat to chooseChat orVoice Chat . For voice, wait for readiness and use your wake phrase, shortcut, or the voice bar. Try:"Help me plan a small project. Ask me what you need to know."
- Build out your workspace. Add a specialist underAgents , connect a service inPlugins / Skills / MCP , or openJarvis Voice for dictation. Find generated files inArtifacts and recurring work underScheduled .
The sidebar keeps these areas within reach. More opens additional views, including knowledge, coding tools, settings, and help. Voice Chat is a conversation with Jarvis; Jarvis Voice is speech-to-text for the app you are already using.
First-run setup · First chat · First voice conversation · Desktop tour
Manual installation and headless use #
git clone https://github.com/PersonalJarvis/PersonalJarvis.git
cd PersonalJarvis
python -m venv .venv
Activate with .\.venv\Scripts\Activate.ps1 on Windows or
source .venv/bin/activate on macOS/Linux, then run:
pip install -e ".[full]"
jarvis # Desktop app
jarvis serve # Headless API and browser UI
For a minimal server installation, use pip install personal-jarvis and
jarvis serve. Open the local address reported at startup; the default is
http://localhost:47821. Remote browser microphone access requires HTTPS.
See the headless deployment guide.
Build a team you can return to. Each agent has an identity, a direct conversation, standing instructions, and access to the tools you grant it. Pick a connected model or supported agent account for the work, and keep the conversations in one workspace.
<sub>An illustrative agent conversation, from brief to draft. The GIF plays once and holds the reply; click to replay the video.</sub>
- Talk directly to a specialist. Select an agent from the roster and continue its chat.
- Give it a standing brief. Configure its instructions, model access, and tools.
- Set up recurring work. Per-agent routines expose instructions, scheduling, and execution history.
- Inspect its work. Read messages and tool activity, and open produced files in Artifacts.
- Explore the world view. The workspace also has a visual map of the team.
Persistent agents and isolated coding missions have different lifecycles. Coding missions can use worktree isolation and critic review; an ordinary agent chat is not a new isolated worktree on every message.
Agent guide · Agent learning · Routines
Speak into the app you already use. Hold your dictation shortcut or toggle hands-free recording, then insert the transcript into the focused text field. Dictation is speech-to-text; Voice Chat is a conversation with Jarvis.
The Jarvis Voice section brings together dictation history, your dictionary, shortcuts, language settings, and speech-provider setup. Optional cleanup improves the transcript, translation writes into a selected language, and Prompt Mode can turn a dictation into a structured prompt. Review and recover entries in history when you need to revisit a transcript.
Choose local speech recognition to process audio on your machine. Provider-backed cleanup or translation can still send text to the configured provider.
Dictation guide · Speech dictionary · Languages and voices
Use local models, hosted providers, or a mixture. Local models helps you discover the local server, see available models, configure their roles, and check readiness. Text generation, tools, and image input have different model requirements.
| Part | Local option |
|---|---|
| Model and tools | Ollama or a compatible local OpenAI-style endpoint. Tool support depends on the model. |
| Speech recognition | On-device Whisper or Nemotron, with the corresponding engine and model installed. |
| Speech output | On-device Piper voices. |
| Realtime conversation | A compatible self-hosted realtime server; experimental, with its own hardware requirements. |
Downloads, hardware needs, languages, and capabilities vary by model. External APIs, hosted coding accounts, connected services, and telephony still send the relevant work to those services.
Local AI setup · Provider setup
Bring supported coding CLIs into Agentic IDE with a project folder and live terminal panes. Work with tools such as Claude Code or Codex using the access supported by that tool, and keep their sessions visible alongside the rest of Jarvis. Terminal and chat views provide different ways to follow the work.
Panes have call signs so you can address a particular session through Jarvis: "Tell T1 to run the tests" or "What is T2 working on?" Return to the workspace to inspect output, respond to a prompt, or take over manually. A terminal becoming idle is not proof that its result is correct; inspect its changes and validation.
Ordinary command-line connections are managed separately from interactive coding panes. CLIs & CLI Test Hub helps discover, configure, and test those tools before you ask Jarvis to use them.
Connect the workspace to the tools you already use, and extend how Jarvis works without replacing the assistant. The Plugins / Skills / MCP hub separates three complementary kinds of extension:
| Extension | What it adds | Learn more |
|---|---|---|
| Plugins | Connections to supported services and their tools, with their own setup and connection state. | Plugin guide |
| Skills | Reusable instructions for a task or workflow, with explicit activation and configuration. | Skill guide |
| MCP servers | Tools supplied by local or remote Model Context Protocol servers, according to granted access. | MCP connections |
Connect the services you need and inspect their status in the app. A tool being listed does not establish that its account is connected or that a particular action will succeed. Authentication requirements and permissions vary by integration. For a self-hosted example, see connecting Home Assistant.
Keep reusable knowledge in the Knowledge Wiki, a local Markdown vault with pages, links, and a visual memory map. It gives facts and notes a place beyond a single conversation, and lets you browse the material that later work can use. You can also connect the vault to Obsidian.
Profile information, contacts, and standing instructions add different kinds of context. Use them to describe preferences and people, and shape the assistant's behavior. They are editable parts of your workspace, so you do not need to repeat the same background in every prompt.
Wiki and memory · Obsidian · Profile and contacts · Instructions and persona
Recurring work belongs in a schedule. Use agent routines for a specialist's recurring brief and the scheduling views to manage tasks and inspect their run history. Instructions, timing, and previous executions stay visible so you can change the work as your needs change.
Workflows and app commands provide additional ways to trigger supported actions. Scheduling depends on the relevant Jarvis runtime being available and the required providers and connections being ready; saving a schedule is not a completed run.
Routines · Tasks and reminders · Workflows and commands
Jarvis can interact with desktop applications through the configured computer-use path. Screen context supplies visual information; computer use goes further and can act through the mouse and keyboard. Desktop access and the required OS permissions are necessary, and actions pass through the configured safety policy.
Optional messaging channels and Twilio calling extend the ways requests and conversations reach the system. These need their own supported accounts and setup. Calling is a hosted-service capability, not an offline feature; the phone-call guide covers numbers, credentials, and webhooks.
Computer use · Screen context · Permissions
A conversation can produce something you keep. Artifacts brings generated reports, documents, pages, images, and other files together for preview and download. Follow the output back to the work that produced it, then open or reuse it outside Jarvis.
Session history and Run Inspector help explain what happened: recorded turns, tool activity, timing, and errors. Spend shows recorded provider usage and available cost information, including supported coding-session usage. Coverage depends on what each provider and execution path reports.
Outputs and files · Sessions and Run Inspector
The desktop is a React/TypeScript interface inside a pywebview window, backed by a Python/FastAPI application. REST endpoints handle application operations; WebSockets deliver live state and activity. Headless mode exposes the API and browser interface without the native desktop shell.
The execution path follows the request and available capabilities. A simple answer can stay in the conversation; a service action needs an available tool; longer work can run through an agent or mission lifecycle. Persistent agent chats and worktree-isolated missions remain distinct.
| Boundary | Technical role |
|---|---|
| Protocols | Shared contracts in jarvis/core/protocols.py separate orchestration from provider and platform implementations. |
| Event bus | Typed, immutable events carry trace IDs so components exchange activity without direct coupling. |
| Streaming providers | Brain, speech, and harness interfaces stream output; realtime sessions handle live audio through their selected provider. |
| Tool executor | A central execution path applies the risk policy and approval requirements before an action runs. |
| Mission lifecycle | Coding missions can use isolated Git worktrees, progress events, critic review, cancellation, and retained output. |
| Extension points | Provider plugins use Python entry points; MCP servers and connected services contribute tools through their adapters. |
| Persistence | Conversations and run records use dedicated stores, the Knowledge Wiki uses local Markdown, and credentials use the secret-storage layer. |
For the current live-voice path, see GPT-Live. For deeper engineering detail, read the architecture overview, architecture decisions, and OS parity.
Configure providers and credentials in the app. Keys use the operating system's
credential store when available, with supported environment/file fallback for
other setups. Keep secrets out of chat, jarvis.toml, and version control.
The configuration example documents advanced settings.
Wake-word listening runs locally. Your speech and model providers determine where subsequent audio, text, and tool context are processed. The Knowledge Wiki stays in local files; integrations receive the information needed for the actions you ask them to perform.
Tool execution uses a risk policy with safe, monitor, ask, and block tiers. Permissions, approvals, and run history let you inspect and control actions.
For scripts and other agents, the CLI reaches the same application API:
jarvis system status
jarvis --json brain status
jarvis api <tag> <op>
CLI guide · Control API · Safety and approvals · Privacy · Security policy
Open Docs in the app for searchable product guides, or follow the topics here.
| Guide | Contents |
|---|---|
| Getting started | Installation, first conversation, and desktop tour. |
| Dictation | Shortcuts, history, cleanup, and speech input. |
| Plugins | Connect services and inspect their status. |
| Skills | Reusable instructions and workflows. |
| MCP | Local and remote tool servers. |
| Local AI | Model setup and capability checks. |
| Troubleshooting | Setup, connection, and device problems. |
| Architecture | Components, data flow, and provider boundaries. |
| GPT-Live | The native live-voice path and tool execution. |
| Agent learning | Private, evidence-backed learning for persistent agents. |
| Routines | Scheduling and recurring work. |
| OS parity | Platform coverage and limitations. |
| Contributor guide | Development setup and pull requests. |
| Architecture decisions | Design decisions and their context. |
| README media source | Provenance, example data, still previews, and rendering commands. |
Follow the project on the website, X, and Instagram.
Questions, ideas, and bug reports are welcome on Discord and GitHub. Watch walkthroughs on the Personal Jarvis channel.
Read CONTRIBUTING.md before a pull request. Hardware reports, provider integrations, accessibility improvements, and native-language feedback are especially useful. AI-assisted contributions are welcome; review focuses on the change and its evidence. Repository contributions are written in English. Report vulnerabilities privately through SECURITY.md.
Thank you to everyone who contributes. This wall is updated from commit history.
Apache 2.0: free to use, modify, and distribute, including commercially. See NOTICE, licensing details, and trademark guidance. Releases through version 1.6.0 retain their original MIT license.