Kuma Voice is a self-hosted, watch-only voice assistant prototype. The Watch streams microphone audio to a FastAPI backend, which keeps provider credentials off the device and streams a spoken answer back. Optional tools add web search, Notion notes, and Google Calendar access.
This repository does not include a public backend or any credentials. Run your own server and point the Watch app at it.
watchos/KumaVoice.xcodeproj
— watch-only SwiftUI appbackend/app
— FastAPI server and provider integrationsbackend/tests
— tests that use fakes and do not consume API credits
- Xcode 16 or later and an Apple Watch simulator or device
- Python 3.12 or later uv- An OpenAI API key for the realtime voice flow
OpenRouter powers the older REST fallback. Exa, Notion, and Google Calendar are optional.
cd backend
cp .env.example .env
uv sync
Add at least this value to backend/.env
:
OPENAI_API_KEY=your-key-here
Then start the server:
uv run uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
The liveness check is at http://127.0.0.1:8000/health, and local API documentation is at
http://127.0.0.1:8000/docs
- Open
watchos/KumaVoice.xcodeproj
in Xcode. - Select the
KumaVoice
target and choose your development team under Signing & Capabilities. - Start the backend, select an Apple Watch simulator, and press Run.
- Tap the microphone button, or use Double Tap on supported hardware, to start and stop a turn.
The checked-in configuration uses http://127.0.0.1:8000
. For a physical Watch or hosted server, create a private local configuration:
cp watchos/Local.xcconfig.example watchos/Local.xcconfig
Edit watchos/Local.xcconfig
with your own HTTPS backend URL. If the backend has
an API token, put the same value in KUMA_API_TOKEN
. This file is ignored by Git. Local HTTP is allowed for development; remote deployments must use HTTPS.
KUMA_API_TOKEN
is optional only for local development. If it is empty, the API accepts requests without authentication. Always set a long random value when the server is reachable from the internet, then put the same value in the ignored Watch configuration described above.
The token protects every /api/v1
HTTP and WebSocket endpoint. /health
remains public but returns only {"status":"ok"}
. This is lightweight shared- secret protection for a personal prototype, not multi-user authentication.
The included fly.toml
deliberately contains no app name or personal region. Choose your own globally unique name:
KUMA_FLY_APP=your-unique-app-name
fly auth login
fly apps create "$KUMA_FLY_APP"
fly secrets set -a "$KUMA_FLY_APP" OPENAI_API_KEY=... KUMA_API_TOKEN=...
fly deploy -a "$KUMA_FLY_APP"
Set KUMA_BACKEND_URL
in watchos/Local.xcconfig
to your assigned HTTPS URL,
then rebuild the Watch app. Use openssl rand -hex 32
to generate a suitable token and keep it in a password manager.
Provider secrets belong only in backend/.env
locally or in your host's secret
store. The full list and safe placeholders are in backend/.env.example
.
Set EXA_API_KEY
to let the realtime assistant search the web.
- Create an internal integration at notion.so/my-integrations. - Share only the pages Kuma should access with that integration.
- Set
NOTION_API_KEY
and, optionally,NOTION_NOTES_PAGE_ID
.
- Enable the Google Calendar API and create a Desktop OAuth client in the
Google Cloud Console. - Set
GOOGLE_CALENDAR_CLIENT_ID
andGOOGLE_CALENDAR_CLIENT_SECRET
locally. - Run
uv run python -m scripts.google_calendar_auth
frombackend
and save the resultingGOOGLE_CALENDAR_REFRESH_TOKEN
. - Set
GOOGLE_CALENDAR_TIMEZONE
to an IANA zone such asUTC
.
Use a dedicated test calendar and least-privilege Notion pages when evaluating the prototype.
Microphone audio and conversation content are sent to the providers configured on your backend. Web searches, notes, and calendar requests may also be sent to their respective services. Review those providers' retention settings before using personal or sensitive data.
Never commit backend/.env
or watchos/Local.xcconfig
. If a credential is ever exposed, rotate it rather than merely deleting it from Git.
GET /health
— minimal liveness responseWS /api/v1/realtime
— realtime PCM voice conversationPOST /api/v1/transcriptions
— audio file to textPOST /api/v1/responses
— messages to assistant textPOST /api/v1/speech
— text to MP3 audioPOST /api/v1/conversations
— start an in-memory conversationPOST /api/v1/conversations/{id}/turns
— run a complete voice turnDELETE /api/v1/conversations/{id}
— end a conversation
Conversations are kept only in memory and disappear when the backend restarts. Realtime audio is 24 kHz mono, 16-bit PCM.
cd backend
uv run pytest