{"slug": "how-i-built-a-self-hosted-family-ai-health-steward-your-health-data-on-your", "title": "How I Built a Self-Hosted Family AI Health Steward (Your Health Data, on Your Shelf)", "summary": "A developer has open-sourced AI Health Steward, a self-hosted AI health manager that reads photos of lab reports using multimodal LLMs, builds structured per-person health profiles, and answers health questions grounded in the user's own data. The system runs entirely on the user's server, with privacy as the core design principle, and supports pluggable model providers including local Ollama for offline use.", "body_md": "TL;DR— I built and open-sourcedAI Health Steward, a self-hosted, private AI health manager for families. It reads photos of lab reports with multimodal LLMs, builds a structured per-person health profile, shows trends on a dashboard, and answers health questions grounded inyour actual data— all running on your own server. Privacy isn't a feature; it's the whole point.[Star it on GitHub].\n\nEvery family has a shoebox — or a folder — of medical reports: blood tests, blood-pressure logs, prescriptions, scan findings. And every \"convenient\" health app wants to hold those records for you. But hold them *where*? On someone else's cloud, to be monetized, analyzed, or lost when the startup pivots.\n\nHealth records are the most sensitive data you own. They shouldn't be a product. They should live on **your** shelf.\n\nSo I built the opposite: a self-hosted AI health steward where the data never leaves your server.\n\n```\n┌──────────────┐     ┌──────────────────────────────────────┐\n│   WebUI      │────▶│            FastAPI backend           │\n│ React + Vite │     │  API / services / providers / prompts │\n└──────────────┘     └───────────────┬──────────────────────┘\n┌──────────────┐                     │\n│ Feishu bot   │────▶  WebSocket ◀───┤\n└──────────────┘                     ▼\n                     ┌──────────────────────────────────────┐\n                     │  PostgreSQL 16 + pgvector (RAG)      │\n                     └──────────────────────────────────────┘\n                     ┌──────────────────────────────────────┐\n                     │  Model providers (pluggable)         │\n                     │  OpenAI-compatible API / Ollama      │\n                     └──────────────────────────────────────┘\n```\n\n`docker compose up -d`\n\nFastAPI gives clean async handlers and Pydantic-validated schemas — perfect for the AI-agent-style tool-calling layer. React + Vite keeps the dashboard snappy. PostgreSQL + pgvector avoids a second vector database — one storage engine for structured data *and* embeddings keeps the deploy story simple for a home server.\n\n**1. The \"person-level profile\" as a single source of truth.**\n\nEvery extracted value carries provenance — which report, which date, confirmed or not. The AI consultation layer reads from this profile rather than re-interpreting raw uploads every time. This is what makes answers *grounded* instead of hallucinated.\n\n**2. Cost is a real concern for a home app.**\n\nI aggressively cut LLM calls: no duplicate metric-extraction calls per conversation, and periodic summaries skip the LLM entirely when a period had no new data. A self-hosted app that costs pennies to run actually gets used.\n\n**3. Structured output + human confirmation.**\n\nAI extraction is powerful but not infallible. Every report goes through *extract → confirm → archive*, so garbage never silently enters the health record.\n\n**4. Privacy as the default posture.**\n\nData stays local. Model calls go only to the provider *you* configure, and you can go fully offline with Ollama. Optional Bearer-token auth + per-member rate limiting protects the instance.\n\nA full dashboard is powerful but heavy for \"hey, is this blood pressure okay?\" So there are **two entrances**:\n\nData flows between both automatically. Same profile, same single source of truth.\n\n```\ngit clone https://github.com/wangzhengpengjay/AI-Health-Steward.git\ncd AI-Health-Steward\ncp .env.example .env        # set MULTIMODAL_API_KEY and TEXT_API_KEY\ncp .env backend/.env\ndocker compose up -d\ndocker exec health-steward-backend alembic upgrade head\n# WebUI: http://localhost:5173   |  API docs: http://localhost:8000/docs\n```\n\nA one-command demo-data seed makes it easy to explore before wiring up real accounts.\n\n**If this resonates, give it a ⭐** — it helps other people who want their health data on their own shelf find it. And PRs and feature suggestions are genuinely welcome.\n\n*AI Health Steward is an open-source project ( MIT). It is not a medical device, does not provide diagnoses, and is not a substitute for professional medical care.*", "url": "https://wpnews.pro/news/how-i-built-a-self-hosted-family-ai-health-steward-your-health-data-on-your", "canonical_source": "https://dev.to/wang_zhengpeng_jay/how-i-built-a-self-hosted-family-ai-health-steward-your-health-data-on-your-shelf-59f8", "published_at": "2026-08-11 03:14:41+00:00", "updated_at": "2026-08-11 03:45:43.168802+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-products", "ai-tools", "developer-tools"], "entities": ["AI Health Steward", "FastAPI", "React", "PostgreSQL", "pgvector", "Ollama", "Docker", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/how-i-built-a-self-hosted-family-ai-health-steward-your-health-data-on-your", "markdown": "https://wpnews.pro/news/how-i-built-a-self-hosted-family-ai-health-steward-your-health-data-on-your.md", "text": "https://wpnews.pro/news/how-i-built-a-self-hosted-family-ai-health-steward-your-health-data-on-your.txt", "jsonld": "https://wpnews.pro/news/how-i-built-a-self-hosted-family-ai-health-steward-your-health-data-on-your.jsonld"}}