{"slug": "show-hn-lia-a-self-hosted-multi-agent-ai-assistant-fastapi-and-langgraph", "title": "Show HN: LIA – a self-hosted multi-agent AI assistant (FastAPI and LangGraph)", "summary": "LIA, a self-hosted multi-agent AI assistant built with FastAPI and LangGraph, has been released as version 1.25.5. The assistant features LangGraph orchestration, human-in-the-loop controls, enterprise observability, and full i18n support across six languages, addressing issues like unpredictable costs and hallucinations.", "body_md": "**Intelligent multi-agent conversational assistant with LangGraph orchestration, Human-in-the-Loop, enterprise-grade observability, and full i18n support (6 languages)**\n\n**If you find my project and work valuable, I would be grateful for a star on GitHub. Thank you !**\n\n[Features](#features) •\n[Admin & Monitoring](#administration--monitoring) •\n[Quick Start](#quick-start) •\n[Architecture](#architecture) •\n[Documentation](#documentation) •\n[Contributing](#contributing)\n\n**Version 1.25.5** — **The phone assistant becomes trustworthy — and the planner learns to converge.** A full day of live-call debugging turns agentic telephony from a promising prototype into a disciplined agent. **On the call:** an instant localized greeting at pickup (an empty first message caused a multi-second silent standoff), a `current_datetime`\n\nanchor so \"tomorrow\" resolves to the right weekday out loud, a pinned multilingual voice over telephony-native `ulaw_8000`\n\naudio, a **pinned thinking-free LLM** (the platform default was caught reciting its English reasoning aloud mid-call), and a real hang-up via the `end_call`\n\nsystem tool. Above all, a **mandate boundary**: the assistant never accepts an unrequested expense or commitment — even a 3€ topping — it captures the offer and its price, defers to the user, and the post-call summary must state every cost and flag every open point for a call-back. **Around the call:** the vendor agent re-syncs lazily on config drift (no more deactivate/reactivate), a stuck \"call already active\" row self-heals by probing the vendor (including the deleted-agent 404 case, grace-window protected), and vendor errors are diagnosable in one log line. **In the pipeline:** the post-call follow-up \"crée le rdv\" kept producing a next-hour default slot instead of the agreed Saturday 9:30 — resolved facts now ride in the planner's *human message* where models actually read them; the semantic validator finally validates **single-step mutations** (skipped as \"trivial\" before — the whole class was unguarded), survives a structured-output prompt conflict with an A/B-proven fix (0/3 → 6/6 tool calls) plus a one-shot retry, rejects fabricated placeholder emails deterministically (RFC 2606), and on replan the planner is finally shown **its own previous plan** with a fix-don't-rebuild directive (it used to oscillate — fixing the date on one pass, losing it on the next). Final net: an invalid *mutation* plan that exhausts its replans is never silently executed — LIA asks the user instead. **Verified:** **10,344 fast unit** tests green, mypy strict clean on 900 files, every fix reproduced then re-proven against live prod logs and real calls. — 17 July 2026.\n\n[Why LIA?](#why-lia)[Try LIA Online](#try-lia-online)[Built by an AI, Directed by a Human](#built-by-an-ai-directed-by-a-human)[Screenshots](#screenshots)[Features](#features)[Administration & Monitoring](#administration--monitoring)[Quick Start](#quick-start)[Architecture](#architecture)[Technologies](#technologies)[Documentation](#documentation)[Tests](#tests)[CI/CD](#cicd)[Performance](#performance)[Security](#security)[Contributing](#contributing)[Support](#support)[License](#license)[Acknowledgments](#acknowledgments)\n\n**LIA** solves the fundamental problems of today's AI assistants:\n\n| Problem | LIA Solution |\n|---|---|\nUnpredictable LLM costs |\nReal-time token tracking, budget alerts, 93% optimization |\nUncontrolled hallucinations |\nHuman-in-the-Loop (HITL) with 6 approval levels |\nFragmented integrations |\nUnified multi-domain orchestration (19+ agents + MCP + sub-agents) |\nLimited observability |\n419 Prometheus metrics, 25 Grafana dashboards, email alerting with runbooks, GeoIP analytics |\nInconsistent performance |\nGemini embedding-001 with asymmetric task types, semantic routing with hybrid scoring |\n\n```\n📅 \"Find my meetings for tomorrow and send a reminder to all participants\"\n📧 \"Summarize my unread emails from this week that have attachments\"\n👥 \"Update the companies of my contacts who work at startups\"\n🔔 \"Remind me tomorrow at 9am to call Marie for her birthday\"\n```\n\nLIA is available as a hosted service at ** https://lia.jeyswork.com/** — no installation required.\n\nClosed beta: Access is currently limited to a restricted number of users, at the administrator's discretion. To request an invitation, contact.[liamyassistant@gmail.com]\n\n\"Speed comes from the AI. Quality comes from the framework.\"\n\nNearly **100% of this codebase was written by an AI**, under human direction: a written\nengineering rulebook, blocking automated checks, systematic review, adversarial audits.\nThe result is measured, not proclaimed:\n\n32 functional domains |\n420,000 lines of code (excl. tests) |\n11,900+ automated tests |\n120+ ADRs |\n153 versions shipped |\n6 languages, parity enforced in CI |\n419 Prometheus metrics |\n8.3/10 technical audit, 24 normalized areas |\n\n**The full story**— method, trade-offs, results and what remains to be done, weaknesses included:[lia.jeyswork.com/story](https://lia.jeyswork.com/story)**The audit itself**— 24 normalized areas mapped to ISO/IEC 25010:2023, every score backed by executed evidence, 7 open worksites included, with the protocol and the full standalone report:[docs/audit/](/jgouviergmail/LIA-Assistant/blob/main/docs/audit/README.md)\n\n[\n](/jgouviergmail/LIA-Assistant/blob/main/docs/assets/screenshot-homepage.png)*Dashboard — Homepage with quick access, usage statistics, and personalized greeting*\n\n[\n](/jgouviergmail/LIA-Assistant/blob/main/docs/assets/screenshot-chat.png)*Chat — Multi-agent conversation with real-time debug panel (right sidebar)*\n\n**More screenshots**\n\n[\n](/jgouviergmail/LIA-Assistant/blob/main/docs/assets/screenshot-chat-debug-panel.png)*Chat — Debug panel: per-message routing, tool calls, token cost and reasoning timeline*\n\n[\n](/jgouviergmail/LIA-Assistant/blob/main/docs/assets/screenshot-chat-interactive-skills.png)*Chat — Interactive skill widgets: maps, dashboards, calendars and mini-apps rendered inline*\n\n[\n](/jgouviergmail/LIA-Assistant/blob/main/docs/assets/screenshot-settings-preferences.png)*Settings — Preferences: connectors, MCP servers, language, timezone, and themes*\n\n[\n](/jgouviergmail/LIA-Assistant/blob/main/docs/assets/screenshot-settings-features.png)*Settings — Features: LIA Style, long-term memory, interests, proactive notifications, scheduled actions, sub-agents, channels*\n\n[\n](/jgouviergmail/LIA-Assistant/blob/main/docs/assets/screenshot-settings-features-memory.png)*Settings — Long-term memory: pinned facts, automatic extraction, edit / delete / pin per memory*\n\n[\n](/jgouviergmail/LIA-Assistant/blob/main/docs/assets/screenshot-settings-features-psyche.png)*Settings — Psyche Engine: Big Five personality traits modulating the assistant's emotional responsiveness*\n\n[\n](/jgouviergmail/LIA-Assistant/blob/main/docs/assets/screenshot-settings-administration.png)*Settings — Administration: LLM config, RAG Spaces, users, connectors, pricing, skills, voice, broadcast, debug*\n\n[\n](/jgouviergmail/LIA-Assistant/blob/main/docs/assets/screenshot-settings-administration-oneclick.png)*Administration — One-click simplicity: every admin action is accessible in a single click, no technical skills required*\n\n[\n](/jgouviergmail/LIA-Assistant/blob/main/docs/assets/screenshot-settings-administration-llm.png)*Administration — LLM Configuration: 7 providers (OpenAI, Anthropic, Google Gemini, DeepSeek, Qwen, Perplexity, Ollama), per-node model selection*\n\n**19+ Specialized Agents**: Contacts, Emails, Calendar, Drive, Tasks, Reminders, Places, Routes, Weather, Wikipedia, Perplexity, Brave, Web Search, Web Fetch, Browser Control (with progressive screenshot streaming), Smart Home (Philips Hue), Context, Query + dynamic MCP agents**ReAct Execution Mode**([ADR-070](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-070-ReAct-Execution-Mode.md)): Alternative to the pipeline — the LLM iteratively reasons about tool outputs and decides next steps autonomously. User-toggleable preference, 4-node LangGraph architecture with native HITL support, timeout enforcement, cross-domain initiative via prompt engineering. Supports all tools including MCP and Skills**MCP (Model Context Protocol)**: Per-user external tool servers with OAuth 2.1, SSRF protection, structured items parsing, MCP Apps (interactive iframe widgets),**Iterative Mode (ReAct)** for complex servers — a dedicated agent reads docs then calls tools correctly**Agent Initiative Phase**: Post-execution cross-domain enrichment — the assistant proactively verifies related information (e.g., weather shows rain → checks calendar for outdoor events). Prompt-driven, read-only, fully configurable**Skills (agentskills.io) with Rich Outputs**: Open standard for expert instructions (SKILL.md), model-driven activation, progressive disclosure (L1/L2/L3), sandboxed scripts, marketplace import, auto-translated multi-language descriptions, ZIP download, admin management.**Rich Skill Outputs**(v1.16.8): skills can return interactive HTML frames (iframe srcDoc or external URL) and/or images in addition to text, via a simple JSON contract (`SkillScriptOutput`\n\n). Automatic theme & locale sync (theme switch propagates live to frames via`postMessage`\n\n), iframe auto-resize, CSP-sandboxed client-side interactivity (`addEventListener`\n\n,`crypto.getRandomValues`\n\n), bundled`segno`\n\nfor QR codes. Seven built-in rich skills:`interactive-map`\n\n,`weather-dashboard`\n\n,`calendar-month`\n\n,`qr-code`\n\n,`pomodoro-timer`\n\n,`unit-converter`\n\n,`dice-roller`\n\n.**Planner skill guard**: multi-domain deterministic skills are protected from false-positive early clarification requests via domain overlap detection (`_has_potential_skill_match`\n\n).**Built-in Skill Generator**: create custom skills in natural language — the assistant guides you through need analysis and archetype selection (the dialogue keeps its context across turns), then validates and**installs the finished skill directly into My Skills**, announced by name and immediately usable. Every import path (chat-generated or manual upload) goes through one hardened pipeline: strict name validation, zip-expansion caps, name-conflict rejection, atomic install with automatic rollback**Agentic Telephony**([ADR-127](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-127-Agentic-Telephony.md)): LIA places real outbound phone calls on your behalf via your own per-user ElevenLabs + Twilio connector (BYO — zero cost on LIA's side). Every call is HITL-confirmed before dialing; the goal-driven voice agent greets the instant the line opens, resolves relative dates against a live temporal anchor, and hangs up when done. Privacy by capability: the call agent can only read free/busy availability — never event titles or contents; no recording, no stored transcript. A**strict mandate boundary** forbids any expense or commitment beyond the objective (offers are captured with their price and deferred to you), and the asynchronous post-call summary must state every cost and flag every open point. Config self-heals: fingerprint-based lazy re-sync of the vendor agent, self-healing one-active-call guard (vendor status probe, deleted-conversation 404 handling), pinned thinking-free agent LLM, telephony-native`ulaw_8000`\n\naudio**AI Image Generation & Editing**: Generate images from text prompts (gpt-image-1), edit existing images with natural language instructions. Multi-provider factory architecture, per-user quality/size preferences, cost tracking with DB-cached pricing, attachment-based storage with cascade cleanup**File Attachments (Images, PDF)**: Upload with client-side compression, configurable LLM vision analysis, PDF text extraction, strict per-user isolation** Semantic Routing**: Binary classification with confidence scoring (high >0.85, medium >0.65)** Multi-Step Planning**: ExecutionPlan DSL with dependencies and conditions** Parallel Execution**: asyncio.gather for independent domains** Intelligent Context Compaction**: LLM-based conversation history summarization when token count exceeds dynamic threshold (ratio of response model context window). Preserves identifiers (UUIDs, URLs, emails).`/resume`\n\ncommand for manual trigger. 4 HITL safety conditions prevent compaction during active approval flows**Scroll-up History Pagination**:`GET /conversations/me/messages`\n\nexposes a keyset cursor (`?before=<created_at>`\n\n) with`has_more`\n\n/`next_cursor`\n\n. The chat UI binds an`IntersectionObserver`\n\non a top sentinel — older pages prepend with id-based dedup, scroll position preserved via a shared`wasPrependRef`\n\nthat skips the auto-scroll-to-bottom for that cycle. Conversations of any length stay fully reachable; the existing`(conversation_id, created_at DESC)`\n\ncomposite index makes each page an index-only seek. Bounds env-tunable (`CONVERSATION_HISTORY_DEFAULT_LIMIT`\n\n/`_MAX_LIMIT`\n\n)\n\n**5-Layer Psychological State**: Big Five personality traits (permanent) → PAD mood space with 14 moods (hours) → 22 discrete emotions with cross-suppression (minutes) → 4-stage relationship progression (weeks) → curiosity/engagement drives (per-session)**Show, Don't Tell**: Mood and emotions subtly influence word choice, sentence rhythm, energy level, and relational tone — the assistant never declares \"I'm feeling happy\"**Emotional Avatar**: Mood-responsive emoji with colored ring on each message. Historical avatars persisted per-message for reload consistency**Evolution Awareness**: The assistant knows how its mood shifted since the last message, providing narrative continuity** 4-Chart Dashboard**: Interactive recharts visualization of mood (PAD), emotions, relationship, and drives over time (24h to 90 days)** Education Guide**: 7-section interactive documentation explaining every layer, with descriptive tables for 14 moods and 22 emotions** Customizable Temperament**: Expressiveness (stoic → highly expressive) and stability (volatile → very stable) sliders. Soft reset (mood only) and full reset (everything) with explicit scope descriptions**Global Injection**: Behavioral directives injected via template variables into all user-facing text generation (response, notifications, reminders, voice) within semantic XML blocks (`<InnerState purpose=\"tone-calibration\">`\n\n)**Safety Guardrail**: Explicit instruction prevents the LLM from projecting its own emotional state onto the user** Self-Report**: Zero-cost emotion tracking via hidden`<psyche_eval/>`\n\ntag — no additional LLM call\n\n**Voice Input (STT)**\n\n**Push-to-Talk**: Hold microphone button to speak, release to transcribe. Optimized for mobile (anti-long-press CSS, touch gesture handling)**Wake Word**: Say \"OK Guy\" to activate hands-free recording. Sherpa-onnx WASM (Whisper Tiny.en) runs entirely in-browser — no audio sent externally for wake word detection**Per-User Language**: STT transcription uses the user's preferred language setting (Whisper Small, 99+ languages, fully offline)** Latency Optimized**: Mic stream reuse, WebSocket pre-warming, parallel setup, cached AudioWorklet (~50-100ms wake-to-record)\n\n**Voice Output (TTS)**\n\n| Provider | Models | Cost | Latency (TTFA) | Notes |\n|---|---|---|---|---|\n| Edge TTS (Microsoft Neural) | `edge-tts` |\nFree | ~250 ms | Multilingual neural voices, free fallback |\n| OpenAI TTS | `tts-1` / `tts-1-hd` |\n$15 / $30 per 1M chars | ~500 ms | 6 stable voices (alloy, echo, fable, onyx, nova, shimmer) |\n| ElevenLabs TTS | `eleven_multilingual_v2` |\n$100 / 1M chars | ~300 ms | High-quality multilingual, Voice Library access |\n`eleven_turbo_v2_5` |\n$50 / 1M chars | ~250 ms | Sweet-spot quality / latency | |\n`eleven_flash_v2_5` |\n$50 / 1M chars | ~75 ms | Ultra-low-latency for conversational agents |\n\n**Catalogue-driven**(ADR-081): provider/model/voice are admin-controlled via Configuration LLM (LLM type`voice_tts`\n\n). Voice + tuning live in`provider_config`\n\nJSONB. No env vars to maintain across deployments.**Sentence streaming**(ADR-082): TTS runs sentence-by-sentence pipelined with the LLM stream. First audio lands in ~1 s on chat mode (was ~5 s).**Per-message cost transparency**:`🔊 N chars · €X.XXX`\n\nbadge on the assistant bubble (paid providers only — Edge stays badge-free as it's $0).**Graceful degradation**: missing API key on a paid provider transparently falls back to Edge with a structured warning log.** Persistent HTTP pool**on ElevenLabs: keep-alive across sentences saves ~100–300 ms TLS handshake per call.\n\n```\n# DSL Syntax\nExecutionStep(\n    tool_name=\"send_email\",\n    for_each=\"$steps.get_contacts.contacts\",\n    for_each_max=10\n)\n```\n\n**HITL Thresholds**: Mutations >= 1 trigger mandatory approval** Bulk Operations**: Send emails, update contacts, mass deletions\n\n| Service | Role | Optimization |\n|---|---|---|\n| QueryAnalyzerService | Routing decision | LRU Cache |\n| SmartPlannerService | ExecutionPlan generation | Pattern Learning |\n| SmartCatalogueService | Tool filtering | 96% token reduction |\n| PlanPatternLearner | Bayesian learning | Bypass >90% confidence |\n\n**Gmail**: Search, read, send, reply, trash** Contacts**: Fuzzy search, list, details (14+ schemas)** Calendar**: Search, create, update events** Drive**: Search, file/folder listing** Tasks**: Full CRUD with completion\n\n**Apple Mail**: Search, read, send, reply, forward, trash (IMAP/SMTP)** Apple Calendar**: Search, create, update, delete events (CalDAV)** Apple Contacts**: Search, list, create, update, delete (CardDAV)\n\n**Outlook**: Search, read, send, reply, forward, trash (Graph API)** Calendar**: Search, create, update, delete events (calendarView)** Contacts**: Search, list, create, update, delete** To Do**: Full CRUD with completion (task lists + tasks)** Multi-tenant**: Personal accounts (outlook.com) and business accounts (Azure AD) via`tenant=common`\n\n- Only one provider per functional category (email, calendar, contacts, tasks)\n- 3 supported providers: Google, Apple, Microsoft\n- Activating a new provider automatically deactivates the active competitor\n\n**Voice-controlled lighting**: Turn lights on/off, adjust brightness and colors via natural language** Room & scene management**: Control entire rooms or activate predefined scenes (\"dim the living room\", \"activate movie mode\")** Local or cloud connection**: Connect via local bridge IP or Philips Hue cloud API** Feature flag**:`PHILIPS_HUE_ENABLED=true`\n\nto enable\n\n| Type | Trigger | Severity |\n|---|---|---|\n| Plan Approval | Destructive actions | CRITICAL |\n| Clarification | Detected ambiguity | WARNING |\n| Draft Critique | Email/Event review | INFO |\n| Destructive Confirm | Deletion of >= 3 items | CRITICAL |\n| FOR_EACH Confirm | Bulk mutations | WARNING |\n| Modifier Review | Review and approve AI-suggested modifications to draft content | INFO |\n\nNote: the plan-approval level is currently auto-approved — tool-level HITL supersedes it (see\n\n[ADR-106]); the other five levels interrupt execution and wait for the user.\n\n**Prometheus**: 419 custom metrics (agents, LLM, infrastructure)** Grafana**: 22 production-ready dashboards** Langfuse**: LLM-specific tracing with prompt versions** Loki**: Structured JSON logs with PII filtering** Tempo**: Distributed cross-service tracing** Probes**: liveness (`GET /health`\n\n, always 200 while the process serves — what Docker healthchecks poll) split from readiness (`GET /ready`\n\n, 503 unless PostgreSQL**and** Redis answer) —[ADR-115](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-115-Liveness-Readiness-Probes.md)**Alerting**: a 13-alert vital core (service/DB/Redis down, disk, container OOM, 5xx rate, SSE latency, backup failure, public-endpoint & TLS-certificate probes, chain self-monitoring) evaluated by Prometheus and emailed by a dedicated Alertmanager — unit-tested with`promtool test rules`\n\n, every alert linking its runbook —[ADR-119](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-119-Alerting-Reactivation-Minimal-Core.md)\n\n| Type | Tracking | Export |\n|---|---|---|\nLLM Tokens |\nPer node, per provider | Detailed CSV |\nGoogle API |\nPer endpoint, per user | Detailed CSV |\nAggregated |\nPer user, per period | CSV summary |\n\n**Google Maps Platform**: Places, Routes, Geocoding, Static Maps** Dynamic Pricing**: Admin UI for full LLM catalogue CRUD — provider, 8 capability flags (max input/output tokens, tools, structured output, strict mode, streaming, vision, reasoning) and pricing per model, all stored in the database. Same surface for image generation models (provider + quality/size/pricing). Cross-worker cache invalidation via Redis Pub/Sub (ADR-063), live cross-sibling refresh in the frontend — no code change, no redeploy**ContextVar Pattern**: Implicit tracking without explicit parameter passing** Admin CSV Exports**: Token usage, Google API usage, Consumption summary (all users or filtered by user)** User CSV Exports**(v1.9.1): Personal consumption export in Settings > Features — users export their own data only (`user_id`\n\nforced server-side, IDOR-safe)\n\n**OAuth 2.1**: PKCE (S256), single-use state token** BFF Pattern**: HTTP-only cookies, Redis session with 24h TTL** Encryption**: Fernet (credentials), bcrypt (passwords)** GDPR**: Automatic PII filtering, pseudonymization, personal data export (Art. 20 data portability)** Per-User Usage Limits**: Token, message, and cost quotas (period/global) with 5-layer defense-in-depth enforcement, admin kill switch, real-time dashboard with WebSocket gauges. Feature flag:`USAGE_LIMITS_ENABLED=true`\n\n**Backups**: Automated daily PostgreSQL dumps (pg_dump sidecar, daily/weekly/monthly rotation, all`.env`\n\n-driven) with a tested one-command restore and a verification drill (`task backup:verify`\n\n) — ADR-109, runbook in`docs/runbooks/DATABASE_BACKUP_RESTORE.md`\n\n**Per-user external servers**: Each user connects their own MCP servers (third-party tools)** Flexible authentication**: None, API Key, Bearer Token, OAuth 2.1 (DCR + PKCE S256)** Enhanced security**: HTTPS-only, SSRF prevention (DNS resolution + IP blocklist), encrypted credentials (Fernet)** Structured Items Parsing**: Automatic JSON array detection into individual items with McpResultCard HTML** Auto-generated descriptions**: LLM analysis of discovered tools to generate domain descriptions optimized for intelligent routing** Per-server rate limiting**: Redis sliding window per server/tool** Feature flag**:`MCP_USER_ENABLED=true`\n\nto enable per-user\n\n**Bidirectional Telegram**: Full chat with LIA via Telegram (text, voice, HITL)** OTP Linking**: Secure account-to-Telegram linking via 6-digit OTP code (single-use, 5min TTL, brute-force protection)** HITL Inline Keyboards**: Approval/rejection buttons localized in 6 languages directly in Telegram** Voice Transcription**: Telegram voice messages to STT (Sherpa Whisper) to text processing** Proactive Notifications**: Reminders and interest alerts also sent via Telegram** Extensible Architecture**:`BaseChannelSender`\n\n/`BaseChannelWebhookHandler`\n\nabstraction for future channels (Discord, WhatsApp)**Observability**: 12 dedicated Prometheus RED metrics (latency, errors, volumes)** Feature flag**:`CHANNELS_ENABLED=true`\n\nto enable\n\n**LLM-driven proactivity**: LIA takes the initiative to inform you when relevant (weather, calendar, interests)** Multi-source aggregation**: Calendar, Weather (with change detection), Tasks, Interests, Memories, Activity — parallel fetch** 2-phase LLM decision**: Phase 1 (structured output, cost-effective model) decides whether to notify, Phase 2 rewrites with user personality and language**Intelligent anti-redundancy**: Recent history + cross-type dedup (heartbeat vs. interests) in the decision prompt** User control**: Push notifications (FCM/Telegram) independently toggleable, configurable daily max (1-8), dedicated time windows (independent from interests)**Weather change detection**: Rain start/end, temperature drops, wind alerts — truly actionable notifications** Feature flag**:`HEARTBEAT_ENABLED=true`\n\nto enable\n\n**Recurring actions**: Schedule repetitive actions executed automatically (send emails, checks, reminders)** Timezone-aware**: Correct timezone handling per user** Retry logic**: Automatic retries on failure with back-off** Auto-disable**: Automatic deactivation after N consecutive failures** Multi-channel integration**: Result notifications via FCM, SSE, and Telegram** Feature flag**:`SCHEDULED_ACTIONS_ENABLED=true`\n\nto enable\n\n**Persistent specialized agents**: Create sub-agents with custom instructions, skills, and LLM configuration** Read-only V1**: Sub-agents perform research, analysis, and synthesis — no write operations** Template-based creation**: Pre-defined templates (Research Assistant, Writing Assistant, Data Analyst)** Invisible to user**: The principal assistant orchestrates sub-agents and presents results naturally** Token guard-rails**: Per-execution budget, daily budget, auto-disable after consecutive failures** Feature flag**:`SUB_AGENTS_ENABLED=true`\n\nto enable (default: false)\n\n**Personal knowledge bases**: Create spaces, upload documents in 15+ formats (PDF, DOCX, PPTX, XLSX, CSV, RTF, HTML, EPUB, and more), automatic chunking and embedding**Google Drive folder sync**: Link Google Drive folders to spaces for automatic file vectorization with incremental change detection (new, modified, deleted). Feature flag:`RAG_SPACES_DRIVE_SYNC_ENABLED`\n\n**Hybrid search**: Semantic similarity (pgvector cosine) + BM25 keyword matching with configurable alpha fusion** Response enrichment**: RAG context automatically injected into assistant responses when active spaces exist** Full cost transparency**: Embedding costs tracked per document and per query, visible in chat bubbles and dashboard** System knowledge spaces**: Built-in FAQ knowledge base (119+ Q/A across 17 sections) indexed from Markdown files (`docs/knowledge/`\n\n).`is_app_help_query`\n\ndetection by QueryAnalyzer, RoutingDecider Rule 0 override, App Identity Prompt injection with lazy loading (zero overhead on normal queries). Auto-indexed at startup with SHA-256 hash-based staleness. Admin UI for reindex and staleness monitoring.[ADR-058](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-058-System-RAG-Spaces.md)**Admin reindexation**: Full reindex when embedding model changes, with Redis mutual exclusion and automatic dimension ALTER. System spaces have independent reindex via admin API**Observability**: 17 Prometheus metrics (14 user + 3 system), dedicated Grafana dashboard** Feature flags**:`RAG_SPACES_ENABLED=true`\n\n(user spaces),`RAG_SPACES_SYSTEM_ENABLED=true`\n\n(system FAQ spaces)\n\n**Introspective notebooks**: The assistant maintains thematic journals (self-reflection, user observations, ideas & analyses, learnings) written in first person, colored by its active personality**Four abstraction levels**: Each entry carries a`level`\n\n—`L0`\n\nraw observation,`L1`\n\noperational directive (`WHEN→DO BECAUSE`\n\n),`L2`\n\ntransversal pattern,`L3`\n\nportrait facet. L2/L3 are produced exclusively at consolidation through active topic clustering ([ADR-079](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-079-Stratified-Journal-Consciousness.md))**Epistemic status**:`confidence`\n\n∈ {low, medium, high} plus`evidence_count`\n\nand`contradiction_count`\n\ncounters per entry. The journal distinguishes hypotheses still in test from observations validated across many turns**Deferred self-evaluation T → T+1**:`MessagesState.injected_journal_ids`\n\ncarries IDs across turns; the post-conversation extractor sees the previous turn's directives + the current user reaction, signals`evidence_outcome=\"evidence\" | \"contradiction\"`\n\n, and the service atomically increments the counters.**Zero added LLM cost**(same extractor call, enriched prompt). Anti-hallucination layer 4: LLM never writes absolute counter values.** Dual trigger**: Post-conversation extraction (fire-and-forget) + periodic consolidation (APScheduler, 4–12 h cooldown)** Gemini dual-vector embeddings**:`gemini-embedding-001`\n\n(1536d) — one vector on title+content, one on`search_hints`\n\nkeywords. Search uses`LEAST(dist_content, dist_keyword)`\n\nper row to bridge the assistant's introspective vocabulary and the user's vocabulary ([ADR-069](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-069-Gemini-Embedding-Migration.md))**Ambient diffusion of the user-model portrait**: Consolidation produces, in the same LLM call, a`portrait_full`\n\n(~200 tokens) for conversation/planner and a`portrait_brief`\n\n(~60 tokens) diffused across 6 secondary flows (ReAct setup, interest proactive, reminder notification, voice, heartbeat, fallback sync+async). Standalone builder`build_journal_user_model_block(user_id, format, flow)`\n\nmirrors`build_psyche_prompt_block`\n\n.**Three corrective levers** on the portrait (never directly editable): edit L3 source entries,`POST /journals/portrait/feedback`\n\n(free text → L0`user_correction`\n\n+ synchronous re-consolidation),`POST /journals/consolidate`\n\n(manual, bypasses cooldown).**Prompt-driven lifecycle**: The assistant manages its own journals — no hardcoded auto-archival. Mandatory pairwise dedup at consolidation STEP 1, classification audit, active L1→L2 clustering at STEP 5**Heartbeat integration**: Journal entries enrich proactive notifications via dynamic second-pass query built from aggregated context. The compiled portrait brief is also injected so the notification voice is aligned with the same user model used by conversation**Full user control**: Enable/disable (data preserved), consolidation toggle, conversation history analysis (with cost warning), 4 configurable numeric settings, group-by Theme/Level toggle, filter \"show only entries never used\", full CRUD in Settings (level + confidence editable)**4-layer anti-hallucination**: prompt guidance with ID reference tables,`field_validator`\n\non UUIDs, known-ID filtering in extraction and consolidation, atomic counter increments**11 Prometheus metrics**:`journal_entries_total{action,theme,source}`\n\n,`journal_evidence_total{outcome}`\n\n,`journal_consolidation_promotions_total{from_level,to_level}`\n\n,`journal_level_distribution{level}`\n\n,`journal_portrait_present_total{flow,format}`\n\n,`journal_portrait_age_hours`\n\n,`journal_portrait_feedback_total{outcome}`\n\n, etc.**Debug panel**: Dedicated \"Personal Journals\" section showing injection metrics AND background extraction results (CREATE/UPDATE/DELETE badges with theme/title/mood, even on partial updates where the LLM omits fields)**Cost transparency**: Real token costs tracked via TrackingContext, visible in Settings and dashboard** GDPR**: Account deletion scrubs the three portrait columns alongside entries; export endpoint includes the compiled portrait under a`portrait`\n\nkey**Feature flags**:`JOURNALS_ENABLED=false`\n\n(system), user-level toggle in Settings > Features. ADRs:[ADR-057](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-057-Personal-Journals.md)→[ADR-064](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-064-Journal-Analyst-Persona.md)→[ADR-069](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-069-Gemini-Embedding-Migration.md)→[ADR-079](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-079-Stratified-Journal-Consciousness.md)\n\n**Two token-authenticated endpoints**(`POST /api/v1/ingest/health/steps`\n\nand`/api/v1/ingest/health/heart_rate`\n\n): an iPhone Shortcut automation pushes daily batches of samples. Each sample carries its own ISO 8601`date_start`\n\n/`date_end`\n\n— UTC-normalized server-side and second-truncated to keep uniqueness stable.**Polymorphic single-table storage**(`health_samples`\n\n): one row per sample with a`kind`\n\ndiscriminator (`heart_rate`\n\n|`steps`\n\n). Extending to`spo2`\n\n/`sleep`\n\n/`calories`\n\nreduces to a new`kind`\n\nvalue — no new table, no new endpoint.**Idempotent UPSERT**(`ON CONFLICT (user_id, kind, date_start, date_end) DO UPDATE`\n\n) using PostgreSQL's`RETURNING (xmax = 0)`\n\ntrick to split insert vs update counts in a single round-trip. Re-sending the same batch is free — last value wins.**Flexible body parser**: accepts JSON array, NDJSON,`{\"data\": [...]}`\n\nenvelope, and the iOS Shortcuts \"Dictionnaire\" wrapping (`{\"<ndjson_blob>\": {}}`\n\n) — no contract pressure on the user's Raccourci authoring.**Per-user hashed tokens**: SHA-256 digest stored, raw value (`hm_xxx`\n\n) returned once at generation, display prefix shown in Settings, individually revocable. Multiple tokens may coexist for rotation.**Mixed per-sample validation**: out-of-range / malformed / missing-field / invalid-date samples are individually rejected with their 0-based index + reason, while valid siblings in the same batch persist.**Bucketed aggregation**(`hour / day / week / month / year`\n\n): heart rate averaged (plus min / max), steps SUM-ed per bucket; gaps kept (`has_data=False`\n\n) so the UI displays honest curves.**Settings visualization**: four-section panel (ingestion API + tokens, recharts line/bar charts with period average overlays, statistics, deletion by kind or full wipe).**GDPR-aware**: deletion by kind (`DELETE ?kind=...`\n\n), full erasure (`DELETE /all`\n\n),`ON DELETE CASCADE`\n\non the user FK.**Observability**: bounded-cardinality Prometheus metrics (`health_samples_upserted_total{kind, operation}`\n\n, validation rejections, rate-limit hits, auth failures, token lifecycle, deletions, latency histogram) + Grafana dashboard 21.**Guards**: 60 req/h/token sliding-window rate limit (configurable), 1000 samples/batch cap (`413`\n\nbeyond).**Feature flag**:`HEALTH_METRICS_ENABLED=false`\n\n(system).[ADR-076](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-076-Health-Metrics-Ingestion.md)·[Guide iPhone](/jgouviergmail/LIA-Assistant/blob/main/docs/guides/GUIDE_IPHONE_SHORTCUTS_HEALTH.md)·[Technical doc](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/HEALTH_METRICS.md)\n\n**Single**: steps (summary, daily breakdown, baseline delta), heart rate (summary, baseline delta), cross-kind (overview, change detection). One agent ↔ one domain pattern, mirroring`health_agent`\n\nwith 7 hand-crafted tools`email_agent`\n\n/`event_agent`\n\n.: aggregation tools accept ISO 8601 bounds exactly like`time_min`\n\n/`time_max`\n\nwindowed queries`calendar_tools.search_events_tool`\n\n. The QueryAnalyzer resolves \"this week\" / \"last month\" into concrete date ranges, and the planner splits them across the two parameters.**Inlined figures in the LLM message**: all factual data (totals, averages, per-day values) ship in the`UnifiedToolOutput.message`\n\nso the Response LLM surfaces them without reaching into`structured_data`\n\n(pattern from`weather_tools`\n\n).**Extensible registry**(`HEALTH_KINDS`\n\n): adding sleep / SpO2 / calories = one entry in`kinds.py`\n\n— bounds, merge strategy, aggregation method, baseline kind. Service helpers iterate the registry so cross-kind logic stays generic.**Baseline & variation detection**: rolling 28-day median with`bootstrap`\n\n→`rolling`\n\nmode switch after 7 days of data, tunable thresholds (`HEALTH_METRICS_VARIATION_*`\n\nenv vars).**Heartbeat / Memory / Journal integration**:`health_signals`\n\nsource injected for proactive context;`context_biometric`\n\nJSONB persists deltas and trends in memories (never raw values) when emotional weight crosses a threshold.**Per-user opt-in**: single`health_metrics_agents_enabled`\n\ntoggle governs the four integrations (tool access, Heartbeat, memory extraction, journal injection).`PATCH /auth/me/health-metrics-agents-preference`\n\n.\n\n**Sandboxed iframes via a CSP airlock**(ADR-098): third-party widgets boot through a same-origin shell (`public/widget-frame.html`\n\n) served with its own permissive CSP, so external-CDN widgets (Excalidraw, …) work while the main app keeps a strict policy. Isolation is the iframe`sandbox`\n\n(opaque origin, no parent cookies/DOM), not the CSP; the shell is hardened by anti-abuse locks +`frame-ancestors 'self'`\n\n**JSON-RPC Bridge**: Bidirectional communication between iframe app and chat via PostMessage JSON-RPC 2.0** Excalidraw Iterative Builder**: Intent-based diagram generation via dedicated LLM calls (shapes + arrows) with cheat sheet injection for format accuracy. Runs under a dedicated MCP-step timeout family (300 s floor / 600 s ceiling, ADR-100) so complex diagrams are not cut off mid-generation: MCP servers exposing a`read_me`\n\nconvention`read_me`\n\ntool have their content auto-injected into the planner prompt**Auto-generated descriptions**: LLM analysis of discovered tools for domain description optimized for routing** App-only tools**: Tools with`visibility: [\"app\"]`\n\nfiltered from the LLM catalogue (iframe only)\n\nLIA is fully translated in **6 languages**: English, French, German, Spanish, Italian, and Chinese.\n\n**Complete UI coverage**: All interfaces, dialogs, notifications, error messages, FAQ, and landing page** HITL localized**: Human-in-the-Loop approval prompts adapted per language** Proactive notifications**: Heartbeat and reminders delivered in the user's language** Telegram**: Inline keyboards and messages localized** Skills**: Auto-translated descriptions in all 6 languages** react-i18next**: Namespace-based translations with`locales/{lang}/translation.json`\n\n**Animated hero chat demo**: three rotating scenarios mirroring the real display modes — HITL draft approval, rich HTML weather card + proactive cross-domain initiative, multi-agent Markdown reply — with per-mode title-bar chips**Proof band**: verifiable engineering numbers (agents, tools, providers, tests, ADRs, releases, audit score) sourced from the codebase (`LANDING_STATS`\n\ndocuments each origin)**Two-mode diagram**: faithful LangGraph topology — router fork, five numbered pipeline steps (human approval highlighted), ReAct reason→act→observe loop, streaming convergence(6 languages): how LIA is built — method, trade-offs, operations, measured audit profile — on the /why–/how guide pattern`/story`\n\nfield report**SEO & OpenGraph**: dynamically generated OG image, per-locale hreflang, JsonLd (WebSite, Organization, SoftwareApplication, breadcrumbs),`llms.txt`\n\nfor AI crawlers**Public-route guard**: the 401 handler's public-page list is pinned by a filesystem-completeness test — a new public page missing from the list fails CI instead of ejecting anonymous visitors to /login**Authenticated redirect**: automatic redirect to dashboard if already logged in\n\nLIA includes a **full-featured administration interface** — giving operators complete control and real-time visibility over the system without touching configuration files or the database.\n\nA web-based administration panel covering every operational aspect:\n\n| Section | Capabilities |\n|---|---|\nLLM Configuration |\nModel selection per node, provider parameters, temperature/token limits, prompt versions |\nRAG Knowledge Spaces |\nManage document spaces, embedding configuration, user reindex operations, system knowledge spaces (FAQ staleness, reindex) |\nPersonalities |\nCreate and manage assistant personalities (tone, language, behavior rules) |\nUser Management |\nUser accounts, roles, permissions, connector status overview |\nConnector Management |\nGoogle/Apple/Microsoft OAuth status, token health, per-user provider activation |\nSkills Management |\nEnable/disable skills, edit descriptions, translate in 6 languages, delete |\nMCP Servers |\nAdmin-level MCP server configuration, tool discovery, domain descriptions |\nLLM Pricing |\nCRUD for the full LLM catalogue — provider, 8 capability flags (max input/output tokens, tools, structured output, strict mode, streaming, vision, reasoning) and pricing (input/output/cache tokens) per model. Source of truth for the LangChain factory and the agent constraints. Live cross-worker invalidation, no redeploy |\nImage Generation Pricing |\nCRUD for image models — provider, quality, size and pricing. Drives the user preferences dropdowns directly |\nGoogle API Pricing |\nPer-endpoint pricing configuration for Google Maps Platform services |\nVoice Settings |\nTTS catalogue management (Edge / OpenAI / ElevenLabs) via Configuration LLM (`voice_tts` type), per-provider tuning, voice picker (live ElevenLabs voices) |\nBroadcasting |\nSend system-wide notifications to all users or targeted groups |\nDebug Settings |\nToggle debug panel visibility, configure diagnostic verbosity per user |\nUsage Limits |\nPer-user token/message/cost quotas (period + global), real-time gauges, manual block/unblock, WebSocket live updates |\nConsumption Export |\nCSV export of token usage, Google API usage, and aggregated consumption per user/period |\n\nA 24-section debug panel embedded in the chat interface, organized into **6 logical groups** with always-visible sections (empty sections show \"N/A\" instead of disappearing):\n\n| Group | Sections |\n|---|---|\nRequest Analysis |\nIntent classification, Domain detection, Routing decision, Query transformations |\nPlanning & Execution |\nPlanner output, Tool selection, Context resolution, Token budget, Execution timeline, ForEach analysis, Execution waves |\nIntelligent Mechanisms |\nCache hits, pattern learning, semantic expansion, Skills activation |\nContext Injection |\nMemory injection (scores), RAG injection (scores), Knowledge enrichment (Brave), Journal injection (per-entry scores, budget) |\nBackground Extraction |\nMemory detection (create/update/delete), Journal extraction, Interest profile |\nLLM & API Pipeline |\nRequest lifecycle (timing breakdown per node), LLM Pipeline (chronological reconciliation), LLM call details (model, tokens, latency, cost), Google API calls |\n\nThe debug panel is designed for\n\ndevelopers and operatorsto diagnose issues, optimize prompts, and understand the agent's decision-making process in real time — without needing external tools or log access.\n\n| Software | Version | Required |\n|---|---|---|\n| Python | 3.12+ | Yes |\n| Node.js | 24 LTS | Yes |\n| Docker | 24+ | Yes |\n| pnpm | 10+ | Yes |\n|\n\nAll commands are defined in `Taskfile.yml`\n\n. Quick start: `task setup`\n\nthen `task dev`\n\n.\n\n```\n# 1. Clone the repository\ngit clone https://github.com/jgouviergmail/LIA-Assistant.git\ncd LIA-Assistant\n\n# 2. Configure environment\ncp .env.example .env  # Edit with your API keys\n\n# 3. Full setup (backend + frontend + git hooks)\ntask setup\n\n# 4. Start all services (API + Web + PostgreSQL + Redis + observability)\ntask dev\n```\n\n**Manual setup (without Task)**\n\n```\n# 1. Start the infrastructure\ndocker compose up -d postgres redis prometheus grafana\n\n# 2. Backend setup\ncd apps/api\npython -m venv .venv && source .venv/bin/activate  # Windows: .venv\\Scripts\\activate\npip install --require-hashes -r requirements.lock.txt  # compiled lockfile (reproducible)\ncp ../../.env.example .env  # Configure your API keys\n\n# 3. Database migrations\nalembic upgrade head\n\n# 4. Frontend setup\ncd ../web\npnpm install\n\n# 5. Start the services\n# Terminal 1 - Backend:\ncd apps/api && uvicorn src.main:app --reload --port 8000\n\n# Terminal 2 - Frontend:\ncd apps/web && pnpm dev\n```\n\n| Service | URL | Credentials |\n|---|---|---|\n| Frontend |\n|\n\n[http://localhost:8000/docs](http://localhost:8000/docs)[http://localhost:3001](http://localhost:3001)[http://localhost:9090](http://localhost:9090)\n\n```\n# Database\nDATABASE_URL=postgresql+asyncpg://user:pass@localhost:5432/lia\nREDIS_URL=redis://localhost:6379/0\n\n# Security (REQUIRED - change in production)\nSECRET_KEY=change-me-in-production-use-openssl-rand-base64-32\nFERNET_KEY=your-fernet-key-here\n\n# LLM Provider API keys are configured via Admin UI after first login\n# (Settings > Administration > LLM Configuration)\n# At least one provider (typically OpenAI) is required.\n\n# Google OAuth (optional)\nGOOGLE_CLIENT_ID=...\nGOOGLE_CLIENT_SECRET=...\n\n# Feature Flags (optional, disabled by default)\nMCP_ENABLED=false              # Admin MCP servers\nMCP_USER_ENABLED=false         # Per-user MCP (requires MCP_ENABLED)\nCHANNELS_ENABLED=false         # Multi-channel messaging (Telegram)\nHEARTBEAT_ENABLED=false        # Autonomous proactive notifications\nSCHEDULED_ACTIONS_ENABLED=false # Recurring scheduled actions\nSUB_AGENTS_ENABLED=false       # Persistent specialized sub-agents\nSKILLS_ENABLED=false           # Skills system (agentskills.io standard)\nRAG_SPACES_ENABLED=true        # RAG Knowledge Spaces (document upload & retrieval)\nFCM_NOTIFICATIONS_ENABLED=false # Firebase push notifications\n```\n\nProduction targets include Raspberry Pi (ARM64) via multi-arch Docker builds (`linux/amd64,linux/arm64`\n\n).\n\n```\n┌─────────────────────────────────────────────────────────────────────────┐\n│                        FRONTEND (Next.js 16 + React 19)                  │\n│    Chat UI • Settings • i18n (6 languages) • SSE Streaming • Voice Mode  │\n└─────────────────────────────┬───────────────────────────────────────────┘\n                              │ HTTP-only cookies (session_id, 24h TTL)\n┌─────────────────────────────┴───────────────────────────────────────────┐\n│                     BACKEND (FastAPI + LangGraph 1.x)                    │\n│                                                                          │\n│  ┌────────────────────────────────────────────────────────────────────┐ │\n│  │                 LangGraph Multi-Agent Orchestration                 │ │\n│  │                                                                      │ │\n│  │   Router → QueryAnalyzer → Planner → ApprovalGate → Orchestrator   │ │\n│  │      ↓                                        ↓                     │ │\n│  │   ┌─────────────────────────────────────────────────────────────┐  │ │\n│  │   │  Contacts │ Emails │ Calendar │ Drive │ Tasks │ Reminders  │  │ │\n│  │   │  Places │ Routes │ Weather │ Wikipedia │ Perplexity      │  │ │\n│  │   │  Brave │ Web Search │ Web Fetch │ Browser │ Context │ Query│  │ │\n│  │   └─────────────────────────────────────────────────────────────┘  │ │\n│  │                              ↓                                      │ │\n│  │               MCP Tools (per-user external servers)                │ │\n│  │                              ↓                                      │ │\n│  │                       Response Node (synthesis)                     │ │\n│  └────────────────────────────────────────────────────────────────────┘ │\n│                                                                          │\n│  ┌─────────────────────────────────────────────────────────────────────┐│\n│  │  Domain Services: Auth, Users, Connectors, RAG, Voice, Skills...    ││\n│  └─────────────────────────────────────────────────────────────────────┘│\n│                                                                          │\n│  ┌─────────────────────────────────────────────────────────────────────┐│\n│  │  Infrastructure: Redis (cache) • PostgreSQL (checkpoints) •         ││\n│  │  MCP Client Pool • Prometheus (metrics) • Langfuse (traces)       ││\n│  └─────────────────────────────────────────────────────────────────────┘│\n└──────────────────────────────────────────────────────────────────────────┘\n```\n\nLIA offers two execution strategies, switchable per user via a toggle in the chat header:\n\n**Pipeline mode** (default) — A feat of engineering that delivers the same power as ReAct with **4–8× fewer tokens**:\n\n- A smart\n**Planner** decomposes the request into an optimized execution plan (DSL) - A\n**Semantic Validator** checks plan coherence (cardinality, scope, dependencies) - An\n**Approval Gate** handles HITL for mutations - A\n**Task Orchestrator** executes tools in parallel waves via`asyncio.gather()`\n\n**Bayesian learning** optimizes planning patterns over time\n\n**ReAct mode** (⚡) — The LLM reasons iteratively, calling tools one by one and adapting to each result. More autonomous but higher token cost. Ideal for exploratory, research, or ambiguous queries.\n\n``` php\ngraph TD\n    A[User Message] --> B[Router Node]\n    B -->|conversation| C[Response Node]\n    B -->|pipeline mode| D[Planner Node]\n    B -->|react mode| R1[ReAct Setup]\n    D --> E[Semantic Validator]\n    E --> F{Approval Gate}\n    F -->|approved| G[Task Orchestrator]\n    F -->|rejected| C\n    G --> H[Domain Agents + Tools]\n    H --> G\n    G --> C\n    R1 --> R2[ReAct Call Model]\n    R2 -->|tool_calls| R3[ReAct Execute Tools]\n    R2 -->|done| R4[ReAct Finalize]\n    R3 --> R2\n    R4 --> C\n    C --> J[SSE Stream]\napps/api/src/\n├── core/                    # Modular configuration (9 modules)\n│   ├── config/              # Settings per domain\n│   ├── constants.py         # Global constants\n│   └── bootstrap.py         # Initialization functions\n├── domains/                 # Bounded Contexts (DDD)\n│   ├── agents/              # LangGraph nodes, services, tools\n│   │   ├── nodes/           # Graph nodes (router, planner, react ×4, response...)\n│   │   ├── services/        # Smart services, HITL\n│   │   ├── tools/           # Domain-specific tools\n│   │   └── orchestration/   # ExecutionPlan, parallel executor\n│   ├── auth/                # JWT, sessions, OAuth\n│   ├── connectors/          # Google + Apple + Microsoft clients, provider resolver\n│   ├── conversations/       # Conversation CRUD & history\n│   ├── google_api/          # Google API pricing & usage tracking\n│   ├── rag_spaces/          # RAG Knowledge Spaces (upload, embed, retrieve, system FAQ)\n│   ├── user_mcp/            # Per-user MCP servers (CRUD, OAuth, domain routing)\n│   ├── voice/               # TTS factory, STT, Wake Word\n│   ├── skills/              # Skills system (agentskills.io standard)\n│   ├── sub_agents/          # Persistent specialized sub-agents (F6)\n│   ├── interests/           # Interest Learning System\n│   ├── heartbeat/           # Autonomous Heartbeat (Proactive Notifications)\n│   ├── channels/            # Multi-channel messaging (Telegram)\n│   ├── reminders/           # Reminder & notification scheduling\n│   ├── scheduled_actions/   # Recurring scheduled actions\n│   ├── journals/            # Personal Journals (introspective notebooks)\n│   ├── health_metrics/      # iPhone Shortcuts health ingestion + charts\n│   └── users/               # User management\n└── infrastructure/          # Cross-cutting concerns\n    ├── cache/               # Redis sessions, LLM cache\n    ├── llm/                 # Factory, providers, embeddings\n    ├── mcp/                 # MCP client pool, auth, security, tool adapters\n    ├── browser/             # Playwright session pool, CDP accessibility\n    ├── rate_limiting/       # Distributed rate limiter\n    └── observability/       # Metrics, logging, tracing\n```\n\n**Tool System (5-layer architecture)** — Tools are built in five composable layers: `ConnectorTool[ClientType]`\n\n(generic base with OAuth auto-refresh), `@connector_tool`\n\n(meta-decorator composing metrics + rate limiting + context save), Formatters (domain-specific result normalization), `ToolManifest`\n\n+ Builder (declarative declaration with semantic keywords), and Catalogue Loader (dynamic introspection). Per-tool boilerplate reduced from ~150 to ~8 lines (94% reduction). Category-based rate limits: Read (20/min), Write (5/min), Expensive (2/5 min).\n\n**Domain Taxonomy** — Each domain is a declarative `DomainConfig`\n\n(agents, `result_key`\n\n, `related_domains`\n\n, priority, routability). The `DOMAIN_REGISTRY`\n\nis the single source of truth consumed by SmartCatalogue (filtering), semantic expansion (adjacent domains), and the Initiative phase (structural pre-filter).\n\n**Data Registry** — An `InMemoryStore`\n\ndecouples tool results from message history. Results survive per-node message windowing (5/10/20 turns) via `@auto_save_context`\n\n, and cross-step references (`$steps.X.field`\n\n) resolve against the registry — this is what makes aggressive windowing viable without losing tool output context.\n\n**Semantic Validator** — Before HITL approval, a dedicated LLM (distinct from the planner) inspects plans against 14 issue types across four categories: Critical (hallucinated capability, ghost dependency), Semantic (cardinality mismatch, scope overflow), Safety (dangerous ambiguity), and FOR_EACH-specific validations.\n\n**Adaptive Re-Planner** — On execution failure, a rule-based analyser classifies the failure pattern and selects a recovery strategy. In **Panic Mode**, the SmartCatalogue expands to all tools for one retry, solving cases where domain filtering was too aggressive.\n\n**Connector Abstraction** — Python protocols enable transparent switching between Google, Apple, and Microsoft providers. Normalizers convert provider-specific responses into unified domain models. The `ProviderResolver`\n\nguarantees only one provider per functional category (email, calendar, contacts, tasks).\n\n**Error Architecture** — All tools return `ToolResponse`\n\n/`ToolErrorModel`\n\nwith a `ToolErrorCode`\n\nenum (18+ types) and a `recoverability`\n\nflag. API-side centralized exception raisers replace raw HTTPException everywhere.\n\n**Feature Flags** — Every optional subsystem is controlled by a `{FEATURE}_ENABLED`\n\nflag, checked at startup, route wiring, and node entry (instant short-circuit).\n\nFull technical details:\n\n[How does LIA work?]— 25-section architecture guide\n\n| Technology | Version | Role |\n|---|---|---|\n| Python | 3.12+ | Primary runtime |\n| FastAPI | 0.136.3 | REST API + SSE framework |\n| LangGraph | 1.2.4 | Multi-agent orchestration |\n| LangChain | 1.3.9 | LLM abstraction + tools |\n| SQLAlchemy | 2.0.50 | Async ORM |\n| Alembic | 1.18.4 | Database migrations |\n| PostgreSQL | 16 + pgvector | Database + vector search |\n| Redis | 7.4.0 | Cache, sessions, rate limiting |\n| Pydantic | 2.13.4 | Validation + serialization |\n| structlog | latest | Structured JSON logging |\n| openai | 2.x | LLM provider |\n| Edge TTS | 7.2+ | Voice synthesis (free) |\n| mcp | 1.9+ | Model Context Protocol SDK (Streamable HTTP) |\n| Docker | 24+ | Containerization (multi-arch amd64/arm64) |\n\n| Technology | Version | Role |\n|---|---|---|\n| Node.js | 24 LTS | JavaScript runtime |\n| Next.js | 16.2.10 | React framework |\n| React | 19.2.7 | UI library |\n| TypeScript | 6.0.2 | Type safety |\n| TailwindCSS | 4.3.2 | Styling |\n| TanStack Query | 5.101 | Server state management |\n| react-i18next | 17.0.8 | i18n (6 languages) |\n| Radix UI | latest | Accessible UI primitives |\n\n**Responsive Design**: Fully optimized for desktop, tablet, and smartphone. Adaptive layouts, touch-friendly interactions, and mobile-first components ensure a seamless experience on any device.\n\n| Provider | Models | Use Case |\n|---|---|---|\n| OpenAI | GPT-5.4, GPT-5.4-mini, GPT-5.2, GPT-5.1, GPT-5, GPT-5-mini/nano, GPT-4.1, GPT-4.1-mini/nano, GPT-4o, o1, o3-mini | Primary (prompt caching, reasoning) |\n| Anthropic | Claude Opus 4.6/4.5, Claude Sonnet 4.6, Claude Haiku 4.5 | Alternative (extended thinking) |\n| Gemini 3.1/3/2.5 Pro, Gemini 3/2.5/2.0 Flash | Multimodal | |\n| DeepSeek | deepseek-v4-flash, deepseek-v4-pro (V4 family — thinking-mode toggle, v1.19.1+), deepseek-chat (V3, legacy), deepseek-reasoner (R1, legacy) |\nCost-effective reasoning. V4 supports tools + structured output via JSON-mode fallback when thinking is on. |\n| Perplexity | sonar-small/large-128k-online | Web-augmented responses. Base URL configurable via `PERPLEXITY_BASE_URL` env var (v1.19.1+). |\n| Qwen | qwen3-max, qwen3.5-plus, qwen3.5-flash | Thinking + tools + vision (Alibaba Cloud DashScope). Base URL configurable via `QWEN_BASE_URL` (regional US/CN swap, v1.19.1+). |\n| Ollama | Any local model (dynamic discovery) | Zero API cost, self-hosted. Base URL configurable via `OLLAMA_BASE_URL` . |\n\n| Technology | Role |\n|---|---|\n| Prometheus | 419 metrics |\n| Grafana | 25 dashboards |\n| Loki | Aggregated logs |\n| Tempo | Distributed tracing |\n| Langfuse | LLM observability |\n| structlog | Structured JSON logs |\n\n| Document | Description |\n|---|---|\n|\n\n[ARCHITECTURE.md](/jgouviergmail/LIA-Assistant/blob/main/docs/ARCHITECTURE.md)[INDEX.md](/jgouviergmail/LIA-Assistant/blob/main/docs/INDEX.md)| Domain | Documents |\n|---|---|\nAgents & LLM |\n|\n\n**HITL**[HITL](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/HITL.md)•[PLAN_HITL_STREAMING_VALIDATION](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/PLAN_HITL_STREAMING_VALIDATION.md)**Voice**[VOICE](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/VOICE.md)•[VOICE_MODE](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/VOICE_MODE.md)**Memory**[LONG_TERM_MEMORY](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/LONG_TERM_MEMORY.md)•[MEMORY_RESOLUTION](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/MEMORY_RESOLUTION.md)**MCP**[MCP_INTEGRATION](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/MCP_INTEGRATION.md)•[GUIDE_MCP_INTEGRATION](/jgouviergmail/LIA-Assistant/blob/main/docs/guides/GUIDE_MCP_INTEGRATION.md)**Heartbeat**[HEARTBEAT_AUTONOME](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/HEARTBEAT_AUTONOME.md)•[GUIDE_HEARTBEAT](/jgouviergmail/LIA-Assistant/blob/main/docs/guides/GUIDE_HEARTBEAT_PROACTIVE_NOTIFICATIONS.md)**Channels**[CHANNELS_INTEGRATION](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/CHANNELS_INTEGRATION.md)•[GUIDE_TELEGRAM](/jgouviergmail/LIA-Assistant/blob/main/docs/guides/GUIDE_TELEGRAM_INTEGRATION.md)**Scheduled Actions**[SCHEDULED_ACTIONS](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/SCHEDULED_ACTIONS.md)•[GUIDE_SCHEDULED_ACTIONS](/jgouviergmail/LIA-Assistant/blob/main/docs/guides/GUIDE_SCHEDULED_ACTIONS.md)**Skills**[SKILLS_INTEGRATION](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/SKILLS_INTEGRATION.md)** Sub-Agents**[SUB_AGENTS](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/SUB_AGENTS.md)** RAG Spaces**[GUIDE_RAG_SPACES](/jgouviergmail/LIA-Assistant/blob/main/docs/guides/GUIDE_RAG_SPACES.md)•[ADR-055](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-055-RAG-Spaces-Architecture.md)•[ADR-058](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-058-System-RAG-Spaces.md)**Browser Control**[BROWSER_CONTROL](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/BROWSER_CONTROL.md)•[ADR-059](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-059-Browser-Control.md)**Personal Journals**[JOURNALS](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/JOURNALS.md)•[ADR-057](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-057-Personal-Journals.md)**LLM Providers**[LLM_PROVIDERS](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/LLM_PROVIDERS.md)** CI/CD**[CI_CD](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/CI_CD.md)** Security**[SECURITY](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/SECURITY.md)•[OAUTH](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/OAUTH.md)•[RATE_LIMITING](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/RATE_LIMITING.md)**Observability**[OBSERVABILITY_AGENTS](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/OBSERVABILITY_AGENTS.md)•[METRICS_REFERENCE](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/METRICS_REFERENCE.md)**Cost Tracking**[LLM_PRICING_MANAGEMENT](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/LLM_PRICING_MANAGEMENT.md)•[GOOGLE_API_TRACKING](/jgouviergmail/LIA-Assistant/blob/main/docs/technical/GOOGLE_API_TRACKING.md)| Guide | Description |\n|---|---|\n|\n\n[GUIDE_AGENT_CREATION](/jgouviergmail/LIA-Assistant/blob/main/docs/guides/GUIDE_AGENT_CREATION.md)[GUIDE_TOOL_CREATION](/jgouviergmail/LIA-Assistant/blob/main/docs/guides/GUIDE_TOOL_CREATION.md)[GUIDE_TESTING](/jgouviergmail/LIA-Assistant/blob/main/docs/guides/GUIDE_TESTING.md)[GUIDE_DEBUGGING](/jgouviergmail/LIA-Assistant/blob/main/docs/guides/GUIDE_DEBUGGING.md)100+ ADRs (numbered up to ADR-108) documenting major architectural decisions:\n\n[ADR-007: Service Layer Pattern for Node Complexity](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-007-Service-Layer-Pattern-For-Node-Complexity.md)[ADR-048: Semantic Tool Router](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-048-Semantic-Tool-Router.md)[ADR-051: Reminder & Notification System](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR-051-Reminder-Notification-System.md)[View all ADRs](/jgouviergmail/LIA-Assistant/blob/main/docs/architecture/ADR_INDEX.md)\n\n```\ncd apps/api\n\n# Unit tests (parallel: task test:backend:unit:fast, ~4 min)\npytest tests/unit -v\n\n# Integration tests (require PostgreSQL + Redis)\npytest tests/integration -v\n\n# LangGraph agent tests\npytest tests/agents -v\n\n# Full coverage\npytest --cov=src --cov-report=html -v\n# Report: htmlcov/index.html\n```\n\n| Metric | Value |\n|---|---|\n| Total backend tests | ~11,970 (pytest collected, 670 test files) |\n| Backend breakdown | unit fast ~10,150 · agents ~970 · integration ~580 (zero skips) |\n| Frontend tests (vitest) | 1,222 (+ 17 hermetic Playwright E2E incl. axe/dark/zoom) |\n| Coverage target | 45% backend (ratchet) · frontend thresholds locked per category |\n| CI Workflows | 3 (CI, Security, Release) |\n| Technical audit | 8.3/10 across 24 normalized areas —\n|\n\nLIA uses a two-layer quality gate: a **local pre-commit hook** (fast, on staged files only) and a **GitHub Actions CI pipeline** (comprehensive, on every push/PR to `main`\n\n).\n\n```\nPre-commit (local)              GitHub Actions CI\n===================             ==================\n.bak files check                Lint Backend (Ruff + Black + MyPy)\nSecrets grep                    Lint Frontend (ESLint + TypeScript)\nRuff + Black + MyPy             Fast unit tests + coverage (45%)\nFast unit tests                 Integration tests (PostgreSQL + Redis)\nCritical pattern detection      Agents suite\ni18n keys sync                  Code Hygiene (i18n, Alembic, lockfiles, patterns)\nAlembic migration conflicts     Docker build smoke test\n.env.example completeness       Secret scan (Gitleaks)\nESLint + TypeScript check       ──────────────────────\n                                Security workflow (weekly)\n                                  CodeQL (Python + JS)\n                                  Dependency audit (pip-audit + pnpm audit)\n                                  Trivy filesystem scan\n                                  SBOM generation\n```\n\n| Practice | Implementation |\n|---|---|\nSHA-pinned Actions |\nAll GitHub Actions pinned by commit SHA (supply-chain security) |\nReproducible builds |\nUniversal Python lockfiles (linux/amd64 + arm64 + Windows), SHA256 hash-verified installs everywhere; CI guard fails manifest edits without lock regeneration (\n|\n\n**Least privilege**`permissions: contents: read`\n\non CI workflow**Branch protection****Dependabot****Pre-commit / CI alignment****Coverage threshold**| Workflow | Trigger | Jobs |\n|---|---|---|\nCI (`ci.yml` ) |\nPush to `main` , PR |\n8 jobs: lint, unit tests, integration tests, code hygiene, docker build, secret scan |\nSecurity (`security.yml` ) |\nPR, weekly schedule, manual | CodeQL, dependency audit, Trivy, SBOM |\nRelease (`release.yml` ) |\nTag `v*` |\nDocker multi-arch build + push (ghcr.io), GitHub Release |\n\nFull details:\n\n[CI/CD Documentation]\n\n| Metric | Value | SLO |\n|---|---|---|\n| API Latency | 450ms | < 500ms |\n| First SSE event (request acknowledged) | 380ms | < 500ms |\n| Router Latency | 800ms | < 2s |\n| Planner Latency | 2.5s | < 5s |\n| Gemini Embedding | ~100-200ms | < 300ms |\n| Token Reduction (Windowing) | 93% | > 80% |\n| Context Compaction Savings | ~60% per compaction | — |\n\nThese figures measure the infrastructure. The full perceived response time depends on the LLM call cascade (seconds to tens of seconds depending on request complexity and hardware) — this is the main optimization programme in progress, measured in production. The\n\n[July 2026 technical audit]scores Performance 7.5/10: instrumentation and caching are in place, but no sustained load campaign has been executed yet.\n\n**Message Windowing**: 5/10/20 turns depending on node** Context Compaction**: LLM summarization of old messages (dynamic threshold from response model context window, configurable via`COMPACTION_*`\n\nsettings)**Prompt Caching**: OpenAI/Anthropic (90% discount)** Gemini Embeddings**: gemini-embedding-001 with asymmetric task types (multilingual)** Parallel Execution**: asyncio.gather for independent domains** Redis O(1)**: Optimized operations (vs O(N) SCAN)** Connection Pooling**: httpx persistent connections\n\n| Standard | Status |\n|---|---|\n| GDPR | PII filtering, data minimization |\n| OWASP Top 10 | XSS, SQL injection, CSRF protection |\n| Prompt Injection | External content wrapping (`<external_content>` safety markers) |\n| OAuth 2.1 | Mandatory PKCE |\n| Supply chain | Hash-verified universal lockfiles, pip-audit on the full transitive tree, SBOM per release |\n\n**DO NOT create a GitHub Issue for security vulnerabilities.**\n\nSend an email to ** liamyassistant@gmail.com** with:\n\n- Description of the vulnerability\n- Steps to reproduce\n- Potential impact\n\nWe respond within 48 hours.\n\nWe welcome all contributions! See our [Contributing Guide](/jgouviergmail/LIA-Assistant/blob/main/CONTRIBUTING.md) to get started.\n\n```\n# 1. Fork and clone\ngit clone https://github.com/YOUR-USERNAME/LIA-Assistant.git\ncd LIA-Assistant\n\n# 2. Create a branch\ngit checkout -b feature/my-feature\n\n# 3. Full setup (backend + frontend + git hooks)\ntask setup\n\n# 4. Develop and test\ntask test:backend:unit:fast\n\n# 5. Commit (Conventional Commits)\ngit commit -m \"feat(agents): add weather forecast agent\"\n\n# 6. Push and create PR\ngit push origin feature/my-feature\n```\n\n- Bug fixes\n- New features\n- Documentation\n- Tests\n- i18n translations (6 supported languages)\n- Performance optimizations\n\n**Python**: Black + Ruff + MyPy (strict)** TypeScript**: ESLint + Prettier** Commits**:[Conventional Commits](https://www.conventionalcommits.org/)** Coverage**: >= 45% enforced in CI (ratchet +2 per release, never lowered)** Pre-commit hook**: Installed via`task setup`\n\n— runs linters + tests on staged files**CI**: All PRs must pass 7 status checks before merge (see[CI/CD](#cicd))\n\n| Channel | Usage |\n|---|---|\n|\n\n[GitHub Discussions](https://github.com/jgouviergmail/LIA-Assistant/discussions)[liamyassistant@gmail.com](mailto:liamyassistant@gmail.com)This project is licensed under the **GNU Affero General Public License v3.0 (AGPL-3.0)**.\n\nSee [LICENSE](/jgouviergmail/LIA-Assistant/blob/main/LICENSE) for details.\n\nA commercial license is also available for organizations that cannot comply with AGPL-3.0 terms. Contact [liamyassistant@gmail.com](mailto:liamyassistant@gmail.com) for details.\n\nThis project builds on excellent open source technologies:\n\n**Backend & Infrastructure**\n\n[Python](https://www.python.org/)- Primary runtime[FastAPI](https://fastapi.tiangolo.com/)- Modern async web framework[LangGraph](https://github.com/langchain-ai/langgraph)- Multi-agent orchestration[LangChain](https://python.langchain.com/)- LLM abstraction & tools[SQLAlchemy](https://www.sqlalchemy.org/)- Async ORM[Pydantic](https://docs.pydantic.dev/)- Data validation & settings[Alembic](https://alembic.sqlalchemy.org/)- Database migrations[PostgreSQL](https://www.postgresql.org/)+[pgvector](https://github.com/pgvector/pgvector)- Database & vector search[Redis](https://redis.io/)- Cache, sessions, rate limiting[Google Gemini Embeddings](https://ai.google.dev/gemini-api/docs/embeddings)- gemini-embedding-001 for multilingual semantic search[Edge TTS](https://github.com/rany2/edge-tts)- Free neural voice synthesis[structlog](https://www.structlog.org/)- Structured JSON logging[Docker](https://www.docker.com/)- Containerization & multi-arch builds\n\n**Frontend**\n\n[Node.js](https://nodejs.org/)- JavaScript runtime[Next.js](https://nextjs.org/)- React framework[React](https://react.dev/)- UI library[TypeScript](https://www.typescriptlang.org/)- Type safety[TailwindCSS](https://tailwindcss.com/)- Utility-first styling[Radix UI](https://www.radix-ui.com/)- Accessible UI primitives[TanStack Query](https://tanstack.com/query/)- Server state management[react-i18next](https://react.i18next.com/)- Internationalization (6 languages)\n\n**Observability**\n\n[Prometheus](https://prometheus.io/)- Metrics & alerting[Grafana](https://grafana.com/)- Dashboards & visualization[Loki](https://grafana.com/oss/loki/)- Log aggregation[Tempo](https://grafana.com/oss/tempo/)- Distributed tracing[Langfuse](https://langfuse.com/)- LLM observability & prompt management\n\n**LIA** — Next-Generation Intelligent Conversational Assistant\n\nBuilt with ❤️ using Python, Node.js, FastAPI, LangGraph, and Next.js", "url": "https://wpnews.pro/news/show-hn-lia-a-self-hosted-multi-agent-ai-assistant-fastapi-and-langgraph", "canonical_source": "https://github.com/jgouviergmail/LIA-Assistant", "published_at": "2026-07-17 21:21:58+00:00", "updated_at": "2026-07-17 21:52:12.059276+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-tools", "ai-infrastructure", "large-language-models"], "entities": ["LIA", "FastAPI", "LangGraph", "Gemini"], "alternates": {"html": "https://wpnews.pro/news/show-hn-lia-a-self-hosted-multi-agent-ai-assistant-fastapi-and-langgraph", "markdown": "https://wpnews.pro/news/show-hn-lia-a-self-hosted-multi-agent-ai-assistant-fastapi-and-langgraph.md", "text": "https://wpnews.pro/news/show-hn-lia-a-self-hosted-multi-agent-ai-assistant-fastapi-and-langgraph.txt", "jsonld": "https://wpnews.pro/news/show-hn-lia-a-self-hosted-multi-agent-ai-assistant-fastapi-and-langgraph.jsonld"}}