{"slug": "show-hn-democr-ai-self-hosted-agentic-ai-runtime-with-audit-and-rbac", "title": "Show HN: Democr.ai – self-hosted agentic AI runtime with audit and RBAC", "summary": "Democr.ai, a self-hosted agentic AI runtime framework, launched in public beta, offering server-driven UI, OS-level sandboxing, RBAC, and triple-layer audit for production-grade AI applications. The Python-based framework targets environments with compliance and operational constraints, providing an integrated runtime with pluggable model orchestration and multi-tenancy.", "body_md": "A Python framework for building agentic AI applications with server-driven UI, native observability, OS-level sandboxing, pluggable model orchestration, and a strict extension boundary — designed for environments where reproducibility, auditability, and operational control matter.\n\nStatus: Beta 0.0.1b4 — public preview. The core architecture is implemented and actively tested, but APIs and operational behavior may still change before the first stable release. The software ships without warranty.\n\n- Website:\n[democr.ai](https://democr.ai) - Documentation:\n[democr.ai/docs/](https://democr.ai/docs/)\n\nDemocr.ai is a complete runtime framework for AI applications. It provides:\n\n- Server-driven UI for agentic contexts, implementing and extending Google's A2UI protocol\n- Multi-client rendering of the same UI definition across web and desktop clients\n- Native multi-tenancy enforced at write time\n- Cross-platform OS-level sandboxing for engine and extension processes\n- Triple-layer audit through SQLAlchemy hooks, with sensitive field redaction\n- RBAC with declarative module manifests\n- Pluggable AI engine orchestration across local, remote, and distributed runtimes\n- Engine quotas for request and token governance by engine and subject scope\n- Knowledge subsystem with vector and graph backends\n- A strict modular architecture where everything — including authentication — is a module built against the public SDK\n\nThe framework is designed around a clear boundary between core runtime and extensions. The SDK is the only supported way to build on top.\n\nMost AI application stacks combine a chat UI, an LLM client, a workflow library, and separate infrastructure for audit, permissions, sandboxing, and observability. Democr.ai puts those concerns in one runtime contract:\n\n- UI is declared server-side and rendered by independent clients.\n- Modules, engines, and extractors are installable extensions with a public SDK boundary.\n- Model calls flow through an orchestrator that supports local providers, remote providers, process isolation, quotas, and multi-node execution.\n- Security and audit are runtime primitives, not application conventions.\n- Knowledge ingestion, retrieval, media handling, tools, MCP, and agent skills share the same request and observability context.\n\nThat combination is intentionally broad. The project is still beta, but the architecture is already aimed at production-grade constraints rather than demos or single-chatbot prototypes.\n\nLet agents build UI components that render as native application surfaces, not as screenshots or markup fragments. Democr.ai modules describe UI declaratively on the server, then stream structured updates as the agent works.\n\nThe short demo shows the important part: one backend-declared YAML surface reflected through client renderers, while the backend keeps ownership of behavior, actions, and runtime effects.\n\nDemocr.ai targets applications where:\n\n- The deployment environment has compliance, regulatory, or operational constraints\n- Observability and reproducibility are required from day one, not added later\n- Avoiding vendor lock-in is a hard constraint: the framework is fully self-hostable, model-agnostic, and ships no proprietary cloud services\n- Auditability across user actions, data mutations, and AI calls is required at the deployment level\n- The team prefers an integrated runtime over assembling separate pieces (LLM library + UI framework + observability stack + audit + RBAC + workflow engine)\n\nIt is not a thin wrapper, a model router, or a chat UI library. It is a runtime where modules, engines, extractors, and clients compose into a single coherent system.\n\nThere are no privileged components. Authentication, system administration, and the demo applications shipped in this repository are all built as modules, using the same SDK that third-party developers use. There are no reserved APIs, no hidden hooks, and no internal-only services.\n\nThe modules included in this repository (`auth`\n\n, `system`\n\n, `components`\n\n) are primarily examples of how the framework works and how applications integrate with it. They provide a working baseline, but deployments are not required to use them as-is: teams can replace them, extend them, or build their own domain-specific implementations.\n\n- The runtime is fully self-hostable.\n- AI engines are pluggable: local inference (vLLM, llama.cpp) and cloud providers (OpenAI, Anthropic, Google, and others), or custom integrations.\n- Storage backends are pluggable for vector (sqlite-vec, pgvector, and others), knowledge graph, and core persistence (SQLite, PostgreSQL).\n- No proprietary serialization formats. State is portable across deployments.\n\nObservability is not added as later instrumentation — it is part of how the framework expects developers to build. Every action, AI call, and module operation flows through a correlation chain that allows step-by-step reconstruction of any request. Pipelines are observable. The audit subsystem captures mutations at three layers (request, ORM, AI invocation) automatically, with sensitive field redaction.\n\nReproducibility is achieved through deterministic configuration, structured event logs with stable IDs, and an immutable audit trail.\n\nThe framework provides security as foundational primitives, not as opt-in features. See [Threat model](#threat-model) for the precise boundary between framework guarantees and deployer responsibilities.\n\nThe UI layer is an extended implementation of Google's A2UI protocol for server-driven UI in agentic contexts. The same UI definition renders consistently across multiple clients with no client-side framework code required from module authors. New clients can be added without changes to existing modules.\n\nDemocr.ai is composed of distinct subsystems:\n\n**Core runtime**— bootstrap, lifecycle, request handling, dependency graph** Network**— WebSocket, IPC, HTTP under a unified protocol layer** AI engine orchestrator**— job-based scheduling, batching, gRPC process isolation, HITL support** Knowledge**— ingestion queue, projection workers, vector + graph retrieval** Sandbox**— OS-level isolation (Landlock + seccomp + iptables + helper process | seatbelt | Windows Low Integrity + WFP)** Multi-tenancy**— write-time enforcement with materialized scope filters** RBAC**— declarative access policies in module manifests** Modules**— framework extension surface, via SDK** Engines**— pluggable AI inference providers** Extractors**— pluggable knowledge extraction providers** Clients**— independent rendering surfaces for web, desktop, and mobile-oriented clients** Platform**— agents, skills, tools, MCP, UI builder\n\nDetailed subsystem documentation is available at [democr.ai/docs/](https://democr.ai/docs/).\n\n| Capability | Status | Notes |\n|---|---|---|\n| Server-driven UI (extended A2UI) | Implemented | websocket and ipc transport |\n| Multi-client rendering | Implemented | Web, Qt desktop, Tauri, React variants |\n| Multi-tenancy | Implemented | Write-time enforcement |\n| RBAC | Implemented | Declarative manifests |\n| AI engine orchestration | Implemented | Job-based, gRPC isolated |\n| Knowledge subsystem (vector + KG) | Implemented | Hybrid retrieval |\n| Audit (triple layer) | Implemented | Sensitive field redaction |\n| OS sandbox (Landlock + seccomp + iptables) | Implemented | |\n| OS sandbox on macOS | Implemented | Seatbelt |\n| OS sandbox on Windows | Implemented | Low Integrity + WFP; egress enforcement uses an elevated helper |\n| Local AI inference | Implemented | vLLM, llama.cpp, ONNX, Ollama, and others |\n| Cloud AI providers | Implemented | OpenAI, Anthropic, Google, and others |\n| Multi-node engine orchestration | Implemented | Shared queue, node registry, Redis response streams |\n| mTLS for internal gRPC services | Roadmap | Server-side TLS + service JWT first |\n| Context budget guard | Implemented | Heuristic guard for oversized prompt messages and tool outputs, scaled by model context settings |\n| Multi-node tuning | Roadmap | Distributed fine-tuning workflows |\n| Mobile clients | Experimental | React Native client exists but is not certified |\n| Engine quotas | Implemented | Request and token limits by engine and subject scope |\n| Request observability | Implemented | Correlation chain across layers |\n\nThe following numbers are local stress-test snapshots for the authenticated\nrequest path: login, session handling, and server-side rendering of the\nserver-driven UI route `/components/_effects/yaml`\n\n. They document observed\nbehavior under concurrent load on one machine; they are not universal\nperformance claims or comparisons with other frameworks.\n\nEnvironment:\n\n- CPU: Intel Core i9-14900K\n- RAM: 64 GB\n- GPU: NVIDIA RTX 3090 Suprim, not used by this benchmark path\n- OS: Ubuntu 24\n- Python: 3.12.3\n- Providers: remote providers\n\n| Instances | Workers / instance | Clients / worker | Duration | Requests | Errors | Throughput | Avg latency | p95 latency |\n|---|---|---|---|---|---|---|---|---|\n| 4 | 4 | 4 | 303.69s | 468,840 | 0 | 1,543.83 req/s | 33.28ms | 47.05ms |\n| 8 | 4 | 4 | 302.39s | 718,050 | 0 | 2,374.55 req/s | 43.30ms | 47.94ms |\n\nCommand shape:\n\n```\nPYTHONPATH=. .venv/bin/python scripts/multi_core_benchmark.py \\\n  --instances 4 \\\n  --workers 4 \\\n  --clients 4 \\\n  --duration-seconds 300 \\\n  --profile custom \\\n  --profile-log-summary\n```\n\nFor the 4-instance run, aggregate resource usage was:\n\n| Metric | Value |\n|---|---|\n| Avg total CPU | 1326.9% |\n| Peak RSS | 7092.1 MB |\n| Peak USS | 5936.8 MB |\n| Peak PSS | 5986.8 MB |\n\nRSS can overstate standalone memory use because shared pages are counted more than once.\n\nThe framework offers security as foundational primitives: OS-level sandboxing, immutable audit, RBAC, scoped data access, and at-rest encryption of secrets stored in the database. Modules operate within these primitives according to the policies they declare (sandbox allow-lists, RBAC permissions, access scope).\n\nThe threat model covered by the framework is **accidental supply-chain compromise** — a third-party module with a bug or compromised dependency must not be able to harm the system beyond its declared scope.\n\nThe threat model **not covered** is a deliberately malicious module installed consciously by the deployer. The selection of modules to install and the configuration of security policies are deployer responsibilities.\n\nThe framework's `config.yaml`\n\nis plaintext and is generated locally by the setup wizard. It is not designed to be deployed. Secrets are encrypted at rest only when stored in the database.\n\n- Python >=3.12,<3.14\n- Linux is the most tested environment for full security features\n- Optional: NVIDIA GPU with CUDA for local inference\n\nFrom the repository root:\n\n```\n./setup_venv.sh\n\n# alternatively, install and run desktop mode with ./run.sh\n```\n\nFor the React web clients and the Tauri desktop client, install frontend dependencies before running them. Tauri also requires Cargo.\n\n```\nyarn --cwd clients/webclient install\nyarn --cwd clients/reactbootstrap install\nyarn --cwd clients/reactfluent install\nyarn --cwd clients/tauri install\n```\n\nDefault startup is desktop mode with the Qt client:\n\n```\npython main.py\n```\n\nEquivalent explicit form:\n\n```\npython main.py --mode desktop --client qtdesktop\npython main.py --mode server --host 127.0.0.1 --port 8000\n```\n\nDesktop mode uses IPC by default. Add `--http`\n\nto also expose HTTP/WebSocket:\n\n```\npython main.py --mode desktop --http --client qtdesktop\n```\n\nWeb clients are independent React applications. The runner launches them with `yarn dev`\n\n:\n\n```\npython main.py --mode server --client webclient\npython main.py --mode server --client reactbootstrap\n```\n\nWeb clients are independent React applications. The runner launches them with `yarn dev`\n\n:\n\n```\npython main.py --client tauri\n```\n\nOn first run, the framework starts a setup wizard via the connected client. The wizard configures the database, AI providers, and the initial administrator user. The resulting configuration is generated locally and not designed to be deployed.\n\nInitial setup can also run from the core CLI:\n\n```\npython main.py setup path/to/setup.yaml\n```\n\nThe setup YAML contains `admin`\n\ncredentials and the runtime `config`\n\npayload. If `admin.password`\n\nis omitted, the command prompts for it interactively; in non-interactive runs use an environment variable such as `${DEMOCRAI_ADMIN_PASSWORD}`\n\n. The command only runs before an application config exists and will not overwrite an existing installation.\n\n| Command | Purpose |\n|---|---|\n`python main.py` |\nDefault desktop mode |\n`python main.py --mode server` |\nServer mode (HTTP + WebSocket) |\n`python main.py --mode desktop` |\nDesktop mode (IPC transport) |\n`python main.py --http` |\nAlso expose HTTP in desktop mode |\n`python main.py migrate all` |\nRun all pending database migrations |\n`python main.py migration-status all` |\nShow migration status |\n`python main.py create-migration db -m \"desc\" --autogenerate` |\nCreate a new core DB migration |\n`python main.py create-migration data --module <name> -m \"desc\" --autogenerate` |\nCreate a new module data migration |\n`python main.py rollback data --steps 1` |\nRoll back one data migration step |\n`python main.py validate-config` |\nValidate current configuration |\n`python main.py setup examples/setup.example.yaml` |\nRun initial setup from YAML |\n`python main.py install-engines examples/engine-install.example.yaml` |\nInstall and activate engines/models from YAML |\n`python main.py install-extractors examples/extractor-install.example.yaml` |\nInstall and activate extractors from YAML |\n`python main.py reset-install` |\nReset the installation state |\n`python main.py module-status` |\nShow module discovery and load status |\n`python main.py knowledge-rebuild --all --dry-run` |\nRebuild knowledge projections (dry run) |\n\nThe core CLI can install and activate engine instances from a YAML file:\n\n```\npython main.py install-engines path/to/engines.yaml\n```\n\nThe command starts the application runtime, uses the core SDK context, and follows the same engine lifecycle used by the runtime: registry upsert, runtime config check, install, activation, model download when requested, and model binding activation. It does not require the `system`\n\nmodule to be installed.\n\nSupported options:\n\n```\npython main.py install-engines path/to/engines.yaml --reset-mode keep\npython main.py install-engines path/to/engines.yaml --reset-mode selected\npython main.py install-engines path/to/engines.yaml --reset-mode full --yes\npython main.py install-engines path/to/engines.yaml --json\n```\n\nReset modes:\n\n| Mode | Behavior |\n|---|---|\n`keep` |\nKeep existing engines and models; update declared items and skip already available downloads |\n`selected` |\nReset only engines/models declared in the YAML |\n`full` |\nReset all engine registry rows, model bindings, and downloaded model artifacts; requires `--yes` outside interactive confirmation |\n\nYAML shape:\n\n```\nengines:\n  - provider: openai\n    name: openai-main\n    config:\n      api_key: \"${OPENAI_API_KEY}\"\n    models:\n      - id: gpt-5.5\n        activate: true\n\n  - provider: gemini\n    instances:\n      - name: gemini-main\n        config:\n          api_key: \"${GEMINI_API_KEY}\"\n        models:\n          - id: gemini-2.5-pro\n            activate: true\n      - name: gemini-fast\n        config:\n          api_key: \"${GEMINI_API_KEY}\"\n        models:\n          - id: gemini-2.5-flash\n            activate: true\n\n  - provider: ollama\n    instances:\n      - name: ollama-local\n        config:\n          base_url: http://localhost:11434\n        models:\n          - id: llama3.2\n            activate: true\n\n  - provider: llamacpp\n    name: llamacpp-local\n    models:\n      - id: tiny-llama3-test-q2-k\n        download: true\n        activate: true\n```\n\nSensitive config values can reference environment variables with `${NAME}`\n\n. When multiple instances use the same provider, provider installation runs once and each instance is activated separately. A complete provider-oriented example is available at [ examples/engine-install.example.yaml](/democr-ai/democrai/blob/main/examples/engine-install.example.yaml).\n\nThe core CLI can install and activate knowledge extractors, then configure MIME routing:\n\n```\npython main.py install-extractors path/to/extractors.yaml\n```\n\nSupported options mirror `install-engines`\n\n:\n\n```\npython main.py install-extractors path/to/extractors.yaml --reset-mode keep\npython main.py install-extractors path/to/extractors.yaml --reset-mode selected\npython main.py install-extractors path/to/extractors.yaml --reset-mode full --yes\npython main.py install-extractors path/to/extractors.yaml --json\n```\n\nYAML shape:\n\n```\nextractors:\n  - id: docling\n    install_config:\n      ocr_engine: rapidocr\n    runtime_config:\n      ocr_enabled: true\n      chunk_size: 1200\n    mime_bindings:\n      - application/pdf\n      - text/plain\n\n  - id: ai_audio\n    runtime_config:\n      model_registry_id: 12\n    mime_bindings:\n      - audio/mpeg\n```\n\n`ai_audio`\n\nand `ai_image`\n\nrequire a real `model_registry_id`\n\nfor an already configured model with the required capability. If `model_registry_id`\n\nis omitted, the command can still install and activate the extractor, but the runtime configuration is incomplete and extraction will not be usable until that field is saved. A complete example is available at [ examples/extractor-install.example.yaml](/democr-ai/democrai/blob/main/examples/extractor-install.example.yaml).\n\nThe runner sets default extension path variables if they are not already configured:\n\n```\nDEMOCRAI_MODULES_PATH=/path/to/modules\nDEMOCRAI_ENGINES_PATH=/path/to/engines\nDEMOCRAI_EXTRACTORS_PATH=/path/to/extractors\n```\n\nEach variable can contain one or more existing directories. Multiple directories\nuse the platform path separator (`:`\n\non Linux/macOS, `;`\n\non Windows).\n\nAt startup, the core runtime discovers and registers extension packages from these paths. Modules, engines, and extractors are all extension types: modules add application behavior, engines add AI provider/runtime integrations, and extractors add document or media ingestion capabilities.\n\nThe components listed below have working implementations covered by development or integration tests. The list will be refined as engines, extractors, and specific model combinations are certified for stable releases.\n\nNote: a complete and versioned list of tested models, engines, and extractors will accompany the first stable release. Provider implementations may exist in the repository before they are certified. Components not listed here should be treated as available implementation work, not as a stable support promise.\n\n| Engine | CPU/GPU | Status |\n|---|---|---|\n`onnx` |\nCPU | Implemented |\n`llamacpp` |\nCPU/GPU | Implemented |\n`vllm` |\nGPU | Implemented |\n`parler` |\nGPU | Implemented |\n`qwen_tts` |\nGPU | Implemented |\n`whisper` |\nGPU | Implemented |\n`espeak` |\nCPU | Implemented |\n`rebel` |\nGPU | Implemented |\n`gliner+glirel` |\nGPU | Implemented |\n`ollama` |\nGPU (local) | Implemented |\n`openai compatible` |\n- | Implemented |\n`openai` |\n- | Implemented |\n`gemini` |\n- | Implemented |\n`anthropic` |\n- | Implemented |\n`openai whisper` |\n- | Implemented |\n`openai tts` |\n- | Implemented |\n`edge tts` |\n- | Implemented |\n`nvidia_nim` |\n- | Implemented |\n\n| Extractor | Tested with | Status |\n|---|---|---|\n`docling` |\nTesseract/RapidOCR | Implemented |\n`ai_audio` |\nWhisper (faster whisper) | Implemented |\n`ai_image` |\nGemma 4 26B A3B | Implemented |\n\n- SQLite (default, recommended for development)\n- PostgreSQL (recommended for production multi-tenant deployments)\n- sqlite-vec (vector, default)\n- pgvector (vector, when PostgreSQL is in use)\n\n- Pinecone\n- ClickHouse\n\n`qtdesktop`\n\n— PySide6-based desktop client`webclient`\n\n— React web client`reactbootstrap`\n\n— React web client with Bootstrap/AGID theme`tauri`\n\n— Tauri-based desktop client\n\nAdditional client implementations exist in the repository, including `reactfluent`\n\n, `electron`\n\n, and `reactnativeclient`\n\n, but are not part of the certified client set yet.\n\n`main.py`\n\n— runner for local development and packaged launcher flows`pyproject.toml`\n\n— build metadata for the`democrai`\n\nPython package`democrai/core/`\n\n— internal runtime: bootstrap, storage, network, sandbox, orchestration, registries`democrai/sdk/`\n\n— public SDK boundary for extensions (the only supported import surface for modules)`democrai/third_party/`\n\n— bundled runtime assets (e.g.,`libmagic`\n\n)`modules/`\n\n— bundled demo modules (`auth`\n\n,`system`\n\n,`components`\n\n)`engines/`\n\n— AI engine implementations and manifests`extractors/`\n\n— knowledge and media extractor implementations`clients/qtdesktop/`\n\n— Qt desktop client`clients/webclient/`\n\n— React web client`clients/reactbootstrap/`\n\n— React web client with Bootstrap/AGID theme`clients/reactfluent/`\n\n— React web client variant`clients/tauri/`\n\n— Tauri-based desktop client`clients/electron/`\n\n— Electron wrapper for web clients`clients/reactnativeclient/`\n\n— React Native client`docs_site/`\n\n— MkDocs documentation pipeline`tests/`\n\n— Python test suites\n\nModules may import only from `democrai.sdk`\n\n. Imports from `democrai.core`\n\nare not part of the supported boundary and locked by module runtime.\n\nFull documentation is built with MkDocs:\n\n```\nsource .venv/bin/activate\n.venv/bin/mkdocs build -f docs_site/mkdocs.yml --strict\n```\n\nGenerated output is in `docs_site/docs_html/`\n\n.\n\nNode tooling for documentation post-processing is managed inside `docs_site/`\n\n:\n\n```\ncd docs_site\nnpm install\n```\n\nThe items below are planned for upcoming releases, in approximate order:\n\n**Multi-node tuning**— distributed fine-tuning workflows** Client certification**— broaden the certified set beyond`webclient`\n\n,`reactbootstrap`\n\n,`qtdesktop`\n\n, and Tauri-based desktop**Mobile clients**— complete and certify the React Native client** Knowledge subsystem improvements**— additional projection backends, refined retrieval, expanded extractor coverage** And many more**— see the issue tracker for the full backlog and active proposals\n\nRun the full framework test suite:\n\n```\n.venv/bin/python -m pytest -c tests/pytest.ini\n```\n\nRun the runtime subset:\n\n```\n.venv/bin/python -m pytest -c tests/pytest.ini tests/runtime\n```\n\nWith coverage:\n\n```\n.venv/bin/python -m pytest -c tests/pytest.ini --cov-config=tests/.coveragerc --cov=democrai tests/runtime\n.venv/bin/coverage html --rcfile=tests/.coveragerc\n```\n\nSome tests depend on external engines or services. Verify the local environment before running broader suites.\n\nDemocr.ai uses a split licensing model:\n\nis licensed under the`democrai/core/`\n\n**GNU Affero General Public License v3.0 (AGPL-3.0)**. Modifications to the core that are made available over a network must be released under the same license.are licensed under the`democrai/sdk/`\n\n,`engines/`\n\n,`extractors/`\n\n,`modules/`\n\n, and`clients/`\n\n**Apache License 2.0**. You can build, ship, integrate, and commercialize extensions, engines, extractors, modules, and clients freely, subject to attribution.\n\nIf your use case requires a commercial license for the core (for example, proprietary modifications without source disclosure), contact `license@democr.ai`\n\n.\n\nThe repository root [ LICENSE](/democr-ai/democrai/blob/main/LICENSE) file describes the split and lists the directory-specific licenses. Full license texts are in the corresponding directories (\n\n`democrai/core/LICENSE`\n\nfor AGPL-3.0, `democrai/sdk/LICENSE`\n\nfor Apache-2.0).The A2UI protocol is a Google open specification for agentic UI. Democr.ai extends it to make the whole application server-driven and potentially controllable by agents.\n\nThis project builds on the work of the open-source Python ecosystem, including SQLAlchemy, FastAPI, Alembic, Pydantic, gRPC, Qt, and many others.", "url": "https://wpnews.pro/news/show-hn-democr-ai-self-hosted-agentic-ai-runtime-with-audit-and-rbac", "canonical_source": "https://github.com/democr-ai/democrai", "published_at": "2026-07-16 15:13:52+00:00", "updated_at": "2026-07-16 15:25:08.942746+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "ai-safety", "ai-policy", "developer-tools"], "entities": ["Democr.ai", "Google", "SQLAlchemy"], "alternates": {"html": "https://wpnews.pro/news/show-hn-democr-ai-self-hosted-agentic-ai-runtime-with-audit-and-rbac", "markdown": "https://wpnews.pro/news/show-hn-democr-ai-self-hosted-agentic-ai-runtime-with-audit-and-rbac.md", "text": "https://wpnews.pro/news/show-hn-democr-ai-self-hosted-agentic-ai-runtime-with-audit-and-rbac.txt", "jsonld": "https://wpnews.pro/news/show-hn-democr-ai-self-hosted-agentic-ai-runtime-with-audit-and-rbac.jsonld"}}