# Fruth – A semantic state inference engine for AI > Source: > Published: 2026-11-01 10:14:07+00:00 Intent opens a possibility space. Fruth records candidates and uses relevance and validated promotion to determine which work is owed. Its interpretive inference layer can infer meaning from evidence and propose interpretations, work and reviews; runtime contracts, validation and evidence govern accepted transitions. **Model inference generates token continuations; Fruth derives required work, permitted state transitions and fulfillment from the request, validated contracts and recorded evidence.** Recorded frames and artifact evidence support inspection and continuation of work. Mutable work, frozen frames and append-only successors remain distinct. Durable means supported persistence and continuation of recorded state; it does not mean infallible conclusions or guaranteed survival of every in-flight operation. **Status:** Fruth `0.1.9` — experimental, usable, and still evolving within the `0.x` series. **Conceived, designed, created, built, and developed by [@fl0ri0](https://github.com/fl0ri0).** **Independent project.** Fruth is built and maintained by one person, not a company, research lab, or funded team. It is human-led and AI-enhanced — a vibe-coded project in the literal, accountable sense. Product direction and release decisions remain human; AI tools helped explore, implement, test, and document the system. The project began with ideas developed over time before taking shape in early terminal scripts. It grew from a prototype whose small changes were still copied by hand into the current runtime substrate. Previously released as Ollmo, it continues as Fruth. This release does not imply institutional backing, a formal security audit, or a support organization. Fruth combines a local browser interface and control plane with local model backends, interpretive inference routing, durable response frames, history, and materialized artifacts. Its core loop is: ``` php Intent -> possibility space -> relevance -> promoted contracts -> runtime work -> review -> frozen state ``` A response is a frozen truthful moment of work: what was requested, what became owed work, what runtime evidence exists, what remains pending or blocked, and which outputs or artifacts can be referenced later. Fruth uses a consistent vocabulary to connect intent, execution, evidence, and durable state. These terms give developers and AI coding agents shared reference points across code, tests, and documentation. When reviewing the architecture, consider whether each term captures a useful distinction and is applied consistently—not only whether it is immediately familiar. Run `./fruth self-attack` for adversarial conformance across discovered interpretive inference and runtime control profiles. It writes a human-readable report and JSON results; see [self-attack conformance](https://github.com/fl0ri0/fruth/blob/main/docs/SELF_ATTACK.md) for coverage, live execution, budgets and regression replay. The [October 31 validation status](https://github.com/fl0ri0/fruth/blob/main/docs/SELF_ATTACK_STATUS_2026-10-31.md) records the bounded property-based investigation, responsibility refactoring, process and PDF corrections, and their source-bound checks. Release validation records exact dependencies and source identities. Earlier local, AFM and PCC campaigns and reference workflows retain their original execution dates and scope. Five [reference workflows](https://github.com/fl0ri0/fruth/blob/main/examples/README.md) preserve exact prompts, completed frames, monitor records and verified saved artifacts. The image-review and saved-file examples are from October 29; the other three are from October 27. See the report for [historical results](https://github.com/fl0ri0/fruth/blob/main/docs/SELF_ATTACK_STATUS_2026-10-31.md#historical-results) and the distinction between checked runtime contracts and model-answer accuracy. If you use Fruth, its software design, or its evaluation artifacts in research or a published system, cite the software using [`CITATION.cff`](https://github.com/fl0ri0/fruth/blob/main/CITATION.cff). Identify the exact version and commit or local snapshot used. Reference runs preserve their exact recorded execution identities and artifact evidence. The intent-preserving block-resolution principle means that a block is not permission to rewrite the request. Its solution is its own verified resolution, handled gently through the substrate: keep intent anchored, keep the obligation visible, record the runtime evidence, choose the right-sized state transition, and freeze truthful state rather than the desired story. See [Principle 10](https://github.com/fl0ri0/fruth/blob/main/docs/PRINCIPLES.md#10-resolve-blocks-by-preserving-intent). Within one canonical response, Fruth can route different promoted branches independently to compatible model instances and capabilities, materialize their distinct outputs through Late Fill, and bind each evidenced result back into the same response state. This is capability- and availability-dependent; it does not imply that every response uses more than one instance. For research collaboration, replications, talks, panels, or work around Fruth, use the public contact methods on [@fl0ri0's GitHub profile](https://github.com/fl0ri0), with no guaranteed response time. The source is provided for inspection, use, research, and independent forks. The maintainer is open to collaboration and is still learning the issue, pull-request, and review process. Please bring clear explanations and patience; responses, reviews, merges, and ongoing support cannot be guaranteed. See [`CONTRIBUTING.md`](https://github.com/fl0ri0/fruth/blob/main/CONTRIBUTING.md) for participation guidance. Useful public context: - [Vision Alignment](https://github.com/fl0ri0/fruth/blob/main/docs/VISION_ALIGNMENT.md) - [State Substrate Architecture](https://github.com/fl0ri0/fruth/blob/main/docs/diagrams/fruth-state-substrate-architecture.html) - [Verified Reference Runs](https://github.com/fl0ri0/fruth/blob/main/examples/README.md) - [Project Participation and Capacity](https://github.com/fl0ri0/fruth/blob/main/CONTRIBUTING.md) The primary tested environment is a recent macOS release on Apple Silicon with Python 3.11 or newer. Local capabilities require their corresponding Ollama, MLX, llama.cpp, or Apple Foundation Models backend and model. Windows, Linux, Intel Mac, remote hosting, and multi-user operation are outside the 0.1.9 support promise. Extract the source archive, enter its directory (`fruth-0.1.9` for the release, or `fruth-main` for the current repository ZIP), and run: ``` ./fruth start ``` The executable launcher creates `.venv` and installs `requirements.txt` on first start. The selected `python3` must be Python 3.11 or newer linked to OpenSSL 1.1.1 or newer; Apple's older Python/LibreSSL is unsuitable. Choose a suitable installed interpreter on PATH before first start, for example with Homebrew Python 3.11: `PATH="$(brew --prefix python@3.11)/libexec/bin:$PATH" ./fruth start`. Internet access is normally required for the initial dependency installation. Backend runtimes, model weights, Apple Intelligence availability and installed PCC Shortcuts are separate prerequisites for their respective capabilities. Backend runtimes and model weights are not bundled. For Apple PCC, choose **Set up** on its Models card or select PCC in the startup list. Fruth opens the bundled shortcuts on the Mac; review and click **Add Shortcut**, then start PCC. Manual import is also available in the [PCC setup guide](https://github.com/fl0ri0/fruth/blob/main/fruth_integrations/shortcuts/README.md#import). Both signed shortcut exports and assisted setup are included in the source tarball. Fruth startup initializes missing Research stores automatically and preserves existing candidates, reviews and Gold data. No separate Research startup command is needed. See the [Research guide](https://github.com/fl0ri0/fruth/blob/main/fruth_research/README.md) for candidate review, Gold curation and evidence retention. `./fruth start` runs model selection and can create the environment itself. Press Enter at model selection to continue without starting a model. Startup starts the Flask webserver in the background, or reuses a verified webserver from this checkout on port 5011, and prints the dashboard URL. Verification checks the exact Python/script command, working directory, process birth identity and listener PID. Another service, another checkout, conflicting registry evidence or unavailable identity fails startup without a dashboard URL or a ready claim. A new launch must own the listener too; TCP occupancy alone is insufficient. This is process verification, not an HTTP/application health check. Use `./fruth status` to inspect the stack and `./fruth restart` to run stop followed by start. Refresh an open browser tab after restarting. Startup installs Python requirements only when it creates a new `.venv`. Before model discovery it checks that environment's own Python/TLS and uses pip's offline, non-installing checks for active direct requirements and transitive consistency. An unsuitable PATH Python does not block a valid existing `.venv`. Changed requirements remain usable when satisfied. Missing/broken interpreters, absent `requirements.txt`, incomplete installations and incompatible dependencies stop startup with repair guidance; startup never installs into, upgrades, deletes or rebuilds an existing environment. Keep model-backend environments separate; updating the main `.venv` does not update MLX, audio or native backend executables. Preserve your existing `state/`, `artifacts/`, Research data and settings; the release contains empty defaults for new installations, not a replacement for your saved data. While Fruth is stopped, explicitly apply declared requirements to a valid existing environment (also after an interrupted first installation): ``` .venv/bin/python3 -m pip install -r requirements.txt .venv/bin/python3 -I -B scripts/bootstrap_preflight.py --venv .venv --requirements requirements.txt ``` Preflight requires pip >=22.2 for its supported `install --dry-run` CLI, paired with `pip check`. If pip is absent, repair it explicitly with `.venv/bin/python3 -m ensurepip`; if it is too old, explicitly install a supported pip with `.venv/bin/python3 -m pip install 'pip>=22.2'`. Startup does neither. Checks disable pip configuration, index access, caches and version lookups. The startup manifest supports one named distribution per line, optional version constraints and PEP 508 markers, blank lines and comments. URLs, local projects, archive paths, extras, includes, continuations and pip options are rejected before pip: dry runs of arbitrary projects can execute build hooks. These are metadata checks, not package import, native PDF, model or backend qualification. For a missing/broken or incompatible interpreter, first restore its original base Python installation if possible. If replacement is necessary, stop Fruth, retain the old environment at a unique backup path, and create a new `.venv` explicitly with a suitable installed interpreter, for example: ``` # Only when .venv.previous does not exist; keep it for recovery. mv .venv .venv.previous "$(brew --prefix python@3.11)/bin/python3.11" -m venv .venv .venv/bin/python3 -m pip install -r requirements.txt .venv/bin/python3 -I -B scripts/bootstrap_preflight.py --venv .venv --requirements requirements.txt ``` Omit the move when `.venv` is absent. A moved venv is a backup, not a runnable environment: restore it to its original path with its matching base interpreter to roll back. Missing source files must be restored from the matching source archive. Start Fruth explicitly after the checks pass; preflight does not start it. Existing JSON response histories require explicit migration before ordinary writes. Finish active work, stop all response writers, and verify a complete backup before running these commands from the updated checkout: ``` .venv/bin/python scripts/migrate_response_frame_index.py migrate --frames-dir state/response_frames --check-only .venv/bin/python scripts/migrate_response_frame_index.py migrate --frames-dir state/response_frames --writers-stopped .venv/bin/python scripts/migrate_response_frame_index.py check --frames-dir state/response_frames ``` Resume writers only after verification succeeds. Migration preserves the canonical Ledger, snapshots and evidence and selects one SQLite Index; retained JSON is inactive. Already migrated histories need no repeat migration. New installations initialize SQLite through the frame writer. The same tool provides explicit `rebuild` and current-history `rollback` operations; see the [Response Frame contract](https://github.com/fl0ri0/fruth/blob/main/docs/RESPONSES_CONTRACT.md#response-frame). Fruth's default backend ports are 11434 (Ollama), 11435–11500 (dedicated Ollama), 11501–11550 (MLX), 11551–11600 (llama.cpp), and 11601–11650 (Apple Foundation Models). The optional Apple PCC Shortcuts adapter uses 11651–11700. Apple AI supports local text and image analysis through the installed `fm serve` with the API model `system`; multiple instances appear separately in the existing UI. See the Apple AI **Image Mode** control for explicit OCR and barcode/QR recognition through the installed CLI tools, with verified tool-result receipts. See the [Apple AI guide](https://github.com/fl0ri0/fruth/blob/main/docs/APPLE_FOUNDATION_MODELS.md) for prerequisites, metadata, streaming and the observed interpretive-inference context limitation. **Apple PCC** is also available as a startable chat and vision instance and an existing preferred-II choice. It automatically prefers Cloud Pro and falls back to Cloud on confirmed Pro access/availability errors. It sends task text, attached images and selected context to Apple, including the full Fruth policy for interpretive inference. Direct single-instance chat has no automatic Fruth policy or role. PCC is separate from local Apple AI. See [PCC setup and metadata](https://github.com/fl0ri0/fruth/blob/main/fruth_integrations/shortcuts/README.md). The model registry remains local to the Fruth checkout. The webserver log is `logs/flask_webserver.log`. `./fruth stop` (also `shutdown` or `down`) runs the shutdown sequence: stop the Flask webserver, collect registered PIDs and listeners in the backend port ranges, stop those processes and the default Ollama server, then finalize runtime status and archive session logs. Apple AI and PCC use ownership-checked stopping; an unverified Apple bridge process preserves recovery state and makes shutdown fail. It prints the process/port diagnostics in the terminal. An already-stopped stack is a successful stop, so `restart` continues into startup normally. External provider projections are refreshed only by an explicit `./fruth sync`. Open the local interface: ``` ./fruth dashboard ``` The dashboard is served directly at `http://127.0.0.1:5011/`; `/dashboard` remains a compatibility alias. The separate static project page is available at `http://127.0.0.1:5011/site/` while Fruth is running, or by opening `site/index.html` directly from the checkout. The self-contained `site/` directory is also the GitHub Pages publication artifact. It uses only relative, bundled assets and is not the live control surface. Useful lifecycle commands: ``` ./fruth status ./fruth shutdown ``` To inspect locally runnable models and start one chat model explicitly: ``` ./fruth ctl models list --json --runnable-only ./fruth ctl start --model "" --backend "" --capability chat --json ``` Use an identifier and backend reported by the first command. Fruth does not bundle model weights. The release includes a dedicated Codex skill that teaches Codex how to inspect Fruth runtime truth, use interpretive inference routing safely, and work with canonical responses and artifacts. During broader Codex work, the skill also makes already-running local text, image, vision/OCR, speech, embedding, and multimodal capabilities available as optional bounded tools while Codex remains the supervisor. From the extracted Fruth directory: ``` FRUTH_CODEX_SKILLS_DIR="${CODEX_HOME:-$HOME/.codex}/skills" mkdir -p "$FRUTH_CODEX_SKILLS_DIR" if [ -e "$FRUTH_CODEX_SKILLS_DIR/fruth" ]; then printf 'Fruth skill already exists at %s\n' "$FRUTH_CODEX_SKILLS_DIR/fruth" else cp -R skills/fruth "$FRUTH_CODEX_SKILLS_DIR/fruth" fi ``` Restart Codex, open the extracted Fruth directory as the active workspace, and then mention Fruth normally or invoke `$fruth` explicitly. When working from a different workspace, set `FRUTH_HOME` to the extracted Fruth root or provide the checkout path when the skill asks. Installing the skill does not start Fruth, change Codex model-provider settings, or enable cloud routing. The maintained source is `skills/fruth/`; an installed copy is a distribution, not a second authority. For an explicitly requested update, compare source and installed files, preserve any local changes in a protected recovery location, then copy the package contents into the existing installed `fruth/` directory using the same copy-based installation process. Compare every shipped file after copying, including `NOTICE`, `agents/openai.yaml` and references. Do not silently overwrite divergent local customizations or switch discovery directories. The entrypoint retains essential guardrails; `references/fruth-contract.md` is loaded for the relevant request/result recipe. Detailed contracts are resolved from the selected checkout via `FRUTH_HOME` or the documented discovery process, so starting Codex in another directory does not break those references. Repository contributors should begin with [CONTRIBUTING.md](https://github.com/fl0ri0/fruth/blob/main/CONTRIBUTING.md) and the architecture/contract reading map below; the operator skill remains separately usable. Reload/restart the client as required by its instruction-discovery behavior; an existing task's context is not proof of a fresh load. Fruth runs locally by default. It can optionally use ChatGPT as an external target through Codex after the user reviews the cloud disclosure and explicitly enables the connection. Fruth first looks for the Codex executable bundled with the ChatGPT app and then falls back to a separately installed `codex` executable. The integration reuses the login owned by that executable; Fruth does not read, copy, or store the credentials and does not set a fixed model. Once enabled, ChatGPT appears as a clearly marked cloud tab, as an explicit Fruth target, and as a preferred (primary or fallback) interpretive inference target. It can be turned off again at any time. Fruth explicitly supplies the current prompt, context it promotes for that turn, and selected local files or Fruth artifacts for OpenAI processing. Input staging does not confine all downstream reads: the requested read-only command sandbox restricts writes, while the Codex client and enabled tools have their own access boundaries. See the [security boundary](https://github.com/fl0ri0/fruth/blob/main/SECURITY.md#chatgpt-through-codex-and-cloud-processing). Recognized images use Codex's native image-input path. Other selected regular files are copied under neutral names into a temporary working directory, with byte counts and digests recorded. One request can include at most 5 files, no more than 100 MiB per file and 250 MiB in total. URLs, folders, and direct leaf symbolic links are rejected; regular files reached through symlinked parent directories are accepted. ChatGPT returns text through this route; Fruth does not guarantee that every regular-file format can be interpreted semantically by the selected model and its available tools. The ChatGPT target never appears as a running local instance and has no Start, Stop, Pull, or Delete actions. Direct tab turns are ephemeral and independent; Fruth can promote bounded relevant context for a referential turn, but does not create a persistent provider session. Model selection is automatic, and the exact GPT variant is not exposed to Fruth; model self-descriptions are not runtime proof. Direct API-key management is outside this route's 0.1.9 contract. Apple PCC is a separate optional integration described above. Fruth's own source code and project documentation are licensed under the [Apache License 2.0](https://github.com/fl0ri0/fruth/blob/main/LICENSE), unless a file says otherwise. Bundled and optional third-party components retain their own licenses; see [`NOTICE`](https://github.com/fl0ri0/fruth/blob/main/NOTICE) and [` THIRD_PARTY_NOTICES.md`](https://github.com/fl0ri0/fruth/blob/main/THIRD_PARTY_NOTICES.md). Project repository: [GitHub](https://github.com/fl0ri0/fruth). Release tooling builds and verifies local archives; it does not publish or upload them. Canonical artifact storage lives under `artifacts/`: generated files are written to `artifacts/`, saved request inputs to `artifacts/inputs/`, and audit/report-style outputs to `artifacts/audits/`. Fruth separates model proposals from runtime authority. Its interpretive inference layer anchors current intent and derives candidate work. Promotion turns selected candidates into explicit obligations. The resolver and Late Fill materialize those obligations through compatible runtimes, and Closure Review compares the result with runtime and artifact evidence before the state can freeze. The mutable `working_frame` records the fluid pre-freeze state, including the possibility space, candidate graph, promotion review, branch-local work, and closure state. The frozen `response_frame` preserves the resulting request, route, runtime evidence, outputs, artifacts, and unresolved work. Canonical frames are appended to `state/response_frames/responses.jsonl`; normal bounded wire views expose handles and content-addressed references, while explicit truth views restore the exact referenced state. The [semantic core overview](https://github.com/fl0ri0/fruth/blob/main/docs/ARCHITECTURE_MAP.md#semantic-core) connects these mechanisms: preserve required meaning, respect each operation's authority and contract, and support claims with evidence for the exact subject, version and scope. End-to-end checks follow the obligation across handoffs and recovered views. Current code ownership is split across: - `fruth_webserver.py` for the Flask API/UI composition root; - `fruth_server/` for request, routing, execution, transport, and Late Fill runtime owners; - `fruth_services/` for response frames, artifacts, history, inference, and related product services; - `fruth_inference/` for current-turn semantic interpretation and graph derivation; - `fruth_orchestration/` for mutable pre-freeze working state; - `fruth_runtime/` and`fruth_core/` for backend lifecycle and shared runtime truth; - `fruth_integrations/` for external-client adapter infrastructure and the current Codex integration boundary; and - `static/ui/` plus`fruth_webUI.html` for the browser control surface. Reference docs: `fruthctl` is the common CLI adapter layer over the current Fruth Flask control plane. Use it when you want external or user-built wrappers, shells, and scripts to call one stable command surface instead of reaching into startup scripts or raw ports. The bundled Codex skill is the validated 0.1 agent integration. Entry point: - `python3 scripts/fruthctl.py ...` Base URL resolution: - `--base-url http://127.0.0.1:5011` - or environment variable `FRUTHCTL_BASE` - fallback: `FRUTH_WEB_BASE` - default: `http://127.0.0.1:5011` Local runtime recovery order: - `fruthctl` talks to the current Fruth Flask control plane on`127.0.0.1:5011` . - First dependency: if local runtime commands hit `connection refused` on`5011` , check process/listener evidence and the client permission boundary before concluding that the control plane is down. - Read-like `fruthctl` commands such as`instances list` ,`models list` ,`inference` , and`responses get` do not recover/start the control plane by default. Pass`--recover-control-plane` only when that recovery behavior is intentional. - Mutating or lifecycle-oriented `fruthctl` commands may auto-attempt a one-shot local recovery by starting`fruth_webserver.py` and retrying once for the default local base URL. - If `5011` is up but`instances list --json` is empty, start the required model with`fruthctl start ...` . - `doctor runtime` stays diagnostic and reports the broken state instead of auto-starting anything. - If the one-shot recovery still fails, run `./fruth start` for the full local stack and inspect`logs/flask_webserver_auto.log` . Core examples: ``` python3 scripts/fruthctl.py models list --json python3 scripts/fruthctl.py instances list --json python3 scripts/fruthctl.py inference --json python3 scripts/fruthctl.py doctor runtime --json python3 scripts/fruthctl.py start --model mlx-community/Qwen3.5-27B-4bit --backend mlx --capability chat --json python3 scripts/fruthctl.py send mlx-community__Qwen3.5-27B-4bit-mlx-11502 "hello" python3 scripts/fruthctl.py send mlx-community__whisper-large-v3-mlx-mlx-11501 --file "/absolute/path/to/audio.mp3" --json python3 scripts/fruthctl.py send mlx-community__Qwen3.5-27B-4bit-mlx-11502 "hello" --truth-json python3 scripts/fruthctl.py responses get resp_abc123 python3 scripts/fruthctl.py responses get resp_abc123 --json python3 scripts/fruthctl.py responses get resp_abc123 --truth-json python3 scripts/fruthctl.py events list --category infer --status started --json python3 scripts/fruthctl.py events tail --category infer --instance-id mlx-community__whisper-large-v3-mlx-mlx-11506 python3 scripts/fruthctl.py wait mlx-community__whisper-large-v3-mlx-mlx-11506 --json python3 scripts/fruthctl.py history infer --limit 20 --json ``` Most important command: - `send` - targets the canonical `/api/responses` endpoint - lets Fruth decide internally whether the request becomes chat or infer/capability dispatch - defaults to a long timeout suitable for STT/TTS and other long-running infer requests - `--json` performs one bounded POST and prints its public wire envelope - `--truth-json` performs that POST, fetches the same id through exact`view=truth` , and prints a normalized canonical summary - targets the canonical - `responses get` - reads canonical `/api/responses/?view=truth` lookup state - default output summarizes `lifecycle_state` ,`status_semantics` , frame identity, output provenance, work-tree authority, and durability - `--json` preserves the raw canonical truth-view payload, while`--truth-json` emits only the normalized response-truth summary for scripts - `--json` and`--truth-json` are mutually exclusive on both`send` and`responses get` - reads canonical - `wait` - polls Fruth event history until the latest matching request for an instance reaches a terminal state - `events tail` - polls and prints new matching event log entries as they appear - `inference` - reads Fruth's runtime-intelligence summary with recommended defaults, current issues, and recovery hints `fruth_inference` is the package boundary for the self-describing runtime-intelligence layer inside Fruth. Its runtime identity remains `fruth-inference`. It does not orchestrate agents. It explains and uses the runtime: - what is running - which instance is the current best default for each capability - what problems are active - which recovery commands should be tried next - which live capability/instance best fits an Auto request according to merged runtime truth rather than a hardcoded provider preference Current Auto routing behavior: - `Auto` routes from merged control-plane truth, not a fixed provider table - The interpretive inference layer, resolver transformation, control-hint filling, and late fill are separate responsibilities: - the interpretive inference layer chooses the next truthful phase and compatible runtime target - Fruth can then fill truthful request fields such as language, speaker, format, style, or image aspect controls for that chosen target - downstream phases can continue through the resolver + late fill under the same request/response identity - local backend model calls materialize selected phases or branches; runtime code decides fulfillment from graph, slot, output, artifact, and status truth - the request phase graph is now the shared substrate for interpretive inference-owned requests: - every interpretive inference-owned request freezes into a request phase graph - `candidate_graph` records possible outputs, workload tasks, context, references, evidence, repairs, continuations, and learning hints - `promotion_review` decides what becomes executable owed work and what remains reserved, waived, rejected, stale, or merely possible state - branch-local workload tasks carry focused payloads, prompts, dependencies, artifact refs, evidence, output contracts, visibility, review criteria, and freeze state - deterministic `review_criteria` stay runtime/closure checks; only explicit`semantic_review_criteria` or non-deterministic qualitative criteria require semantic review - the graph is frozen in intent and fluid in state - plain chat may end at phase 1 - image/audio materialization defaults to `chat -> downstream branches` - downstream phases remain explicit instead of being flattened into one capability - later branches should consume branch-local task truth rather than replaying the full root prompt, and missing generated input artifacts should become `repair_dependency_chain` instead of blind same-branch retry - canonical `/api/responses` now runs one bounded pre-freeze closure review against the frozen graph/work-tree/artifact truth: - the review is exposed as `runtime.graph_closure_review` plus developer diagnostics where applicable - it should continue resolver/late fill work for still-missing frozen obligations - it must not reinterpret the user request or invent new semantic goals - the review is exposed as - live interpretive inference route context is intentionally short and direct: - current turn stays dominant - fresh turns normally use `current_turn_only` context strategy - recent thread context is injected only for clearly referential or continuation-like turns - when that thread context is needed, the interpretive inference layer sees one answered prior user turn plus its direct assistant reply window - artifact continuity is output-centered: all artifacts from the last relevant assistant artifact message, then latest-by-type fallback - old tool calls and old artifacts do not become new intent just because they happened earlier - explicit stable preferences are explicit-only; live routing does not carry a separate derived `memory` block - Auto preview and final execution share the same resolver/control-hint/runtime path so the UI sees the same required-field truth the backend will execute - Auto route preview and selection are read-only planning surfaces. They must not start models, load providers, or turn route choice into runtime load feedback. Model lifecycle starts require an explicit lifecycle source such as a frontend play/start action. - embedding helpers remain internal interpretive inference tools; they can attach routing-context hints and narrow tie-break signals for anchored follow-ups, and those audits remain visible in runtime metadata - explicit low-level direct contracts such as direct `instance_id` requests or`batch_prompts` remain explicit exceptions; they are not the normal interpretive inference path - The interpretive inference layer now exposes file-backed semantic roles as advisory decision lenses. Any valid `fruth_inference/semantic_roles/*.md` role definition becomes part of the role catalog. - `semantic_role_ids` accepts canonical role IDs, such as`["repairer", "evidence_reasoner"]` , as advisory orientation only. - `semantic_role_profile` can travel through request meta, route preview, working frames, and canonical Responses runtime payloads as advisory orientation only. - `FRUTH_INFERENCE.md` is the primary runtime-policy anchor the interpretive inference layer uses for current-turn interpretation; the backend now injects it into both interpretive inference routing decisions and interpretive inference-owned user-facing chat turns, while`FRUTH_FOR_AGENTS.md` remains the operator/client guide - visible assistant text must not claim `saved locally` ,`artifact created` , or similar file/artifact success unless runtime truth actually contains that saved output Surfaces: - `GET /api/inference` - `GET /api/agent_contract` - `python3 scripts/fruthctl.py inference` - [FRUTH_INFERENCE.md](https://github.com/fl0ri0/fruth/blob/main/FRUTH_INFERENCE.md) - [FRUTH_FOR_AGENTS.md](https://github.com/fl0ri0/fruth/blob/main/FRUTH_FOR_AGENTS.md) Frontend note: - the Responses workbench can now use `Auto` to route prompts and file-backed requests by capability instead of forcing one fixed target instance - the UI label is `Auto` , while the backend currently keeps the internal`inference_route` /`inference_preview` names for this routing path - Interpretive inference `primary_target` /`fallback_target` preferences remain interpretive inference and chat-routing hints; they do not override non-chat execution when the request resolves to image, TTS, or another incompatible capability - Explicit frontend play/start is the user lifecycle path for starting an additional instance. The frontend sends `start_source: "frontend_button"` and sends`force_start: true` only when deliberately starting another same-model instance; the interpretive inference layer, route preview, late fill, and backend automatic paths must not use`force_start` . Fruth now exposes one canonical execution endpoint on top of the existing route split: - `POST /api/responses` - `POST /v1/responses` Use this when you want one stable request/response language for execution calls. The older specialized routes still exist: - `/api/chat` - `/api/infer` but the canonical path is now `responses`, and first-party UI flows should stay on it. This is the official external contract for wrappers, agents, CLIs, editors, and new integrations. Effectful HTTP calls require `X-Fruth-Action: 1`, including multipart, streaming and explicit-refresh GETs. The UI and shipped native clients emit it; older clients and existing Fruth-routed provider configurations must add it. Host and exact-Origin admission, passive reads, strict preference JSON errors and the browser/native trust distinction are in the [HTTP caller contract](https://github.com/fl0ri0/fruth/blob/main/docs/HTTP_API_BOUNDARY.md). Prefer `responses` even for file-backed, OCR, image, STT, and TTS work. `/api/responses` now executes shared internal helpers directly; external callers should not need to pick `/api/chat` or `/api/infer` unless they intentionally want the older compatibility surface. Reviewed redraw remains part of the same intent-aligned scope ladder. The lower bounded additive repair rungs stay autonomously active under the default-deny graph-repair policy; partial-subtree and full-successor rebase are the higher-risk rungs, with non-mutating `shadow` as their product default rather than a separate shadow layer. Read their canonical evidence gates at `GET /api/graph_rebase/readiness`. The report is observer-only and does not promote work. An operator advances one exact proposal through `POST /api/responses//graph_rebase/operator` in strict order: `adjudicate`, then durable audit-only `stage`, then gate-controlled `authorize_partial`. The last action is compare-and-swap bound to the finalized response/frame/proposal/base/candidate/class identities and joins authority only from Fruth's trusted operator registry. Full durable state and bounded observation truth must identify the same latest frame, and current root truth is rechecked at replay, apply, persistence, and Late Fill. A successful partial authorization appends one branch-local successor frame under the same response id before scheduling its exact Late Fill work; it never rewrites the parent or replays the root prompt through another prompt carrier. Explicit `FRUTH_GRAPH_REBASE_AUTONOMY=off` blocks the `stage` and `authorize_partial` transitions; evidence-only `adjudicate` remains available because it grants no rebase authority. The product stays in `shadow` while evidence gates are not green, and full successor rebase remains non-executable under safe partial v1. The mutating endpoint is dormant unless startup explicitly supplies both a 32-or-more-character `FRUTH_GRAPH_REBASE_OPERATOR_TOKEN` and an exact `FRUTH_GRAPH_REBASE_OPERATOR_IDENTITY`. Calls must present the token as a Bearer token or `X-Fruth-Graph-Rebase-Operator-Token` and the configured identity as `X-Fruth-Graph-Rebase-Operator`; neither value is persisted or inherited by model subprocesses. Runtime itself produces replay confirmation by deterministically revalidating the frozen proposal. Trusted-only or runtime-only stage halves do not count toward promotion. A no-proposal `false_negative` adjudication is evidence-only and uses the exact sentinel `expected_proposal_id=no_formal_proposal`; it cannot become stage or execution authority. It remains historical truth, but a later exact replay-verified `useful_proposal` adjudication may append `resolves_record_id=` so only unresolved false negatives block promotion. Current scope: - supports direct targeting with a running `instance_id` - also supports routed/default targeting layers such as Auto/`inference_route` , wrapper alias/profile, or canonical`capability` when the caller or backend chooses the target - accepts one normalized request body with `input` , optional`instructions` , and optional file context (`file_path` or multipart`file` ) - keeps explicit `input_artifacts` separate from explicit`reference_artifacts` - supports `stream=true` with Responses-style`text/event-stream` events - returns one normalized response envelope with: - `outputs` - `status` - `mode` - `instance_id` - `model` - `output_text` - `artifacts` - `warnings` On the normal non-streaming path, that envelope is serialized from the compact indexed Response Frame after canonical persistence. A live fallback is not a durable-commit receipt; see the [finalization boundary](https://github.com/fl0ri0/fruth/blob/main/docs/RESPONSES_CONTRACT.md#finalization-and-durable-completion). Normal POST/default/UI/status/debug results are byte-budgeted to at most 8 MiB for the complete serialized outer envelope. Indexed POST/default/UI/status projection avoids sidecar hydration; debug may hydrate selected Closure/rebase observations, and exceptional ledger recovery may reconstruct canonical state before projection. Compaction is based on bytes rather than a fixed item limit: ordinary 5,000-character text and 65 tiny records remain inline when they fit, while strings from 256 KiB and collections from 1 MiB may become a bounded preview plus exact count/length, byte size, SHA-256, and an adjacent content-addressed `*_snapshot_ref`. Public lifecycle, frame identity, outputs/slots, artifact handles, and effective content-SHA refs remain on the wire; hydrated Runtime, Working Frame, and duplicate full-frame bodies do not. `GET /api/responses/?view=debug` adds bounded Closure/rebase observation evidence. `view=full`, `view=raw`, and `view=truth` recursively restore exact CAS-backed canonical truth and can be larger than 8 MiB. Canonical hydration validates every authoritative ref recursively and fails with HTTP 409 when a ref is missing, malformed, or corrupt. This keeps full truth by reference without semantic loss. Artifact-bundle operations likewise resolve exact CAS truth and fail closed rather than exporting an incomplete preview. Example: ``` POST /api/responses { "instance_id": "x/flux2-klein:latest-1", "input": "seven frogs posing like a family portrait on the forest floor" } ``` Streaming note: - canonical streaming currently uses Responses-style SSE over the finalized normalized response payload; the final serialized SSE event, including its outer wrapper, obeys the same 8 MiB ceiling - this is already useful for one client language and UI compatibility - for chat-capable Ollama, MLX, llama.cpp and Apple AI runtimes, canonical streaming forwards real backend deltas - Apple PCC through Shortcuts emits a buffered completed answer, not provider token deltas - non-chat capabilities still use completed-response style payloads/synthetic SSE where appropriate - the main UI chat flow now uses canonical responses streaming for ordinary chat turns Fruth should not invent token limits on behalf of models. - app-side guessed token limits have been removed from the active model-manager/UI path - available model payloads should only expose token limits when a runtime or provider actually reports them - downstream sync layers should omit unknown `contextWindow` /`maxTokens` instead of filling in guessed defaults Use `./fruth start` as the operator-facing entrypoint. It delegates to `./start_multi_models.sh` and starts the canonical runtime/UI stack. - Starts model management and the current Flask UI/API. - Starts MLX support when available. - Leaves external-client provider projections untouched; projection is an explicit operator action. The active startup flow is the runtime/UI stack only. External provider projection is an explicit operator action, not part of start/stop/restart. When you want a local green-field reset without touching anything outside this repo, use: ``` ./fruth clean ``` This is a repo-local maintenance helper implemented by `./clean_repo_state.sh`. It does not edit `~/.codex` or any other external config. Before maintenance, cleanup checks listeners on its existing ports (5011 and 11434–11550). It stops only verified processes belonging to this checkout and rechecks identity before both SIGTERM and any SIGKILL escalation. Model processes need an unambiguous matching `model_ports.json` record with `process_identity` and `server_command`, plus the matching live working directory. Records without that evidence are left untouched; cleanup never fills in ownership from a PID or port. The webserver uses its exact script/interpreter and live checkout identity because it has no model registry record. New Ollama, MLX and llama.cpp launches capture birth/command evidence immediately from their own child process and persist it with the existing instance record after verifying readiness and exclusive port ownership. The Ollama default server also receives a service record in `model_ports.json`; it is excluded from model selection and retained across model-only registry updates. An existing default server is reused only with verified checkout-bound service evidence. Older or independently started processes without that evidence remain untouched and block ownership-dependent maintenance or default-server reuse. No records are backfilled from occupied ports. Cleanup's port scope remains unchanged. Unrelated, other-checkout, stale, conflicting or unverifiable listeners cause a nonzero exit before retention writes, archive copying or deletion. Resolve the reported process ownership before retrying; do not assume the port identifies Fruth. `--dry-run` sends no signals, describes verified candidates and reports unresolved ownership with the same nonzero exit. The storage and retention scope below is unchanged. See [process ownership limits](https://github.com/fl0ri0/fruth/blob/main/docs/CONTROL_KNOBS.md#cleanup-and-startup-process-ownership). Default cleanup removes repo-local generated/runtime ballast: - contents under `artifacts/*` buckets while preserving the standard empty artifact bucket directories, including`artifacts/bundles/` ; missing standard directories are created only as a fallback - `logs/*` - root-level `outputs` , when present (legacy cleanup compatibility; archive mode moves it into the verified archive run before live cleanup) - `state/chat_history/*` - `state/events.jsonl` - `state/infer_history.jsonl` - `state/artifact_registry.jsonl` - current artifact-centered backing ledger used by artifact dossiers, provenance, metadata, and enrichments - `state/generated_image_provenance.jsonl` - legacy/generated-image-specific provenance ledger when present - `state/response_frames/*` - compact response ledger, SQLite Index generations and selector, retained legacy Index files, and sidecar snapshots under `snapshots/` - compact response ledger, SQLite Index generations and selector, retained legacy Index files, and sidecar snapshots under - `state/runtime_status.json` - Python/test cache files and directories such as `__pycache__/` ,`.pytest_cache/` ,`*.pyc` ,`*.pyo` ,`.coverage` ,`htmlcov/` ,`.mypy_cache/` , and`.ruff_cache/` Default cleanup preserves: - `model_ports.json` - `state/llama_cpp_catalog.json` - `state/inference_preferences.json` - `state/inference_compiled_memory.json` - `state/inference_compiled_memory.md` - `state/self_learning/` - `state/inference_learning_archives/` - protected learning state and learning-reset archives remain outside recursive cache/Finder sweeps, including retained files named `*.pyc` ,`*.pyo` or`.DS_Store` - protected learning state and learning-reset archives remain outside recursive cache/Finder sweeps, including retained files named - `state/self_attack/` ,`state/benchmarks/` ,`state/diagnostics/` , and`.fruth_archiv/` , including retained cache files If you want a true empty repo-local state, combine: ``` ./fruth clean --forget-inference --reset-registry --reset-llama-catalog ``` The shorter equivalent is: ``` ./fruth clean --full ``` For the current recommended archive-first rotation into a clean live state, use: ``` ./fruth archive --full ``` `archive --full` snapshots protected interpretive inference state when present, but preserves the active checkout copies. Protected interpretive inference state includes `state/inference_preferences.json`, `state/inference_compiled_memory.json`, `state/inference_compiled_memory.md`, `state/self_learning/`, and the accepted-learning policy snapshot under `state/self_learning/accepted_policy_snapshot.json`. Preview it first with: ``` ./fruth archive --full --dry-run ``` Optional deeper reset flags: - `./fruth clean --full` - equivalent to `--forget-inference --reset-registry --reset-llama-catalog` - equivalent to - `./fruth archive --full` - archive-first cleanup plus registry/catalog reset, while preserving protected interpretive inference state in the active checkout - `./fruth clean --reset-registry` - rewrite `model_ports.json` to`[]` - rewrite - `./fruth clean --forget-inference` - remove interpretive inference preferences and retired compiled-memory files when present; learning-reset archives and active self-learning state remain preserved If you want to reset the interpretive inference layer's local event/log state without wiping the durable frame ledger, use: ``` python3 scripts/fruthctl.py inference --reset-learning-state --json ``` This archives the prior interpretive inference event/log state plus `logs/flask_webserver.log` under `state/inference_learning_archives//`, recreates fresh `state/events.jsonl`, removes any retired learned-policy or compiled-memory residue if it still exists, and intentionally preserves `state/response_frames/responses.jsonl`. Interpretive inference learning reset is deliberately separate from `archive --full` and `clean --forget-inference`. If the reset-learning command removes `state/self_learning/`, it archives that directory first. - `./fruth clean --reset-llama-catalog` - remove `state/llama_cpp_catalog.json` - remove - `./fruth clean --dry-run` - preview the cleanup without deleting anything Use this when the repo has accumulated logs, artifacts, history, or cache ballast and you want a predictable local reset before starting again. macOS/Chrome note: `clean` and `archive` preserve the standard `artifacts/` bucket structure. Chrome can keep or lose local folder permissions across that rotation. If a local `artifacts/bundles/.../index.html` opens but bundle CSS or images fail with `net::ERR_ACCESS_DENIED`, quit Chrome and re-grant Chrome access under macOS Privacy & Security (`Files and Folders` / `Desktop Folder`, or `Full Disk Access`). If needed, reset the Desktop Folder permission and approve it again: ``` tccutil reset SystemPolicyDesktopFolder com.google.Chrome ``` To make macOS ask again on the next Chrome `file://` bundle access, include the opt-in cleanup flag: ``` ./fruth archive --full --reset-chrome-file-access-prompt ``` Live generated/operator file buckets now belong under `artifacts/`, especially: - `artifacts/inputs/` for saved request inputs - `artifacts/audits/` for audit/report-style outputs as part of the basic artifact skeleton recreated by the reset helper - `artifacts/bundles/` for post-response, local-openable response artifact bundles - `artifacts/settings/` for explicitly promoted reusable settings artifacts from response-frame control snapshots If you want the same live cleanup, but do not want to lose the cleaned runtime/generated data immediately, use: ``` ./fruth archiv ``` `archiv` and `archive` both point to the same archive-first cleanup mode. It copies the `artifacts/` tree into the archive, archives the other useful cleaned ballast, and stores it under: - `.fruth_archiv//` and then applies the same live cleanup to `logs/` and the volatile `state/` paths. For `artifacts/`, live cleanup removes generated contents while preserving the standard bucket directories, including `artifacts/bundles/`. Missing standard directories are created only as a fallback if they were manually deleted or do not exist yet. This is intentionally only a hidden repo-local data store, not a live runtime path. Archive also preserves completed Self-Attack, benchmark and diagnostic evidence by copying the entire `state/self_attack/`, `state/benchmarks/` and `state/diagnostics/` trees into the same relative archive paths. Completion conventions vary, so this conservative rule includes unknown/in-progress evidence and leaves every live evidence source in place. Internal directory structure, fixtures and campaign-local script copies are retained. Current tooling under `scripts/`, `tests/` and other source/documentation directories stays active. Clean does not independently prune these evidence trees or existing archives, including their cache files. `--dry-run` lists evidence entries, selected copy bytes, free space, the destination and the required safety margin (the larger of 256 MiB or 5% of selected copy bytes). Insufficient space aborts before listener stops or cleanup. Copies are verified by tree membership, file mode, symlink target and SHA-256 before cleanup; copy failure, detected source changes or verification failure aborts and retains sources and any partial archive. Existing timestamp destinations are rejected. Repeated archives duplicate these evidence trees; no automatic pruning is provided. Stop campaign writers before requesting a consistent final snapshot: active copies are not atomic. Relative links within copied trees keep their topology. Absolute checkout links and external references remain unchanged; artifacts/logs are included only through the existing archive scopes, with no dependency crawling or historical link rewrite. Important distinction: - `./fruth clean` - hard cleanup, no archive - `./fruth archiv` - archive-first cleanup for temporary retention - `./fruth archive` - archive-first cleanup for temporary retention Notes: - `archiv` /`archive` can also be combined with`--forget-inference` ,`--reset-registry` , or`--reset-llama-catalog` ; if explicit removal/reset flags are used, the affected files are archived or snapshotted first and only then cleaned/reset. - `archiv` /`archive --full` is the concise archive-first rotation for runtime/generated ballast plus registry/catalog reset. It does not replace`--forget-inference` or a self-learning reset. `fruth_research/` survives every clean/full/forget/reset mode unchanged, including nested caches, candidate reviews, Gold metadata and retained evidence. Archive includes one byte-verified copy of the whole tree in its existing space preflight; the active Research tree remains. A verification failure prevents cleanup. Successful conventional self-learning eval persistence synchronizes an observational candidate queue. Source labels stay historical; Gold promotion requires explicit curation. Secondary sync failure logs a warning without rolling back learning. Before a fresh reset, validate the current retained Gold store; after reset, validate Gold again and refresh candidate availability. Candidates can retain review metadata while losing uncurated raw provenance. See [Research lifecycle](https://github.com/fl0ri0/fruth/blob/main/fruth_research/README.md) and [candidate recovery commands](https://github.com/fl0ri0/fruth/blob/main/fruth_research/candidates/README.md). Retention is internal only and does not authorize export or execution of a reset. Use exactly one owner for `ollama serve` at a time. - If Fruth manages your model lifecycle, stop the Homebrew background service first: - `brew services stop ollama` - Then start Fruth normally and let Fruth launch the `ollama serve` processes it needs. - Do not keep both Homebrew `ollama` service and Fruth-managed`ollama serve` processes active in parallel. - `./start_multi_models.sh` now checks this and stops early with a clear fix message unless you explicitly override it via`FRUTH_ALLOW_BREW_OLLAMA_SERVICE=1` . Why this matters: - parallel `ollama serve` owners can leave stale processes running after code changes, - restart/stop logs become misleading, - environment-dependent fixes can appear to "not work" because requests still hit an older process. Homebrew path note: - `/opt/homebrew/bin/ollama` and`/opt/homebrew/opt/ollama/bin/ollama` are normal Homebrew paths to the same installed binary, - seeing both in `ps` output does not mean Ollama was installed in the wrong place, - the real problem is duplicate process ownership, not the path spelling. The canonical project environment is: - `.venv/` Install Python dependencies from `requirements.txt`. Local test runner: - `python -m pytest` MLX runtime note: - `MLX_PYTHON` controls the shared MLX LM/VLM/Whisper interpreter path. - `MLX_AUDIO_PYTHON` can point to a separate interpreter for`mlx_audio.server` . - If unset, Fruth also checks `/opt/mlx-audio/venv/bin/python` automatically for TTS. - Use a separate TTS venv when `mlx-audio` pins package versions that conflict with the LM/VLM stack in`/opt/mlx/venv` . - Whisper discovery safely preflights its Numba dependency without importing MLX or initializing Metal. An incompatible NumPy/Numba pair is reported as degraded and is not startable. - Run `python scripts/fruthctl.py doctor runtime --json` to distinguish a package/dependency incompatibility from a stale degraded instance that only needs a bounded restart. `/api/start_model` is capability-aware and validates startup requests before launching a model instance. Supported capabilities: - `chat` : text/chat models (default fallback). - `vision_analysis` : OCR/vision analysis models. - `image_generation` : image-generation models (for example Flux family). - `speech_to_text` : speech transcription models (for example Whisper on MLX). - `text_to_speech` : speech synthesis models (for example Qwen3-TTS on MLX Audio). Capability-specific runtime behavior: - `chat` (MLX LM): starts`mlx_lm.server` ; registry metadata now records`backend_package=mlx_lm` ,`backend_contract=mlx_lm.server` , and the LM-specific runtime knobs surfaced by Fruth (`prefill_step_size` ,`prompt_cache_size` ,`prompt_cache_bytes` ). - `image_generation` (Ollama): starts a dedicated Ollama instance without chat preload. - `speech_to_text` (MLX): starts the local`mlx_whisper` HTTP shim (`backend_package=mlx_whisper_shim` ), not native`mlx-audio` ; the shim exposes`/healthz` ,`/v1/audio/transcriptions` , and`/api/transcribe` . Transcription uses JSON with a local`audio_path` , listener-bound Host/Origin checks and`X-Fruth-Action: 1` , which the Fruth client supplies. See the[Whisper caller contract](https://github.com/fl0ri0/fruth/blob/main/docs/HTTP_API_BOUNDARY.md#mlx-whisper-adapter) ; the marker is not native-process authentication. - `text_to_speech` (MLX): starts`mlx_audio.server` and serves OpenAI-style speech synthesis through`/v1/audio/speech` ; Fruth records that the upstream package is broader than the current instance contract and also covers STT/STS upstream. - `vision_analysis` (MLX VLM): starts`mlx_vlm.server` and uses its OpenAI-style multimodal routes; Fruth records the richer upstream contract including`/v1/chat/completions` ,`/v1/responses` ,`/v1/models` ,`/health` , and`/unload` , plus the one-model-at-a-time / lazy-load semantics. This is also the generic path for MLX multimodal snapshots whose metadata resolves to`image-to-text` or`image-text-to-text` . Unsupported backend/capability combinations return `400` with a clear message instead of generic `500`. Available-model discovery is intentionally broader than startup eligibility: - `/api/available_models` and`fruthctl models list` can show cached source entries that are present locally but not yet startable through the current backend contract. - Those entries carry `discovery_state` ,`runnable` ,`runnable_checks` , and`disabled_reason` so callers can distinguish "seen in cache/catalog" from "can be launched now." - Interactive startup surfaces should only offer runnable entries as start choices. Keep `/api/infer` only for lower-level debugging, narrow compatibility cases, or older direct clients that have not moved yet. It is now a compatibility wrapper around the same shared infer execution used by `/api/responses`. New integrations and first-party UI flows should prefer `/api/responses`. - `chat` : - JSON prompt only or multipart with text/image/text-file attachments. - `vision_analysis` : - multipart image upload, or PDF upload (text-layer extraction + scan fallback via rendered pages). - `image_generation` : - prompt required; optional image attachment for image-to-image flows. - `speech_to_text` : - multipart audio upload required. - `text_to_speech` : - text prompt required; backend persists generated audio to `artifacts/audio/` and returns the saved path. - Session Controls for TTS now support speaker/style tuning via `voice` ,`instruct` ,`speed` , and`pitch` . - text prompt required; backend persists generated audio to Current direct UI/internal behavior: - Direct backend chat route `/api/chat` still exists only as a compatibility wrapper for specialized or older callers. - First-party UI execution flows, voice input, `fruthctl send` , and the Responses workbench use`/api/responses` as the canonical execution surface. - Session Controls are derived from the actual target model. MLX models that advertise `reasoning_effort` use their declared non-Off default on first use; an explicit user choice, including Off, persists per model/backend. - General vision-language capability does not imply OCR/PDF controls. Document controls are shown only for recognized OCR-specialist families such as DeepSeek-OCR and GLM-OCR; other VLMs keep independently advertised controls such as reasoning. - `/api/responses` now rejects plain chat text that claims image/audio generation for a non-chat routed turn when no real image/audio artifact was produced. - Arena mode stays chat-only. - Voice input button (UI): - Requires one running `speech_to_text` instance (for example Whisper). - Records microphone audio, auto-stops on silence (with a max-duration safety cap), transcribes via `/api/responses` , and inserts text into the prompt box. - If transcript ends with `send` (or`senden` ), UI treats it as a voice command and triggers`Run` automatically. - Works as prompt input for chat, vision, and image-generation tabs. - Requires one running - Speech output handling: - UI renders an inline audio player plus `Open Folder` and`Download Audio` actions for`text_to_speech` responses. - Backend persists synthesized audio under `artifacts/audio/` . - In a canonical multi-phase `text_to_speech -> speech_to_text` response, artifact existence is not enough to prove spoken meaning. The TTS fill records the exact final backend prompt and SHA-256 as`tts_semantic_source` ; its directly dependent STT fill records`tts_stt_semantic_evidence` against only that producer source. Missing, drifted, ambiguous, or mismatching evidence blocks the branch with`DEPENDENCY_CHAIN_REPAIR_REQUIRED` /`repair_dependency_chain` , even when a WAV and transcript physically exist. - For counted audio variants, explicitly labelled contiguous variants are the candidate authority. Runtime selects one speakable field per index and excludes sibling transcript claims, analysis, code, and JSON; duplicate, missing, or ambiguous slots fail closed instead of being spoken. - For Qwen3-TTS model families: - `CustomVoice` variants are the right fit when you want one stable named speaker plus optional emotion/style instructions. - `VoiceDesign` variants are the right fit when you want the voice itself described in natural language. - UI renders an inline audio player plus - Image output handling: - UI renders `Open Folder` (local file manager) and`Download` buttons below generated images. - Backend persists generated images to `artifacts/images/` when needed and returns the saved path in API payload for local open actions. - UI renders - PDF document handling: - Local PDF parsing and rendering run in owned disposable processes with a separate 120-second preparation deadline, one shared preparation slot, and explicit input/output budgets. Timeouts and authoritative branch cancellation terminate and reap preparation; failures cannot count as partial success. See [PDF preparation controls](https://github.com/fl0ri0/fruth/blob/main/docs/CONTROL_KNOBS.md#pdf-page-preparation) . This is process containment, not a filesystem/network sandbox or a hard memory ceiling. - PDF attachments are accepted in `/api/responses` and still supported by`/api/infer` compatibility calls. - File-aware external agent routes receive the original selected PDF under their existing file-sharing controls; page conversion is confined to backends that require text or images. - For `vision_analysis` models and chat instances advertising image input (including Apple AI single tabs), PDFs are treated image-first: each selected page is rendered and sent to the selected instance for recognition or analysis. Mixed documents retain both searchable text and text inside images through their rendered pages. - Apple **Read text (OCR)** and**Read barcodes / QR codes** also accept PDFs. They run the existing native image tool on each rendered page and return page-labelled detections with per-page receipts bound to the original PDF digest. A failed tool call is an explicit error; it does not fall back to model prose. - DeepSeek-OCR page extraction follows the official prompt pipeline from the upstream docs: - primary: `` +`<|grounding|>Convert the document to markdown.` - fallback: `` +`Free OCR.` - primary: - Text-only targets and explicit `pdf_prefer_text` requests use`pypdf` to extract the existing text layer as document context. If some pages have no extractable text, a warning identifies that missing coverage; text extraction does not recognize text inside images. Explicit Apple OCR/barcode modes require rendered pages even if`pdf_prefer_text` is set. - For scanned PDFs, backend renders pages as images and runs page-wise OCR/context analysis. - Ollama OCR page execution uses `/api/generate` first and automatically retries with an emergency OCR prompt when Ollama returns empty content or upstream generate errors. - DeepSeek-OCR requests are sent with conservative runner options (`num_ctx=4096` ,`num_keep=0` ) to avoid sequence errors on some Ollama builds. - Prompt-echo responses (where the model repeats user instructions instead of OCR text) are treated as invalid page OCR output and trigger fallback automatically. - Optional request params: - `pdf_max_pages` (blank by default; only set when you want to cap how many pages Fruth renders and OCRs for this request, max`500` ) - `pdf_dpi` (default`260` for`vision_analysis` ,`180` for chat, max`600` ) - `pdf_page_timeout_sec` (default`240` ; timeout per OCR page call) - `pdf_page_retry_dpi` (default auto; retries failed pages with lower DPI) - `pdf_max_image_side` (default`2400` ; constrains rendered page size for stability) - `pdf_prefer_text` (default`false` ; for vision OCR path we render PDF pages first) - `infer_timeout_sec` (default`1200` , max`7200` ) - `pdf_synthesize` (default`false` ; set`true` for one-pass global merge) - Ollama page failures retain partial results and warnings. Empty page answers from compatible chat/vision transports retain a labelled gap and warning; they are not counted as successful recognition. Native Apple tool failures stop with the failing page number and receipts for earlier successful pages. A page cap retains the full document page count and a coverage warning. - Inline PDF output is a convenience view and may be shortened for UI stability; the saved markdown artifact is the completeness source of truth. - Text-layer PDF runs now persist both the extracted source text artifact and the model-produced result artifact when available. - PDF inference history is persisted to `state/infer_history.jsonl` for explicit retrieval and reference. - Each incoming PDF is processed again using the current request settings. History remains available for explicit retrieval; the legacy `reuse_cached` input is accepted for compatibility but has no effect. - Query stored insights via `GET /api/infer_history?file_kind=pdf&limit=50` . - `pypdf` is installed by default for text-layer extraction. - On macOS, PDFKit draws pages into CoreGraphics bitmaps through the `pyobjc-framework-Quartz` dependency. All selected pages, intrinsic rotation, crop boxes and visible annotations are included. ImageIO encodes the page PNGs for the existing OCR/vision routes; rendering itself performs no OCR. Native bitmap rendering does not imply GPU acceleration. - `requirements.txt` installs the Quartz binding only on macOS, along with its PyObjC core/Cocoa dependencies. For an existing environment, apply the updated requirements explicitly using the install command above. Upstream wheels normally avoid compilation; source installs require Apple's command line developer tools ([PyObjC installation](https://pyobjc.readthedocs.io/en/latest/install.html) ). Fruth's source archive contains neither wheels nor Apple frameworks. - Local PDF parsing and rendering run in owned disposable processes with a separate 120-second preparation deadline, one shared preparation slot, and explicit input/output budgets. Timeouts and authoritative branch cancellation terminate and reap preparation; failures cannot count as partial success. See Running model records in `model_ports.json` now include: - `modelName` : canonical model identifier (same semantic value as`model` ). - `backend` : normalized backend (`ollama` ,`mlx` ,`llama_cpp` , or`apple_fm` ). - `capability` : normalized capability used by routing. - `features` : canonical feature flags such as`vision_input` ,`audio_input` ,`image_output` ,`audio_output` ,`tool_calling` ,`function_calling` ,`computer_use` , and`structured_outputs` . - `feature_sources` : per-feature provenance labels such as`explicit_metadata` ,`local_template` ,`curated_override` ,`capability_contract` , or`conservative_default` . - `inputs` : normalized input modality list. - `outputs` : normalized output modality list. - `backend_metadata` for Ollama-backed entries: a doc-aligned`/api/show` summary carrying fields such as`capabilities` ,`details` ,`parameters` , and derived`context_length` . - `backend_metadata` for MLX-backed entries: package-contract metadata describing which surface is in play (`mlx_lm` ,`mlx_vlm` ,`mlx_audio` , or`mlx_whisper_shim` ), the upstream/native endpoint set, package capabilities, and key constraints such as lazy loading or one-model-at-a-time model binding. - `backend_metadata` for Apple AI entries: the observed`fm serve` contract, exact health/models URLs, observation time and raw reported model entry. The API model is`system` ; the UI label is Apple AI. Optional macOS 27 discovery adds`system_model` with the Mac's default AFM variant, context size, source and timestamp, shown in startup and the Models panel. Unreported exact-server variant, revision and context size remain unknown. See[Apple AI](https://github.com/fl0ri0/fruth/blob/main/docs/APPLE_FOUNDATION_MODELS.md) . - `backend_package` /`backend_contract` : additive top-level provenance fields so callers can distinguish generic`backend = mlx` from the actual MLX package contract. - `discovery_state` ,`runnable` ,`runnable_checks` ,`disabled_reason` : available-model fields that now separate cache/catalog discovery from locally runnable backend truth. The legacy `model` field remains for backward compatibility. Dynamic runtime state now lives beside the stable registry in `state/runtime_status.json`. This status registry tracks live metadata such as readiness, activity, port/process health, and recent success/error timing without overloading `model_ports.json`. For Ollama-backed entries it also carries `backend_runtime`, a doc-aligned `/api/ps` summary with live loaded-model facts such as `expires_at`, `size_vram`, `context_length`, `digest`, and backend `details`. For MLX-backed entries the same nested field now carries package-aware live runtime facts such as the native base URL, package-specific health or infer routes, unload/model-list URLs where applicable, and the current model-binding strategy (`model_bound_at_launch` vs `model_selected_per_request`). It is now fed by canonical start/stop plus successful or failed work through chat and `/api/responses` flows, with `/api/infer` and `/api/chat` retained only as compatibility plumbing where still needed. That split is intentional: - `model_ports.json` is the stable registry plus attributed backend metadata. - `state/runtime_status.json` is the live status layer plus backend-specific runtime facts (`/api/ps` for Ollama, native-route/runtime binding facts for MLX, and observed server readiness for llama.cpp and Apple AI). - `fruthctl instances list --json` and`GET /api/running_instances` merge both views so callers do not have to guess from model names alone. For external wrappers, Fruth now exposes a discovery manifest: - `GET /api/runtime_manifest` - alias: `GET /api/routing_table` The manifest advertises: - canonical responses endpoints (`/api/responses` ,`/v1/responses` ) - direct per-instance responses URLs (`/api/local_provider//v1/responses` ) - running instance metadata (`instance_id` ,`model` ,`backend` ,`capability` ,`features` ,`feature_sources` ,`inputs` ,`outputs` ,`backend_metadata` ,`backend_runtime` , readiness/activity) - capability aliases such as `chat` ,`image` ,`ocr` ,`stt` ,`tts` This is intended for external wrappers that need to self-dock against Fruth without hard-coding ports or internal route knowledge. Canonical `/api/responses` can now be targeted in three ways: - explicit `instance_id` - wrapper alias / profile such as `chat` ,`image` ,`ocr` ,`stt` ,`tts` - canonical `capability` when only one default running instance for that capability should be chosen That means wrapper code can first read the manifest and then either: - call `/api/local_provider//v1/responses` directly for a fixed model, or - call `/api/responses` with`alias` ,`profile` , or`capability` instead of hard-coding`instance_id` Wrapper code should not need `/api/infer` or `/api/chat` unless it is intentionally targeting the lower-level compatibility surface. Feature-contract note: - `capability` is the coarse top-level mode. - `features` /`inputs` /`outputs` are the finer-grained routing contract for downstream clients. - Tool/function/computer-use flags stay conservative unless Fruth has explicit evidence, while modality flags and known local template evidence are projected directly when available. Active UI chat history persists under `state/chat_history/`, including the synthetic Responses workbench conversation. This is the canonical backend chat-history store for the active Fruth chat UI. Frontend/history shape note: - the page shell owns root state/startup plus timeline glue, while active frontend behavior lives in `static/ui/messages.js` ,`static/ui/conversations.js` ,`static/ui/settings-history.js` ,`static/ui/models.js` ,`static/ui/message-state.js` , and`static/ui/request-lifecycle.js` - durable chat history persists both request metadata and reusable artifacts - exact canonical Responses truth includes `working_frame` for the final orchestration state plus`response_frame` , a frozen local snapshot of request, target, route, runtime, non-default control snapshots, artifacts, error, output metadata, and the frozen working frame; bounded normal wire views expose refs and handles instead of hydrating these bodies - reusable settings artifacts can be promoted explicitly from response-frame controls through `/api/settings_artifacts` ; they store replay`request_overrides` under`artifacts/settings/` - `message.artifacts[]` is the canonical reusable-file ledger for both user inputs and assistant outputs - `request_snapshot` remains the durable request-context record, with`request_snapshot.input_artifacts[]` kept only for explicit user-side inputs instead of silently mirroring assistant outputs back into user turns - one-shot selected reference artifacts/messages are conversation-scoped UI state for the next turn, not a global workbench carry-over - lineage/rotation state for chat tabs is tracked through conversation metadata plus `slot_history_ids` - active workspaces center on the current or selected draft/conversation - saved chats live in the durable backend-driven `History` archive surface The model maintenance panel is backend-aware: - `Ollama` pulls/removes through the existing Ollama CLI path. - `HF / MLX` pulls/removes through the Hugging Face cache using the MLX venv`hf` CLI. 1. Pull/install the model locally first. 2. Confirm the exact model ID: - Ollama: `ollama list` - MLX / llama.cpp catalog-backed entries: check `./fruth ctl models list --json` or`/api/available_models` 3. Ollama: 4. Pick the correct capability: - chat -> `chat` - OCR/vision -> `vision_analysis` - Flux/image -> `image_generation` - Whisper/STT -> `speech_to_text` - Qwen3-TTS / speech synthesis -> `text_to_speech` 5. chat -> 6. Start via API/UI using matching backend + capability. 7. If startup fails: - Check the API error text for model-name suggestions or unsupported capability/backend guidance. - Re-run `./fruth ctl models list --json` and inspect`runnable` plus`disabled_reason` for the model before retrying. - Verify model naming (for example `x/flux2-klein` vs`flux2-klein:latest` ). ``` POST /api/start_model { "model": "x/flux2-klein", "backend": "ollama", "capability": "image_generation" } POST /v1/audio/transcriptions { "audio_path": "/absolute/path/to/audio.wav", "task": "transcribe", "language": "de" } POST /api/responses { "instance_id": "mlx-community__Qwen3-TTS-12Hz-0.6B-Base-bf16-mlx-11504", "input": "Guten Tag aus Fruth.", "voice": "Chelsie", "instruct": "Warm, calm, elegant German narration with a slight smile.", "speed": 0.95, "pitch": 1.1, "lang_code": "de", "response_format": "wav" } ``` This single-phase example materializes one audio artifact. When a request also asks Fruth to confirm the exact spoken text, use the canonical TTS-to-STT dependency chain and read its Closure plus `tts_semantic_source` / `tts_stt_semantic_evidence`; a non-empty WAV or transcript alone is not semantic fulfillment. The expected source text is verification truth and is never inserted into the STT request.