Fruth – A semantic state inference engine for AI

Developer @fl0ri0 released Fruth 0.1.9, an experimental semantic state inference engine that derives required work, permitted state transitions and fulfillment from a request, validated contracts and recorded evidence rather than generating token continuations like model inference. Fruth, previously released as Ollmo and built by one person as an independent project, combines a local browser interface and control plane with local model backends, interpretive inference routing, durable response frames, history and materialized artifacts, running an Intent -> possibility space -> relevance -> promoted contracts -> runtime work -> review -> frozen state loop. The release ships a `./fruth self-attack` command for adversarial conformance across discovered interpretive inference and runtime control profiles, plus five reference workflows dated October 27 and October 29 that preserve exact prompts, completed frames, monitor records and verified saved artifacts.

Intent opens a possibility space. Fruth records candidates and uses relevance and validated promotion to determine which work is owed. Its interpretive inference layer can infer meaning from evidence and propose interpretations, work and reviews; runtime contracts, validation and evidence govern accepted transitions. Model inference generates token continuations; Fruth derives required work, permitted state transitions and fulfillment from the request, validated contracts and recorded evidence. Recorded frames and artifact evidence support inspection and continuation of work. Mutable work, frozen frames and append-only successors remain distinct. Durable means supported persistence and continuation of recorded state; it does not mean infallible conclusions or guaranteed survival of every in-flight operation. Status: Fruth 0.1.9 — experimental, usable, and still evolving within the 0.x series. Conceived, designed, created, built, and developed by @fl0ri0 https://github.com/fl0ri0 . Independent project. Fruth is built and maintained by one person, not a company, research lab, or funded team. It is human-led and AI-enhanced — a vibe-coded project in the literal, accountable sense. Product direction and release decisions remain human; AI tools helped explore, implement, test, and document the system. The project began with ideas developed over time before taking shape in early terminal scripts. It grew from a prototype whose small changes were still copied by hand into the current runtime substrate. Previously released as Ollmo, it continues as Fruth. This release does not imply institutional backing, a formal security audit, or a support organization. Fruth combines a local browser interface and control plane with local model backends, interpretive inference routing, durable response frames, history, and materialized artifacts. Its core loop is: php Intent - possibility space - relevance - promoted contracts - runtime work - review - frozen state A response is a frozen truthful moment of work: what was requested, what became owed work, what runtime evidence exists, what remains pending or blocked, and which outputs or artifacts can be referenced later. Fruth uses a consistent vocabulary to connect intent, execution, evidence, and durable state. These terms give developers and AI coding agents shared reference points across code, tests, and documentation. When reviewing the architecture, consider whether each term captures a useful distinction and is applied consistently—not only whether it is immediately familiar. Run ./fruth self-attack for adversarial conformance across discovered interpretive inference and runtime control profiles. It writes a human-readable report and JSON results; see self-attack conformance https://github.com/fl0ri0/fruth/blob/main/docs/SELF ATTACK.md for coverage, live execution, budgets and regression replay. The October 31 validation status https://github.com/fl0ri0/fruth/blob/main/docs/SELF ATTACK STATUS 2026-10-31.md records the bounded property-based investigation, responsibility refactoring, process and PDF corrections, and their source-bound checks. Release validation records exact dependencies and source identities. Earlier local, AFM and PCC campaigns and reference workflows retain their original execution dates and scope. Five reference workflows https://github.com/fl0ri0/fruth/blob/main/examples/README.md preserve exact prompts, completed frames, monitor records and verified saved artifacts. The image-review and saved-file examples are from October 29; the other three are from October 27. See the report for historical results https://github.com/fl0ri0/fruth/blob/main/docs/SELF ATTACK STATUS 2026-10-31.md historical-results and the distinction between checked runtime contracts and model-answer accuracy. If you use Fruth, its software design, or its evaluation artifacts in research or a published system, cite the software using CITATION.cff https://github.com/fl0ri0/fruth/blob/main/CITATION.cff . Identify the exact version and commit or local snapshot used. Reference runs preserve their exact recorded execution identities and artifact evidence. The intent-preserving block-resolution principle means that a block is not permission to rewrite the request. Its solution is its own verified resolution, handled gently through the substrate: keep intent anchored, keep the obligation visible, record the runtime evidence, choose the right-sized state transition, and freeze truthful state rather than the desired story. See Principle 10 https://github.com/fl0ri0/fruth/blob/main/docs/PRINCIPLES.md 10-resolve-blocks-by-preserving-intent . Within one canonical response, Fruth can route different promoted branches independently to compatible model instances and capabilities, materialize their distinct outputs through Late Fill, and bind each evidenced result back into the same response state. This is capability- and availability-dependent; it does not imply that every response uses more than one instance. For research collaboration, replications, talks, panels, or work around Fruth, use the public contact methods on @fl0ri0's GitHub profile https://github.com/fl0ri0 , with no guaranteed response time. The source is provided for inspection, use, research, and independent forks. The maintainer is open to collaboration and is still learning the issue, pull-request, and review process. Please bring clear explanations and patience; responses, reviews, merges, and ongoing support cannot be guaranteed. See CONTRIBUTING.md https://github.com/fl0ri0/fruth/blob/main/CONTRIBUTING.md for participation guidance. Useful public context: - Vision Alignment https://github.com/fl0ri0/fruth/blob/main/docs/VISION ALIGNMENT.md - State Substrate Architecture https://github.com/fl0ri0/fruth/blob/main/docs/diagrams/fruth-state-substrate-architecture.html - Verified Reference Runs https://github.com/fl0ri0/fruth/blob/main/examples/README.md - Project Participation and Capacity https://github.com/fl0ri0/fruth/blob/main/CONTRIBUTING.md The primary tested environment is a recent macOS release on Apple Silicon with Python 3.11 or newer. Local capabilities require their corresponding Ollama, MLX, llama.cpp, or Apple Foundation Models backend and model. Windows, Linux, Intel Mac, remote hosting, and multi-user operation are outside the 0.1.9 support promise. Extract the source archive, enter its directory fruth-0.1.9 for the release, or fruth-main for the current repository ZIP , and run: ./fruth start The executable launcher creates .venv and installs requirements.txt on first start. The selected python3 must be Python 3.11 or newer linked to OpenSSL 1.1.1 or newer; Apple's older Python/LibreSSL is unsuitable. Choose a suitable installed interpreter on PATH before first start, for example with Homebrew Python 3.11: PATH="$ brew --prefix python@3.11 /libexec/bin:$PATH" ./fruth start . Internet access is normally required for the initial dependency installation. Backend runtimes, model weights, Apple Intelligence availability and installed PCC Shortcuts are separate prerequisites for their respective capabilities. Backend runtimes and model weights are not bundled. For Apple PCC, choose Set up on its Models card or select PCC in the startup list. Fruth opens the bundled shortcuts on the Mac; review and click Add Shortcut , then start PCC. Manual import is also available in the PCC setup guide https://github.com/fl0ri0/fruth/blob/main/fruth integrations/shortcuts/README.md import . Both signed shortcut exports and assisted setup are included in the source tarball. Fruth startup initializes missing Research stores automatically and preserves existing candidates, reviews and Gold data. No separate Research startup command is needed. See the Research guide https://github.com/fl0ri0/fruth/blob/main/fruth research/README.md for candidate review, Gold curation and evidence retention. ./fruth start runs model selection and can create the environment itself. Press Enter at model selection to continue without starting a model. Startup starts the Flask webserver in the background, or reuses a verified webserver from this checkout on port 5011, and prints the dashboard URL. Verification checks the exact Python/script command, working directory, process birth identity and listener PID. Another service, another checkout, conflicting registry evidence or unavailable identity fails startup without a dashboard URL or a ready claim. A new launch must own the listener too; TCP occupancy alone is insufficient. This is process verification, not an HTTP/application health check. Use ./fruth status to inspect the stack and ./fruth restart to run stop followed by start. Refresh an open browser tab after restarting. Startup installs Python requirements only when it creates a new .venv . Before model discovery it checks that environment's own Python/TLS and uses pip's offline, non-installing checks for active direct requirements and transitive consistency. An unsuitable PATH Python does not block a valid existing .venv . Changed requirements remain usable when satisfied. Missing/broken interpreters, absent requirements.txt , incomplete installations and incompatible dependencies stop startup with repair guidance; startup never installs into, upgrades, deletes or rebuilds an existing environment. Keep model-backend environments separate; updating the main .venv does not update MLX, audio or native backend executables. Preserve your existing state/ , artifacts/ , Research data and settings; the release contains empty defaults for new installations, not a replacement for your saved data. While Fruth is stopped, explicitly apply declared requirements to a valid existing environment also after an interrupted first installation : .venv/bin/python3 -m pip install -r requirements.txt .venv/bin/python3 -I -B scripts/bootstrap preflight.py --venv .venv --requirements requirements.txt Preflight requires pip =22.2 for its supported install --dry-run CLI, paired with pip check . If pip is absent, repair it explicitly with .venv/bin/python3 -m ensurepip ; if it is too old, explicitly install a supported pip with .venv/bin/python3 -m pip install 'pip =22.2' . Startup does neither. Checks disable pip configuration, index access, caches and version lookups. The startup manifest supports one named distribution per line, optional version constraints and PEP 508 markers, blank lines and comments. URLs, local projects, archive paths, extras, includes, continuations and pip options are rejected before pip: dry runs of arbitrary projects can execute build hooks. These are metadata checks, not package import, native PDF, model or backend qualification. For a missing/broken or incompatible interpreter, first restore its original base Python installation if possible. If replacement is necessary, stop Fruth, retain the old environment at a unique backup path, and create a new .venv explicitly with a suitable installed interpreter, for example: Only when .venv.previous does not exist; keep it for recovery. mv .venv .venv.previous "$ brew --prefix python@3.11 /bin/python3.11" -m venv .venv .venv/bin/python3 -m pip install -r requirements.txt .venv/bin/python3 -I -B scripts/bootstrap preflight.py --venv .venv --requirements requirements.txt Omit the move when .venv is absent. A moved venv is a backup, not a runnable environment: restore it to its original path with its matching base interpreter to roll back. Missing source files must be restored from the matching source archive. Start Fruth explicitly after the checks pass; preflight does not start it. Existing JSON response histories require explicit migration before ordinary writes. Finish active work, stop all response writers, and verify a complete backup before running these commands from the updated checkout: .venv/bin/python scripts/migrate response frame index.py migrate --frames-dir state/response frames --check-only .venv/bin/python scripts/migrate response frame index.py migrate --frames-dir state/response frames --writers-stopped .venv/bin/python scripts/migrate response frame index.py check --frames-dir state/response frames Resume writers only after verification succeeds. Migration preserves the canonical Ledger, snapshots and evidence and selects one SQLite Index; retained JSON is inactive. Already migrated histories need no repeat migration. New installations initialize SQLite through the frame writer. The same tool provides explicit rebuild and current-history rollback operations; see the Response Frame contract https://github.com/fl0ri0/fruth/blob/main/docs/RESPONSES CONTRACT.md response-frame . Fruth's default backend ports are 11434 Ollama , 11435–11500 dedicated Ollama , 11501–11550 MLX , 11551–11600 llama.cpp , and 11601–11650 Apple Foundation Models . The optional Apple PCC Shortcuts adapter uses 11651–11700. Apple AI supports local text and image analysis through the installed fm serve with the API model system ; multiple instances appear separately in the existing UI. See the Apple AI Image Mode control for explicit OCR and barcode/QR recognition through the installed CLI tools, with verified tool-result receipts. See the Apple AI guide https://github.com/fl0ri0/fruth/blob/main/docs/APPLE FOUNDATION MODELS.md for prerequisites, metadata, streaming and the observed interpretive-inference context limitation. Apple PCC is also available as a startable chat and vision instance and an existing preferred-II choice. It automatically prefers Cloud Pro and falls back to Cloud on confirmed Pro access/availability errors. It sends task text, attached images and selected context to Apple, including the full Fruth policy for interpretive inference. Direct single-instance chat has no automatic Fruth policy or role. PCC is separate from local Apple AI. See PCC setup and metadata https://github.com/fl0ri0/fruth/blob/main/fruth integrations/shortcuts/README.md . The model registry remains local to the Fruth checkout. The webserver log is logs/flask webserver.log . ./fruth stop also shutdown or down runs the shutdown sequence: stop the Flask webserver, collect registered PIDs and listeners in the backend port ranges, stop those processes and the default Ollama server, then finalize runtime status and archive session logs. Apple AI and PCC use ownership-checked stopping; an unverified Apple bridge process preserves recovery state and makes shutdown fail. It prints the process/port diagnostics in the terminal. An already-stopped stack is a successful stop, so restart continues into startup normally. External provider projections are refreshed only by an explicit ./fruth sync . Open the local interface: ./fruth dashboard The dashboard is served directly at http://127.0.0.1:5011/ ; /dashboard remains a compatibility alias. The separate static project page is available at http://127.0.0.1:5011/site/ while Fruth is running, or by opening site/index.html directly from the checkout. The self-contained site/ directory is also the GitHub Pages publication artifact. It uses only relative, bundled assets and is not the live control surface. Useful lifecycle commands: ./fruth status ./fruth shutdown To inspect locally runnable models and start one chat model explicitly: ./fruth ctl models list --json --runnable-only ./fruth ctl start --model "<model-id " --backend "<backend " --capability chat --json Use an identifier and backend reported by the first command. Fruth does not bundle model weights. The release includes a dedicated Codex skill that teaches Codex how to inspect Fruth runtime truth, use interpretive inference routing safely, and work with canonical responses and artifacts. During broader Codex work, the skill also makes already-running local text, image, vision/OCR, speech, embedding, and multimodal capabilities available as optional bounded tools while Codex remains the supervisor. From the extracted Fruth directory: FRUTH CODEX SKILLS DIR="${CODEX HOME:-$HOME/.codex}/skills" mkdir -p "$FRUTH CODEX SKILLS DIR" if -e "$FRUTH CODEX SKILLS DIR/fruth" ; then printf 'Fruth skill already exists at %s\n' "$FRUTH CODEX SKILLS DIR/fruth" else cp -R skills/fruth "$FRUTH CODEX SKILLS DIR/fruth" fi Restart Codex, open the extracted Fruth directory as the active workspace, and then mention Fruth normally or invoke $fruth explicitly. When working from a different workspace, set FRUTH HOME to the extracted Fruth root or provide the checkout path when the skill asks. Installing the skill does not start Fruth, change Codex model-provider settings, or enable cloud routing. The maintained source is skills/fruth/ ; an installed copy is a distribution, not a second authority. For an explicitly requested update, compare source and installed files, preserve any local changes in a protected recovery location, then copy the package contents into the existing installed fruth/ directory using the same copy-based installation process. Compare every shipped file after copying, including NOTICE , agents/openai.yaml and references. Do not silently overwrite divergent local customizations or switch discovery directories. The entrypoint retains essential guardrails; references/fruth-contract.md is loaded for the relevant request/result recipe. Detailed contracts are resolved from the selected checkout via FRUTH HOME or the documented discovery process, so starting Codex in another directory does not break those references. Repository contributors should begin with CONTRIBUTING.md https://github.com/fl0ri0/fruth/blob/main/CONTRIBUTING.md and the architecture/contract reading map below; the operator skill remains separately usable. Reload/restart the client as required by its instruction-discovery behavior; an existing task's context is not proof of a fresh load. Fruth runs locally by default. It can optionally use ChatGPT as an external target through Codex after the user reviews the cloud disclosure and explicitly enables the connection. Fruth first looks for the Codex executable bundled with the ChatGPT app and then falls back to a separately installed codex executable. The integration reuses the login owned by that executable; Fruth does not read, copy, or store the credentials and does not set a fixed model. Once enabled, ChatGPT appears as a clearly marked cloud tab, as an explicit Fruth target, and as a preferred primary or fallback interpretive inference target. It can be turned off again at any time. Fruth explicitly supplies the current prompt, context it promotes for that turn, and selected local files or Fruth artifacts for OpenAI processing. Input staging does not confine all downstream reads: the requested read-only command sandbox restricts writes, while the Codex client and enabled tools have their own access boundaries. See the security boundary https://github.com/fl0ri0/fruth/blob/main/SECURITY.md chatgpt-through-codex-and-cloud-processing . Recognized images use Codex's native image-input path. Other selected regular files are copied under neutral names into a temporary working directory, with byte counts and digests recorded. One request can include at most 5 files, no more than 100 MiB per file and 250 MiB in total. URLs, folders, and direct leaf symbolic links are rejected; regular files reached through symlinked parent directories are accepted. ChatGPT returns text through this route; Fruth does not guarantee that every regular-file format can be interpreted semantically by the selected model and its available tools. The ChatGPT target never appears as a running local instance and has no Start, Stop, Pull, or Delete actions. Direct tab turns are ephemeral and independent; Fruth can promote bounded relevant context for a referential turn, but does not create a persistent provider session. Model selection is automatic, and the exact GPT variant is not exposed to Fruth; model self-descriptions are not runtime proof. Direct API-key management is outside this route's 0.1.9 contract. Apple PCC is a separate optional integration described above. Fruth's own source code and project documentation are licensed under the Apache License 2.0 https://github.com/fl0ri0/fruth/blob/main/LICENSE , unless a file says otherwise. Bundled and optional third-party components retain their own licenses; see NOTICE https://github.com/fl0ri0/fruth/blob/main/NOTICE and THIRD PARTY NOTICES.md https://github.com/fl0ri0/fruth/blob/main/THIRD PARTY NOTICES.md . Project repository: GitHub https://github.com/fl0ri0/fruth . Release tooling builds and verifies local archives; it does not publish or upload them. Canonical artifact storage lives under artifacts/ : generated files are written to artifacts/ , saved request inputs to artifacts/inputs/ , and audit/report-style outputs to artifacts/audits/ . Fruth separates model proposals from runtime authority. Its interpretive inference layer anchors current intent and derives candidate work. Promotion turns selected candidates into explicit obligations. The resolver and Late Fill materialize those obligations through compatible runtimes, and Closure Review compares the result with runtime and artifact evidence before the state can freeze. The mutable working frame records the fluid pre-freeze state, including the possibility space, candidate graph, promotion review, branch-local work, and closure state. The frozen response frame preserves the resulting request, route, runtime evidence, outputs, artifacts, and unresolved work. Canonical frames are appended to state/response frames/responses.jsonl ; normal bounded wire views expose handles and content-addressed references, while explicit truth views restore the exact referenced state. The semantic core overview https://github.com/fl0ri0/fruth/blob/main/docs/ARCHITECTURE MAP.md semantic-core connects these mechanisms: preserve required meaning, respect each operation's authority and contract, and support claims with evidence for the exact subject, version and scope. End-to-end checks follow the obligation across handoffs and recovered views. Current code ownership is split across: - fruth webserver.py for the Flask API/UI composition root; - fruth server/ for request, routing, execution, transport, and Late Fill runtime owners; - fruth services/ for response frames, artifacts, history, inference, and related product services; - fruth inference/ for current-turn semantic interpretation and graph derivation; - fruth orchestration/ for mutable pre-freeze working state; - fruth runtime/ and fruth core/ for backend lifecycle and shared runtime truth; - fruth integrations/ for external-client adapter infrastructure and the current Codex integration boundary; and - static/ui/ plus fruth webUI.html for the browser control surface. Reference docs: fruthctl is the common CLI adapter layer over the current Fruth Flask control plane. Use it when you want external or user-built wrappers, shells, and scripts to call one stable command surface instead of reaching into startup scripts or raw ports. The bundled Codex skill is the validated 0.1 agent integration. Entry point: - python3 scripts/fruthctl.py ... Base URL resolution: - --base-url http://127.0.0.1:5011 - or environment variable FRUTHCTL BASE - fallback: FRUTH WEB BASE - default: http://127.0.0.1:5011 Local runtime recovery order: - fruthctl talks to the current Fruth Flask control plane on 127.0.0.1:5011 . - First dependency: if local runtime commands hit connection refused on 5011 , check process/listener evidence and the client permission boundary before concluding that the control plane is down. - Read-like fruthctl commands such as instances list , models list , inference , and responses get do not recover/start the control plane by default. Pass --recover-control-plane only when that recovery behavior is intentional. - Mutating or lifecycle-oriented fruthctl commands may auto-attempt a one-shot local recovery by starting fruth webserver.py and retrying once for the default local base URL. - If 5011 is up but instances list --json is empty, start the required model with fruthctl start ... . - doctor runtime stays diagnostic and reports the broken state instead of auto-starting anything. - If the one-shot recovery still fails, run ./fruth start for the full local stack and inspect logs/flask webserver auto.log . Core examples: python3 scripts/fruthctl.py models list --json python3 scripts/fruthctl.py instances list --json python3 scripts/fruthctl.py inference --json python3 scripts/fruthctl.py doctor runtime --json python3 scripts/fruthctl.py start --model mlx-community/Qwen3.5-27B-4bit --backend mlx --capability chat --json python3 scripts/fruthctl.py send mlx-community Qwen3.5-27B-4bit-mlx-11502 "hello" python3 scripts/fruthctl.py send mlx-community whisper-large-v3-mlx-mlx-11501 --file "/absolute/path/to/audio.mp3" --json python3 scripts/fruthctl.py send mlx-community Qwen3.5-27B-4bit-mlx-11502 "hello" --truth-json python3 scripts/fruthctl.py responses get resp abc123 python3 scripts/fruthctl.py responses get resp abc123 --json python3 scripts/fruthctl.py responses get resp abc123 --truth-json python3 scripts/fruthctl.py events list --category infer --status started --json python3 scripts/fruthctl.py events tail --category infer --instance-id mlx-community whisper-large-v3-mlx-mlx-11506 python3 scripts/fruthctl.py wait mlx-community whisper-large-v3-mlx-mlx-11506 --json python3 scripts/fruthctl.py history infer --limit 20 --json Most important command: - send - targets the canonical /api/responses endpoint - lets Fruth decide internally whether the request becomes chat or infer/capability dispatch - defaults to a long timeout suitable for STT/TTS and other long-running infer requests - --json performs one bounded POST and prints its public wire envelope - --truth-json performs that POST, fetches the same id through exact view=truth , and prints a normalized canonical summary - targets the canonical - responses get - reads canonical /api/responses/<id ?view=truth lookup state - default output summarizes lifecycle state , status semantics , frame identity, output provenance, work-tree authority, and durability - --json preserves the raw canonical truth-view payload, while --truth-json emits only the normalized response-truth summary for scripts - --json and --truth-json are mutually exclusive on both send and responses get - reads canonical - wait - polls Fruth event history until the latest matching request for an instance reaches a terminal state - events tail - polls and prints new matching event log entries as they appear - inference - reads Fruth's runtime-intelligence summary with recommended defaults, current issues, and recovery hints fruth inference is the package boundary for the self-describing runtime-intelligence layer inside Fruth. Its runtime identity remains fruth-inference . It does not orchestrate agents. It explains and uses the runtime: - what is running - which instance is the current best default for each capability - what problems are active - which recovery commands should be tried next - which live capability/instance best fits an Auto request according to merged runtime truth rather than a hardcoded provider preference Current Auto routing behavior: - Auto routes from merged control-plane truth, not a fixed provider table - The interpretive inference layer, resolver transformation, control-hint filling, and late fill are separate responsibilities: - the interpretive inference layer chooses the next truthful phase and compatible runtime target - Fruth can then fill truthful request fields such as language, speaker, format, style, or image aspect controls for that chosen target - downstream phases can continue through the resolver + late fill under the same request/response identity - local backend model calls materialize selected phases or branches; runtime code decides fulfillment from graph, slot, output, artifact, and status truth - the request phase graph is now the shared substrate for interpretive inference-owned requests: - every interpretive inference-owned request freezes into a request phase graph - candidate graph records possible outputs, workload tasks, context, references, evidence, repairs, continuations, and learning hints - promotion review decides what becomes executable owed work and what remains reserved, waived, rejected, stale, or merely possible state - branch-local workload tasks carry focused payloads, prompts, dependencies, artifact refs, evidence, output contracts, visibility, review criteria, and freeze state - deterministic review criteria stay runtime/closure checks; only explicit semantic review criteria or non-deterministic qualitative criteria require semantic review - the graph is frozen in intent and fluid in state - plain chat may end at phase 1 - image/audio materialization defaults to chat - downstream branches - downstream phases remain explicit instead of being flattened into one capability - later branches should consume branch-local task truth rather than replaying the full root prompt, and missing generated input artifacts should become repair dependency chain instead of blind same-branch retry - canonical /api/responses now runs one bounded pre-freeze closure review against the frozen graph/work-tree/artifact truth: - the review is exposed as runtime.graph closure review plus developer diagnostics where applicable - it should continue resolver/late fill work for still-missing frozen obligations - it must not reinterpret the user request or invent new semantic goals - the review is exposed as - live interpretive inference route context is intentionally short and direct: - current turn stays dominant - fresh turns normally use current turn only context strategy - recent thread context is injected only for clearly referential or continuation-like turns - when that thread context is needed, the interpretive inference layer sees one answered prior user turn plus its direct assistant reply window - artifact continuity is output-centered: all artifacts from the last relevant assistant artifact message, then latest-by-type fallback - old tool calls and old artifacts do not become new intent just because they happened earlier - explicit stable preferences are explicit-only; live routing does not carry a separate derived memory block - Auto preview and final execution share the same resolver/control-hint/runtime path so the UI sees the same required-field truth the backend will execute - Auto route preview and selection are read-only planning surfaces. They must not start models, load providers, or turn route choice into runtime load feedback. Model lifecycle starts require an explicit lifecycle source such as a frontend play/start action. - embedding helpers remain internal interpretive inference tools; they can attach routing-context hints and narrow tie-break signals for anchored follow-ups, and those audits remain visible in runtime metadata - explicit low-level direct contracts such as direct instance id requests or batch prompts remain explicit exceptions; they are not the normal interpretive inference path - The interpretive inference layer now exposes file-backed semantic roles as advisory decision lenses. Any valid fruth inference/semantic roles/ .md role definition becomes part of the role catalog. - semantic role ids accepts canonical role IDs, such as "repairer", "evidence reasoner" , as advisory orientation only. - semantic role profile can travel through request meta, route preview, working frames, and canonical Responses runtime payloads as advisory orientation only. - FRUTH INFERENCE.md is the primary runtime-policy anchor the interpretive inference layer uses for current-turn interpretation; the backend now injects it into both interpretive inference routing decisions and interpretive inference-owned user-facing chat turns, while FRUTH FOR AGENTS.md remains the operator/client guide - visible assistant text must not claim saved locally , artifact created , or similar file/artifact success unless runtime truth actually contains that saved output Surfaces: - GET /api/inference - GET /api/agent contract - python3 scripts/fruthctl.py inference - FRUTH INFERENCE.md https://github.com/fl0ri0/fruth/blob/main/FRUTH INFERENCE.md - FRUTH FOR AGENTS.md https://github.com/fl0ri0/fruth/blob/main/FRUTH FOR AGENTS.md Frontend note: - the Responses workbench can now use Auto to route prompts and file-backed requests by capability instead of forcing one fixed target instance - the UI label is Auto , while the backend currently keeps the internal inference route / inference preview names for this routing path - Interpretive inference primary target / fallback target preferences remain interpretive inference and chat-routing hints; they do not override non-chat execution when the request resolves to image, TTS, or another incompatible capability - Explicit frontend play/start is the user lifecycle path for starting an additional instance. The frontend sends start source: "frontend button" and sends force start: true only when deliberately starting another same-model instance; the interpretive inference layer, route preview, late fill, and backend automatic paths must not use force start . Fruth now exposes one canonical execution endpoint on top of the existing route split: - POST /api/responses - POST /v1/responses Use this when you want one stable request/response language for execution calls. The older specialized routes still exist: - /api/chat - /api/infer but the canonical path is now responses , and first-party UI flows should stay on it. This is the official external contract for wrappers, agents, CLIs, editors, and new integrations. Effectful HTTP calls require X-Fruth-Action: 1 , including multipart, streaming and explicit-refresh GETs. The UI and shipped native clients emit it; older clients and existing Fruth-routed provider configurations must add it. Host and exact-Origin admission, passive reads, strict preference JSON errors and the browser/native trust distinction are in the HTTP caller contract https://github.com/fl0ri0/fruth/blob/main/docs/HTTP API BOUNDARY.md . Prefer responses even for file-backed, OCR, image, STT, and TTS work. /api/responses now executes shared internal helpers directly; external callers should not need to pick /api/chat or /api/infer unless they intentionally want the older compatibility surface. Reviewed redraw remains part of the same intent-aligned scope ladder. The lower bounded additive repair rungs stay autonomously active under the default-deny graph-repair policy; partial-subtree and full-successor rebase are the higher-risk rungs, with non-mutating shadow as their product default rather than a separate shadow layer. Read their canonical evidence gates at GET /api/graph rebase/readiness . The report is observer-only and does not promote work. An operator advances one exact proposal through POST /api/responses/<response id /graph rebase/operator in strict order: adjudicate , then durable audit-only stage , then gate-controlled authorize partial . The last action is compare-and-swap bound to the finalized response/frame/proposal/base/candidate/class identities and joins authority only from Fruth's trusted operator registry. Full durable state and bounded observation truth must identify the same latest frame, and current root truth is rechecked at replay, apply, persistence, and Late Fill. A successful partial authorization appends one branch-local successor frame under the same response id before scheduling its exact Late Fill work; it never rewrites the parent or replays the root prompt through another prompt carrier. Explicit FRUTH GRAPH REBASE AUTONOMY=off blocks the stage and authorize partial transitions; evidence-only adjudicate remains available because it grants no rebase authority. The product stays in shadow while evidence gates are not green, and full successor rebase remains non-executable under safe partial v1. The mutating endpoint is dormant unless startup explicitly supplies both a 32-or-more-character FRUTH GRAPH REBASE OPERATOR TOKEN and an exact FRUTH GRAPH REBASE OPERATOR IDENTITY . Calls must present the token as a Bearer token or X-Fruth-Graph-Rebase-Operator-Token and the configured identity as X-Fruth-Graph-Rebase-Operator ; neither value is persisted or inherited by model subprocesses. Runtime itself produces replay confirmation by deterministically revalidating the frozen proposal. Trusted-only or runtime-only stage halves do not count toward promotion. A no-proposal false negative adjudication is evidence-only and uses the exact sentinel expected proposal id=no formal proposal ; it cannot become stage or execution authority. It remains historical truth, but a later exact replay-verified useful proposal adjudication may append resolves record id=<false-negative-record-id so only unresolved false negatives block promotion. Current scope: - supports direct targeting with a running instance id - also supports routed/default targeting layers such as Auto/ inference route , wrapper alias/profile, or canonical capability when the caller or backend chooses the target - accepts one normalized request body with input , optional instructions , and optional file context file path or multipart file - keeps explicit input artifacts separate from explicit reference artifacts - supports stream=true with Responses-style text/event-stream events - returns one normalized response envelope with: - outputs - status - mode - instance id - model - output text - artifacts - warnings On the normal non-streaming path, that envelope is serialized from the compact indexed Response Frame after canonical persistence. A live fallback is not a durable-commit receipt; see the finalization boundary https://github.com/fl0ri0/fruth/blob/main/docs/RESPONSES CONTRACT.md finalization-and-durable-completion . Normal POST/default/UI/status/debug results are byte-budgeted to at most 8 MiB for the complete serialized outer envelope. Indexed POST/default/UI/status projection avoids sidecar hydration; debug may hydrate selected Closure/rebase observations, and exceptional ledger recovery may reconstruct canonical state before projection. Compaction is based on bytes rather than a fixed item limit: ordinary 5,000-character text and 65 tiny records remain inline when they fit, while strings from 256 KiB and collections from 1 MiB may become a bounded preview plus exact count/length, byte size, SHA-256, and an adjacent content-addressed snapshot ref . Public lifecycle, frame identity, outputs/slots, artifact handles, and effective content-SHA refs remain on the wire; hydrated Runtime, Working Frame, and duplicate full-frame bodies do not. GET /api/responses/<id ?view=debug adds bounded Closure/rebase observation evidence. view=full , view=raw , and view=truth recursively restore exact CAS-backed canonical truth and can be larger than 8 MiB. Canonical hydration validates every authoritative ref recursively and fails with HTTP 409 when a ref is missing, malformed, or corrupt. This keeps full truth by reference without semantic loss. Artifact-bundle operations likewise resolve exact CAS truth and fail closed rather than exporting an incomplete preview. Example: POST /api/responses { "instance id": "x/flux2-klein:latest-1", "input": "seven frogs posing like a family portrait on the forest floor" } Streaming note: - canonical streaming currently uses Responses-style SSE over the finalized normalized response payload; the final serialized SSE event, including its outer wrapper, obeys the same 8 MiB ceiling - this is already useful for one client language and UI compatibility - for chat-capable Ollama, MLX, llama.cpp and Apple AI runtimes, canonical streaming forwards real backend deltas - Apple PCC through Shortcuts emits a buffered completed answer, not provider token deltas - non-chat capabilities still use completed-response style payloads/synthetic SSE where appropriate - the main UI chat flow now uses canonical responses streaming for ordinary chat turns Fruth should not invent token limits on behalf of models. - app-side guessed token limits have been removed from the active model-manager/UI path - available model payloads should only expose token limits when a runtime or provider actually reports them - downstream sync layers should omit unknown contextWindow / maxTokens instead of filling in guessed defaults Use ./fruth start as the operator-facing entrypoint. It delegates to ./start multi models.sh and starts the canonical runtime/UI stack. - Starts model management and the current Flask UI/API. - Starts MLX support when available. - Leaves external-client provider projections untouched; projection is an explicit operator action. The active startup flow is the runtime/UI stack only. External provider projection is an explicit operator action, not part of start/stop/restart. When you want a local green-field reset without touching anything outside this repo, use: ./fruth clean This is a repo-local maintenance helper implemented by ./clean repo state.sh . It does not edit ~/.codex or any other external config. Before maintenance, cleanup checks listeners on its existing ports 5011 and 11434–11550 . It stops only verified processes belonging to this checkout and rechecks identity before both SIGTERM and any SIGKILL escalation. Model processes need an unambiguous matching model ports.json record with process identity and server command , plus the matching live working directory. Records without that evidence are left untouched; cleanup never fills in ownership from a PID or port. The webserver uses its exact script/interpreter and live checkout identity because it has no model registry record. New Ollama, MLX and llama.cpp launches capture birth/command evidence immediately from their own child process and persist it with the existing instance record after verifying readiness and exclusive port ownership. The Ollama default server also receives a service record in model ports.json ; it is excluded from model selection and retained across model-only registry updates. An existing default server is reused only with verified checkout-bound service evidence. Older or independently started processes without that evidence remain untouched and block ownership-dependent maintenance or default-server reuse. No records are backfilled from occupied ports. Cleanup's port scope remains unchanged. Unrelated, other-checkout, stale, conflicting or unverifiable listeners cause a nonzero exit before retention writes, archive copying or deletion. Resolve the reported process ownership before retrying; do not assume the port identifies Fruth. --dry-run sends no signals, describes verified candidates and reports unresolved ownership with the same nonzero exit. The storage and retention scope below is unchanged. See process ownership limits https://github.com/fl0ri0/fruth/blob/main/docs/CONTROL KNOBS.md cleanup-and-startup-process-ownership . Default cleanup removes repo-local generated/runtime ballast: - contents under artifacts/ buckets while preserving the standard empty artifact bucket directories, including artifacts/bundles/ ; missing standard directories are created only as a fallback - logs/ - root-level outputs , when present legacy cleanup compatibility; archive mode moves it into the verified archive run before live cleanup - state/chat history/ - state/events.jsonl - state/infer history.jsonl - state/artifact registry.jsonl - current artifact-centered backing ledger used by artifact dossiers, provenance, metadata, and enrichments - state/generated image provenance.jsonl - legacy/generated-image-specific provenance ledger when present - state/response frames/ - compact response ledger, SQLite Index generations and selector, retained legacy Index files, and sidecar snapshots under snapshots/ - compact response ledger, SQLite Index generations and selector, retained legacy Index files, and sidecar snapshots under - state/runtime status.json - Python/test cache files and directories such as pycache / , .pytest cache/ , .pyc , .pyo , .coverage , htmlcov/ , .mypy cache/ , and .ruff cache/ Default cleanup preserves: - model ports.json - state/llama cpp catalog.json - state/inference preferences.json - state/inference compiled memory.json - state/inference compiled memory.md - state/self learning/ - state/inference learning archives/ - protected learning state and learning-reset archives remain outside recursive cache/Finder sweeps, including retained files named .pyc , .pyo or .DS Store - protected learning state and learning-reset archives remain outside recursive cache/Finder sweeps, including retained files named - state/self attack/ , state/benchmarks/ , state/diagnostics/ , and .fruth archiv/ , including retained cache files If you want a true empty repo-local state, combine: ./fruth clean --forget-inference --reset-registry --reset-llama-catalog The shorter equivalent is: ./fruth clean --full For the current recommended archive-first rotation into a clean live state, use: ./fruth archive --full archive --full snapshots protected interpretive inference state when present, but preserves the active checkout copies. Protected interpretive inference state includes state/inference preferences.json , state/inference compiled memory.json , state/inference compiled memory.md , state/self learning/ , and the accepted-learning policy snapshot under state/self learning/accepted policy snapshot.json . Preview it first with: ./fruth archive --full --dry-run Optional deeper reset flags: - ./fruth clean --full - equivalent to --forget-inference --reset-registry --reset-llama-catalog - equivalent to - ./fruth archive --full - archive-first cleanup plus registry/catalog reset, while preserving protected interpretive inference state in the active checkout - ./fruth clean --reset-registry - rewrite model ports.json to - rewrite - ./fruth clean --forget-inference - remove interpretive inference preferences and retired compiled-memory files when present; learning-reset archives and active self-learning state remain preserved If you want to reset the interpretive inference layer's local event/log state without wiping the durable frame ledger, use: python3 scripts/fruthctl.py inference --reset-learning-state --json This archives the prior interpretive inference event/log state plus logs/flask webserver.log under state/inference learning archives/<timestamp / , recreates fresh state/events.jsonl , removes any retired learned-policy or compiled-memory residue if it still exists, and intentionally preserves state/response frames/responses.jsonl . Interpretive inference learning reset is deliberately separate from archive --full and clean --forget-inference . If the reset-learning command removes state/self learning/ , it archives that directory first. - ./fruth clean --reset-llama-catalog - remove state/llama cpp catalog.json - remove - ./fruth clean --dry-run - preview the cleanup without deleting anything Use this when the repo has accumulated logs, artifacts, history, or cache ballast and you want a predictable local reset before starting again. macOS/Chrome note: clean and archive preserve the standard artifacts/ bucket structure. Chrome can keep or lose local folder permissions across that rotation. If a local artifacts/bundles/.../index.html opens but bundle CSS or images fail with net::ERR ACCESS DENIED , quit Chrome and re-grant Chrome access under macOS Privacy & Security Files and Folders / Desktop Folder , or Full Disk Access . If needed, reset the Desktop Folder permission and approve it again: tccutil reset SystemPolicyDesktopFolder com.google.Chrome To make macOS ask again on the next Chrome file:// bundle access, include the opt-in cleanup flag: ./fruth archive --full --reset-chrome-file-access-prompt Live generated/operator file buckets now belong under artifacts/ , especially: - artifacts/inputs/ for saved request inputs - artifacts/audits/ for audit/report-style outputs as part of the basic artifact skeleton recreated by the reset helper - artifacts/bundles/ for post-response, local-openable response artifact bundles - artifacts/settings/ for explicitly promoted reusable settings artifacts from response-frame control snapshots If you want the same live cleanup, but do not want to lose the cleaned runtime/generated data immediately, use: ./fruth archiv archiv and archive both point to the same archive-first cleanup mode. It copies the artifacts/ tree into the archive, archives the other useful cleaned ballast, and stores it under: - .fruth archiv/<timestamp / and then applies the same live cleanup to logs/ and the volatile state/ paths. For artifacts/ , live cleanup removes generated contents while preserving the standard bucket directories, including artifacts/bundles/ . Missing standard directories are created only as a fallback if they were manually deleted or do not exist yet. This is intentionally only a hidden repo-local data store, not a live runtime path. Archive also preserves completed Self-Attack, benchmark and diagnostic evidence by copying the entire state/self attack/ , state/benchmarks/ and state/diagnostics/ trees into the same relative archive paths. Completion conventions vary, so this conservative rule includes unknown/in-progress evidence and leaves every live evidence source in place. Internal directory structure, fixtures and campaign-local script copies are retained. Current tooling under scripts/ , tests/ and other source/documentation directories stays active. Clean does not independently prune these evidence trees or existing archives, including their cache files. --dry-run lists evidence entries, selected copy bytes, free space, the destination and the required safety margin the larger of 256 MiB or 5% of selected copy bytes . Insufficient space aborts before listener stops or cleanup. Copies are verified by tree membership, file mode, symlink target and SHA-256 before cleanup; copy failure, detected source changes or verification failure aborts and retains sources and any partial archive. Existing timestamp destinations are rejected. Repeated archives duplicate these evidence trees; no automatic pruning is provided. Stop campaign writers before requesting a consistent final snapshot: active copies are not atomic. Relative links within copied trees keep their topology. Absolute checkout links and external references remain unchanged; artifacts/logs are included only through the existing archive scopes, with no dependency crawling or historical link rewrite. Important distinction: - ./fruth clean - hard cleanup, no archive - ./fruth archiv - archive-first cleanup for temporary retention - ./fruth archive - archive-first cleanup for temporary retention Notes: - archiv / archive can also be combined with --forget-inference , --reset-registry , or --reset-llama-catalog ; if explicit removal/reset flags are used, the affected files are archived or snapshotted first and only then cleaned/reset. - archiv / archive --full is the concise archive-first rotation for runtime/generated ballast plus registry/catalog reset. It does not replace --forget-inference or a self-learning reset. fruth research/ survives every clean/full/forget/reset mode unchanged, including nested caches, candidate reviews, Gold metadata and retained evidence. Archive includes one byte-verified copy of the whole tree in its existing space preflight; the active Research tree remains. A verification failure prevents cleanup. Successful conventional self-learning eval persistence synchronizes an observational candidate queue. Source labels stay historical; Gold promotion requires explicit curation. Secondary sync failure logs a warning without rolling back learning. Before a fresh reset, validate the current retained Gold store; after reset, validate Gold again and refresh candidate availability. Candidates can retain review metadata while losing uncurated raw provenance. See Research lifecycle https://github.com/fl0ri0/fruth/blob/main/fruth research/README.md and candidate recovery commands https://github.com/fl0ri0/fruth/blob/main/fruth research/candidates/README.md . Retention is internal only and does not authorize export or execution of a reset. Use exactly one owner for ollama serve at a time. - If Fruth manages your model lifecycle, stop the Homebrew background service first: - brew services stop ollama - Then start Fruth normally and let Fruth launch the ollama serve processes it needs. - Do not keep both Homebrew ollama service and Fruth-managed ollama serve processes active in parallel. - ./start multi models.sh now checks this and stops early with a clear fix message unless you explicitly override it via FRUTH ALLOW BREW OLLAMA SERVICE=1 . Why this matters: - parallel ollama serve owners can leave stale processes running after code changes, - restart/stop logs become misleading, - environment-dependent fixes can appear to "not work" because requests still hit an older process. Homebrew path note: - /opt/homebrew/bin/ollama and /opt/homebrew/opt/ollama/bin/ollama are normal Homebrew paths to the same installed binary, - seeing both in ps output does not mean Ollama was installed in the wrong place, - the real problem is duplicate process ownership, not the path spelling. The canonical project environment is: - .venv/ Install Python dependencies from requirements.txt . Local test runner: - python -m pytest MLX runtime note: - MLX PYTHON controls the shared MLX LM/VLM/Whisper interpreter path. - MLX AUDIO PYTHON can point to a separate interpreter for mlx audio.server . - If unset, Fruth also checks /opt/mlx-audio/venv/bin/python automatically for TTS. - Use a separate TTS venv when mlx-audio pins package versions that conflict with the LM/VLM stack in /opt/mlx/venv . - Whisper discovery safely preflights its Numba dependency without importing MLX or initializing Metal. An incompatible NumPy/Numba pair is reported as degraded and is not startable. - Run python scripts/fruthctl.py doctor runtime --json to distinguish a package/dependency incompatibility from a stale degraded instance that only needs a bounded restart. /api/start model is capability-aware and validates startup requests before launching a model instance. Supported capabilities: - chat : text/chat models default fallback . - vision analysis : OCR/vision analysis models. - image generation : image-generation models for example Flux family . - speech to text : speech transcription models for example Whisper on MLX . - text to speech : speech synthesis models for example Qwen3-TTS on MLX Audio . Capability-specific runtime behavior: - chat MLX LM : starts mlx lm.server ; registry metadata now records backend package=mlx lm , backend contract=mlx lm.server , and the LM-specific runtime knobs surfaced by Fruth prefill step size , prompt cache size , prompt cache bytes . - image generation Ollama : starts a dedicated Ollama instance without chat preload. - speech to text MLX : starts the local mlx whisper HTTP shim backend package=mlx whisper shim , not native mlx-audio ; the shim exposes /healthz , /v1/audio/transcriptions , and /api/transcribe . Transcription uses JSON with a local audio path , listener-bound Host/Origin checks and X-Fruth-Action: 1 , which the Fruth client supplies. See the Whisper caller contract https://github.com/fl0ri0/fruth/blob/main/docs/HTTP API BOUNDARY.md mlx-whisper-adapter ; the marker is not native-process authentication. - text to speech MLX : starts mlx audio.server and serves OpenAI-style speech synthesis through /v1/audio/speech ; Fruth records that the upstream package is broader than the current instance contract and also covers STT/STS upstream. - vision analysis MLX VLM : starts mlx vlm.server and uses its OpenAI-style multimodal routes; Fruth records the richer upstream contract including /v1/chat/completions , /v1/responses , /v1/models , /health , and /unload , plus the one-model-at-a-time / lazy-load semantics. This is also the generic path for MLX multimodal snapshots whose metadata resolves to image-to-text or image-text-to-text . Unsupported backend/capability combinations return 400 with a clear message instead of generic 500 . Available-model discovery is intentionally broader than startup eligibility: - /api/available models and fruthctl models list can show cached source entries that are present locally but not yet startable through the current backend contract. - Those entries carry discovery state , runnable , runnable checks , and disabled reason so callers can distinguish "seen in cache/catalog" from "can be launched now." - Interactive startup surfaces should only offer runnable entries as start choices. Keep /api/infer only for lower-level debugging, narrow compatibility cases, or older direct clients that have not moved yet. It is now a compatibility wrapper around the same shared infer execution used by /api/responses . New integrations and first-party UI flows should prefer /api/responses . - chat : - JSON prompt only or multipart with text/image/text-file attachments. - vision analysis : - multipart image upload, or PDF upload text-layer extraction + scan fallback via rendered pages . - image generation : - prompt required; optional image attachment for image-to-image flows. - speech to text : - multipart audio upload required. - text to speech : - text prompt required; backend persists generated audio to artifacts/audio/ and returns the saved path. - Session Controls for TTS now support speaker/style tuning via voice , instruct , speed , and pitch . - text prompt required; backend persists generated audio to Current direct UI/internal behavior: - Direct backend chat route /api/chat still exists only as a compatibility wrapper for specialized or older callers. - First-party UI execution flows, voice input, fruthctl send , and the Responses workbench use /api/responses as the canonical execution surface. - Session Controls are derived from the actual target model. MLX models that advertise reasoning effort use their declared non-Off default on first use; an explicit user choice, including Off, persists per model/backend. - General vision-language capability does not imply OCR/PDF controls. Document controls are shown only for recognized OCR-specialist families such as DeepSeek-OCR and GLM-OCR; other VLMs keep independently advertised controls such as reasoning. - /api/responses now rejects plain chat text that claims image/audio generation for a non-chat routed turn when no real image/audio artifact was produced. - Arena mode stays chat-only. - Voice input button UI : - Requires one running speech to text instance for example Whisper . - Records microphone audio, auto-stops on silence with a max-duration safety cap , transcribes via /api/responses , and inserts text into the prompt box. - If transcript ends with send or senden , UI treats it as a voice command and triggers Run automatically. - Works as prompt input for chat, vision, and image-generation tabs. - Requires one running - Speech output handling: - UI renders an inline audio player plus Open Folder and Download Audio actions for text to speech responses. - Backend persists synthesized audio under artifacts/audio/ . - In a canonical multi-phase text to speech - speech to text response, artifact existence is not enough to prove spoken meaning. The TTS fill records the exact final backend prompt and SHA-256 as tts semantic source ; its directly dependent STT fill records tts stt semantic evidence against only that producer source. Missing, drifted, ambiguous, or mismatching evidence blocks the branch with DEPENDENCY CHAIN REPAIR REQUIRED / repair dependency chain , even when a WAV and transcript physically exist. - For counted audio variants, explicitly labelled contiguous variants are the candidate authority. Runtime selects one speakable field per index and excludes sibling transcript claims, analysis, code, and JSON; duplicate, missing, or ambiguous slots fail closed instead of being spoken. - For Qwen3-TTS model families: - CustomVoice variants are the right fit when you want one stable named speaker plus optional emotion/style instructions. - VoiceDesign variants are the right fit when you want the voice itself described in natural language. - UI renders an inline audio player plus - Image output handling: - UI renders Open Folder local file manager and Download buttons below generated images. - Backend persists generated images to artifacts/images/ when needed and returns the saved path in API payload for local open actions. - UI renders - PDF document handling: - Local PDF parsing and rendering run in owned disposable processes with a separate 120-second preparation deadline, one shared preparation slot, and explicit input/output budgets. Timeouts and authoritative branch cancellation terminate and reap preparation; failures cannot count as partial success. See PDF preparation controls https://github.com/fl0ri0/fruth/blob/main/docs/CONTROL KNOBS.md pdf-page-preparation . This is process containment, not a filesystem/network sandbox or a hard memory ceiling. - PDF attachments are accepted in /api/responses and still supported by /api/infer compatibility calls. - File-aware external agent routes receive the original selected PDF under their existing file-sharing controls; page conversion is confined to backends that require text or images. - For vision analysis models and chat instances advertising image input including Apple AI single tabs , PDFs are treated image-first: each selected page is rendered and sent to the selected instance for recognition or analysis. Mixed documents retain both searchable text and text inside images through their rendered pages. - Apple Read text OCR and Read barcodes / QR codes also accept PDFs. They run the existing native image tool on each rendered page and return page-labelled detections with per-page receipts bound to the original PDF digest. A failed tool call is an explicit error; it does not fall back to model prose. - DeepSeek-OCR page extraction follows the official prompt pipeline from the upstream docs: - primary: <image + <|grounding| Convert the document to markdown. - fallback: <image + Free OCR. - primary: - Text-only targets and explicit pdf prefer text requests use pypdf to extract the existing text layer as document context. If some pages have no extractable text, a warning identifies that missing coverage; text extraction does not recognize text inside images. Explicit Apple OCR/barcode modes require rendered pages even if pdf prefer text is set. - For scanned PDFs, backend renders pages as images and runs page-wise OCR/context analysis. - Ollama OCR page execution uses /api/generate first and automatically retries with an emergency OCR prompt when Ollama returns empty content or upstream generate errors. - DeepSeek-OCR requests are sent with conservative runner options num ctx=4096 , num keep=0 to avoid sequence errors on some Ollama builds. - Prompt-echo responses where the model repeats user instructions instead of OCR text are treated as invalid page OCR output and trigger fallback automatically. - Optional request params: - pdf max pages blank by default; only set when you want to cap how many pages Fruth renders and OCRs for this request, max 500 - pdf dpi default 260 for vision analysis , 180 for chat, max 600 - pdf page timeout sec default 240 ; timeout per OCR page call - pdf page retry dpi default auto; retries failed pages with lower DPI - pdf max image side default 2400 ; constrains rendered page size for stability - pdf prefer text default false ; for vision OCR path we render PDF pages first - infer timeout sec default 1200 , max 7200 - pdf synthesize default false ; set true for one-pass global merge - Ollama page failures retain partial results and warnings. Empty page answers from compatible chat/vision transports retain a labelled gap and warning; they are not counted as successful recognition. Native Apple tool failures stop with the failing page number and receipts for earlier successful pages. A page cap retains the full document page count and a coverage warning. - Inline PDF output is a convenience view and may be shortened for UI stability; the saved markdown artifact is the completeness source of truth. - Text-layer PDF runs now persist both the extracted source text artifact and the model-produced result artifact when available. - PDF inference history is persisted to state/infer history.jsonl for explicit retrieval and reference. - Each incoming PDF is processed again using the current request settings. History remains available for explicit retrieval; the legacy reuse cached input is accepted for compatibility but has no effect. - Query stored insights via GET /api/infer history?file kind=pdf&limit=50 . - pypdf is installed by default for text-layer extraction. - On macOS, PDFKit draws pages into CoreGraphics bitmaps through the pyobjc-framework-Quartz dependency. All selected pages, intrinsic rotation, crop boxes and visible annotations are included. ImageIO encodes the page PNGs for the existing OCR/vision routes; rendering itself performs no OCR. Native bitmap rendering does not imply GPU acceleration. - requirements.txt installs the Quartz binding only on macOS, along with its PyObjC core/Cocoa dependencies. For an existing environment, apply the updated requirements explicitly using the install command above. Upstream wheels normally avoid compilation; source installs require Apple's command line developer tools PyObjC installation https://pyobjc.readthedocs.io/en/latest/install.html . Fruth's source archive contains neither wheels nor Apple frameworks. - Local PDF parsing and rendering run in owned disposable processes with a separate 120-second preparation deadline, one shared preparation slot, and explicit input/output budgets. Timeouts and authoritative branch cancellation terminate and reap preparation; failures cannot count as partial success. See Running model records in model ports.json now include: - modelName : canonical model identifier same semantic value as model . - backend : normalized backend ollama , mlx , llama cpp , or apple fm . - capability : normalized capability used by routing. - features : canonical feature flags such as vision input , audio input , image output , audio output , tool calling , function calling , computer use , and structured outputs . - feature sources : per-feature provenance labels such as explicit metadata , local template , curated override , capability contract , or conservative default . - inputs : normalized input modality list. - outputs : normalized output modality list. - backend metadata for Ollama-backed entries: a doc-aligned /api/show summary carrying fields such as capabilities , details , parameters , and derived context length . - backend metadata for MLX-backed entries: package-contract metadata describing which surface is in play mlx lm , mlx vlm , mlx audio , or mlx whisper shim , the upstream/native endpoint set, package capabilities, and key constraints such as lazy loading or one-model-at-a-time model binding. - backend metadata for Apple AI entries: the observed fm serve contract, exact health/models URLs, observation time and raw reported model entry. The API model is system ; the UI label is Apple AI. Optional macOS 27 discovery adds system model with the Mac's default AFM variant, context size, source and timestamp, shown in startup and the Models panel. Unreported exact-server variant, revision and context size remain unknown. See Apple AI https://github.com/fl0ri0/fruth/blob/main/docs/APPLE FOUNDATION MODELS.md . - backend package / backend contract : additive top-level provenance fields so callers can distinguish generic backend = mlx from the actual MLX package contract. - discovery state , runnable , runnable checks , disabled reason : available-model fields that now separate cache/catalog discovery from locally runnable backend truth. The legacy model field remains for backward compatibility. Dynamic runtime state now lives beside the stable registry in state/runtime status.json . This status registry tracks live metadata such as readiness, activity, port/process health, and recent success/error timing without overloading model ports.json . For Ollama-backed entries it also carries backend runtime , a doc-aligned /api/ps summary with live loaded-model facts such as expires at , size vram , context length , digest , and backend details . For MLX-backed entries the same nested field now carries package-aware live runtime facts such as the native base URL, package-specific health or infer routes, unload/model-list URLs where applicable, and the current model-binding strategy model bound at launch vs model selected per request . It is now fed by canonical start/stop plus successful or failed work through chat and /api/responses flows, with /api/infer and /api/chat retained only as compatibility plumbing where still needed. That split is intentional: - model ports.json is the stable registry plus attributed backend metadata. - state/runtime status.json is the live status layer plus backend-specific runtime facts /api/ps for Ollama, native-route/runtime binding facts for MLX, and observed server readiness for llama.cpp and Apple AI . - fruthctl instances list --json and GET /api/running instances merge both views so callers do not have to guess from model names alone. For external wrappers, Fruth now exposes a discovery manifest: - GET /api/runtime manifest - alias: GET /api/routing table The manifest advertises: - canonical responses endpoints /api/responses , /v1/responses - direct per-instance responses URLs /api/local provider/<instance id /v1/responses - running instance metadata instance id , model , backend , capability , features , feature sources , inputs , outputs , backend metadata , backend runtime , readiness/activity - capability aliases such as chat , image , ocr , stt , tts This is intended for external wrappers that need to self-dock against Fruth without hard-coding ports or internal route knowledge. Canonical /api/responses can now be targeted in three ways: - explicit instance id - wrapper alias / profile such as chat , image , ocr , stt , tts - canonical capability when only one default running instance for that capability should be chosen That means wrapper code can first read the manifest and then either: - call /api/local provider/<instance id /v1/responses directly for a fixed model, or - call /api/responses with alias , profile , or capability instead of hard-coding instance id Wrapper code should not need /api/infer or /api/chat unless it is intentionally targeting the lower-level compatibility surface. Feature-contract note: - capability is the coarse top-level mode. - features / inputs / outputs are the finer-grained routing contract for downstream clients. - Tool/function/computer-use flags stay conservative unless Fruth has explicit evidence, while modality flags and known local template evidence are projected directly when available. Active UI chat history persists under state/chat history/ , including the synthetic Responses workbench conversation. This is the canonical backend chat-history store for the active Fruth chat UI. Frontend/history shape note: - the page shell owns root state/startup plus timeline glue, while active frontend behavior lives in static/ui/messages.js , static/ui/conversations.js , static/ui/settings-history.js , static/ui/models.js , static/ui/message-state.js , and static/ui/request-lifecycle.js - durable chat history persists both request metadata and reusable artifacts - exact canonical Responses truth includes working frame for the final orchestration state plus response frame , a frozen local snapshot of request, target, route, runtime, non-default control snapshots, artifacts, error, output metadata, and the frozen working frame; bounded normal wire views expose refs and handles instead of hydrating these bodies - reusable settings artifacts can be promoted explicitly from response-frame controls through /api/settings artifacts ; they store replay request overrides under artifacts/settings/ - message.artifacts is the canonical reusable-file ledger for both user inputs and assistant outputs - request snapshot remains the durable request-context record, with request snapshot.input artifacts kept only for explicit user-side inputs instead of silently mirroring assistant outputs back into user turns - one-shot selected reference artifacts/messages are conversation-scoped UI state for the next turn, not a global workbench carry-over - lineage/rotation state for chat tabs is tracked through conversation metadata plus slot history ids - active workspaces center on the current or selected draft/conversation - saved chats live in the durable backend-driven History archive surface The model maintenance panel is backend-aware: - Ollama pulls/removes through the existing Ollama CLI path. - HF / MLX pulls/removes through the Hugging Face cache using the MLX venv hf CLI. 1. Pull/install the model locally first. 2. Confirm the exact model ID: - Ollama: ollama list - MLX / llama.cpp catalog-backed entries: check ./fruth ctl models list --json or /api/available models 3. Ollama: 4. Pick the correct capability: - chat - chat - OCR/vision - vision analysis - Flux/image - image generation - Whisper/STT - speech to text - Qwen3-TTS / speech synthesis - text to speech 5. chat - 6. Start via API/UI using matching backend + capability. 7. If startup fails: - Check the API error text for model-name suggestions or unsupported capability/backend guidance. - Re-run ./fruth ctl models list --json and inspect runnable plus disabled reason for the model before retrying. - Verify model naming for example x/flux2-klein vs flux2-klein:latest . POST /api/start model { "model": "x/flux2-klein", "backend": "ollama", "capability": "image generation" } POST /v1/audio/transcriptions { "audio path": "/absolute/path/to/audio.wav", "task": "transcribe", "language": "de" } POST /api/responses { "instance id": "mlx-community Qwen3-TTS-12Hz-0.6B-Base-bf16-mlx-11504", "input": "Guten Tag aus Fruth.", "voice": "Chelsie", "instruct": "Warm, calm, elegant German narration with a slight smile.", "speed": 0.95, "pitch": 1.1, "lang code": "de", "response format": "wav" } This single-phase example materializes one audio artifact. When a request also asks Fruth to confirm the exact spoken text, use the canonical TTS-to-STT dependency chain and read its Closure plus tts semantic source / tts stt semantic evidence ; a non-empty WAV or transcript alone is not semantic fulfillment. The expected source text is verification truth and is never inserted into the STT request.