{"slug": "multi-agent-harness-for-visual-design", "title": "Multi-Agent Harness for Visual Design", "summary": "Myli, a provider-neutral, pre-alpha Python harness for agents that propose RFC 6902 changes to application-owned JSON design documents, has been released. The harness never persists or applies returned candidates to application state and is independent of Pydantic, canvas formats, rendering stacks, model providers, ORMs, progress transports, and persistence systems. It includes public protocols for main and vision models, optional LiteLLM implementations, and a domain-neutral AgentTool with capabilities, per-run call budgets, timeouts, and result-size limits.", "body_md": "Myli is a provider-neutral, pre-alpha Python harness for agents that propose RFC 6902 changes to application-owned JSON design documents. It never persists or applies the returned candidate to application state.\n\nThe core is independent of Pydantic, canvas formats, rendering stacks, model providers, ORMs, progress transports, and persistence systems.\n\nRead the [Myli documentation](https://myli.readthedocs.io/en/latest/) for the\ncomplete guide and API reference.\n\n```\nresult = await myli.run(\n    request=user_request,\n    design=current_design,\n    can_edit=True,\n    capabilities={\"media.transform\"},\n    history=history,\n    on_event=handle_event,\n    on_step=persist_step,\n)\n```\n\nRunResult contains the user-facing message, optional validated candidate, changed flag, proposed patch, approved run-scoped assets, all tool outcomes, step traces, run ID, and final provider metadata. Successful and unsuccessful tool outcomes are also available as filtered properties.\n\n``` python\nfrom typing import Any\n\nfrom myli import DesignSpec, ModelRequest, ModelResponse, Myli\n\ndef load_untrusted_document(value: Any) -> dict[str, Any]:\n    if not isinstance(value, dict):\n        raise ValueError(\"document must be an object\")\n    return value\n\ndesign_spec = DesignSpec(\n    name=\"design\",\n    schema={\"type\": \"object\"},\n    validator=load_untrusted_document,\n    serializer=lambda document: document,\n    normalizer=normalize_trusted_stored_document,\n    input_migrator=migrate_stored_document,\n)\n\nclass ApplicationMainModel:\n    async def complete(self, request: ModelRequest) -> ModelResponse:\n        return await application_transport.complete(request)\n\nmyli = Myli(\n    design_spec=design_spec,\n    main_model=ApplicationMainModel(),\n    vision_model=application_vision_model,\n    renderer=application_renderer,\n)\n```\n\nDesignSpec has deliberately separate methods for trusted stored input and untrusted model candidates. Candidate JSON is validated at runtime against the schema and must serialize back without coercion, default insertion, field dropping, or any other silent rewrite. Stored input may opt into migration and normalization before each run.\n\nMainModel and VisionModel are public protocols. Applications may inject different transports, endpoints, credentials, or fakes. LiteLLMMainModel and LiteLLMVisionModel are optional implementations:\n\n```\npython -m pip install \"myli[litellm]\"\n```\n\nThe LiteLLM main implementation supports structured, JSON, and text output modes, rejects options that override client-owned request fields, and translates provider failures into Myli's stable exception hierarchy.\n\nPydantic remains optional:\n\n``` python\nfrom myli.integrations.pydantic import PydanticDesignSpec\n```\n\nAgentTool is domain-neutral. A tool declares a JSON input schema, capabilities, per-run call budget, timeout, and result-size limit, then receives a ToolContext with the current run ID, isolated evidence, zero-based model step, run-unique step and tool-batch IDs, and its zero-based position and size within that batch:\n\n``` python\nasync def execute(arguments, context):\n    ...\n```\n\nMyli validates arguments and results, enforces capabilities and limits, and records a ToolOutcome with one of succeeded, failed, rejected, timed_out, or deferred. Calls are sequential by default. Parallel execution requires both the harness option and an explicit parallel_safe declaration on every call in the batch. Each tool can select FailureMode.RETURN_ERROR or FailureMode.RAISE.\n\nToolMiddleware can allow, reject, or defer work before execution and observe the outcome afterward. Applications can use it for approval, audit, tenancy, ordering, transactions, and rate limiting. The step and batch metadata lets middleware apply a rule exactly once per model work phase, including when a batch executes in parallel.\n\nCandidatePolicy runs before rendering a proposal and before returning the final candidate. CandidateContext includes capabilities, approved assets, and all run-scoped outcomes; successful_tool_outcomes is the authorization-safe subset.\n\nNamed asset providers return generic Asset values with application-defined kinds and metadata. Myli assigns run-scoped references, detects conflicting provider identities, applies search budgets and result limits, and tracks provenance.\n\nAn asset can contain a RenderedArtifact preview or an async preview_loader. Previews load only after discovery and explicit inspection, are cached per run, and are bounded by timeout, byte-size, and media-type checks. Loading failures are returned to the model and can be retried. Binary artifacts never appear in step trace dictionaries.\n\nWhen a renderer and vision model are configured, render_design validates the candidate and application policies before rendering, validates the artifact, and returns visual feedback as a tool result. A changed candidate cannot be rendered while editing is disabled, but the unchanged current design can be rendered diagnostically.\n\nEvents cover the run, model, tool, and validation lifecycle. They contain run and step IDs, status, safe messages, elapsed time, and an optional tool name, but no arguments or results. Applications can map them to logs, metrics, SSE, WebSockets, or ignore them.\n\nStepTrace retains normalized model output, provider-exposed reasoning, tool calls and outcomes, validation failures, response ID, model, finish reason, usage, and latency. It provides to_dict() and redacted(). Async on_step and a configurable trace redactor make no persistence assumptions.\n\nHarnessLimits bounds model steps, retries, renders, searches, search results, asset inspections, patches, documents, history, artifacts, operation timeouts, and optionally the entire run. Every custom tool retains its own limits.\n\napply_json_patch implements add, remove, replace, move, copy, and test, including root operations and escaped pointers, against a deep copy. JsonPatchLimits guard operation count, patch bytes, pointer depth, value bytes, document bytes, invalid indexes, non-JSON values, non-finite numbers, child moves, and candidate expansion.\n\nCancelledError is never wrapped. Cancellation propagates into model, renderer, vision, search, preview, and custom-tool work. All counters, assets, outcomes, and caches live in local run state, so one Myli instance can safely serve concurrent requests.\n\n```\nuv sync\nuv run ruff format --check .\nuv run ruff check .\npython -m unittest discover -s tests -v\n```\n\nMyli is distributed under the MIT License.", "url": "https://wpnews.pro/news/multi-agent-harness-for-visual-design", "canonical_source": "https://github.com/EightPotions/Myli", "published_at": "2026-08-22 15:19:37+00:00", "updated_at": "2026-08-22 15:43:54.487428+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "ai-infrastructure"], "entities": ["Myli", "LiteLLM", "Pydantic", "RFC 6902"], "alternates": {"html": "https://wpnews.pro/news/multi-agent-harness-for-visual-design", "markdown": "https://wpnews.pro/news/multi-agent-harness-for-visual-design.md", "text": "https://wpnews.pro/news/multi-agent-harness-for-visual-design.txt", "jsonld": "https://wpnews.pro/news/multi-agent-harness-for-visual-design.jsonld"}}