{"slug": "building-chatcut-conversational-video-editing-in-the-browser-with-cloud-models", "title": "Building ChatCut: Conversational Video Editing in the Browser with Cloud Models", "summary": "A developer built ChatCut, a conversational video editing workspace in the Anna edition of the open-source Timeline Studio project, which lets users edit video through natural-language requests without installing desktop software or running their own model inference. The browser holds the editing state and runs a tool loop against Anna's llm.complete host API, capping turns at 16 model calls and validating every proposed edit against a captured project state token before it can be committed, with automatic application limited to validated edits and new narration or audio results held for review.", "body_md": "“Shorten the second shot, then move the narration and its captions together.”\n\nThat request sounds simple until you try to connect a language model to an actual video project. The model needs to identify the right clips, understand their timing, preserve links, and avoid applying a plan after the user has changed the timeline.\n\nI am building **ChatCut**, the conversational editing workspace in the Anna edition of my open-source project, **Timeline Studio**. The goal is to let users edit through conversation without installing desktop editing software or setting up their own model inference environment.\n\n[GitHub repository](https://github.com/MartinDelophy/ai-video-editor) · [Anna integration notes](https://github.com/MartinDelophy/ai-video-editor/blob/codex/anna-validation/docs/anna-chatcut.md)\n\nThis is an architecture walkthrough, including cloud video generation and the music integration under development. Availability depends on the Anna release, account grants, and selected model. The standalone editor and the Anna edition have different capabilities.\n\nChatCut combines a conversation panel with the live preview and editable timeline. The cloud model interprets requests and chooses tools. The browser owns the editing state, validates operations, previews changes, and performs composition/export.\n\n```\nUser request\n     |\nBrowser: inspect project and available assets\n     |\nCloud LLM: choose a supported tool\n     |\nBrowser: validate and execute\n     |                        |\nTimeline edit          Cloud generation job\n     |                        |\nReview / apply         Download and decode result\n     |                        |\nEditable timeline <--- Explicit placement request\n     |\nBrowser composition and export\n```\n\n“No desktop install” does not mean “every operation runs on a server.” This division keeps direct editing and playback close to the user while using platform-hosted models for planning and supported generation tasks.\n\nChatCut uses Anna's llm.complete host API. The browser maintains the tool loop; it does not create a separate cloud Agent session or expose an unrestricted standard MCP server.\n\nEach turn starts by inspecting the project. The model receives a filtered catalog of registered editor tools, their input schemas, bounded conversation context, and the user's request.\n\nExamples include:\n\n```\ntimeline_project_inspect\ntimeline_track_inspect\ntimeline_clip_inspect\ntimeline_assets_inspect\ntimeline_edit_preview\ntimeline_video_generate\ntimeline_export_prepare\n```\n\nThe loop has a hard limit of 16 model calls per turn. Only registered operations are dispatched. Model-generated JavaScript, shell commands, and arbitrary media URLs are not executed as editing instructions.\n\nThe model proposes actions. The application enforces their meaning.\n\nA conversation turn captures an initial project state token. Any generated edit preview must still match that state before it can be committed.\n\nIf the user trims a shot while the model is planning, the old result becomes stale. Re-reading the project inside the same turn does not silently authorize a plan based on a different initial state.\n\nThe shared review engine checks asset and clip IDs, supported operations, timing, media availability, and locked tracks.\n\nWhen automatic application is enabled, ordinary validated edits can be committed by the host, with inline **Undo** and **View changes** receipts. New narration and audio-processing results require listening review and remain pending. Historical receipts cannot apply or undo an unrelated transaction.\n\nThis is what makes the conversation part of an editable application rather than a detached suggestion box.\n\nA filename is weak evidence about a video. ChatCut can inspect selected source frames with permission, using an imported asset ID and explicit source times.\n\nThe decoder is independent of the visible preview, so sampling does not move the user's playback position. A call can return at most six bounded JPEG frames, with eighteen frames allowed per conversation request.\n\nThese are source frames. They exclude the editor's rendered captions, overlays, and effects.\n\nSparse samples can help answer a local question about composition. They cannot establish complete scene understanding, exact cut detection, or audio transcription. The model should not claim to have watched the whole video when it has only seen a handful of frames.\n\nThe Anna connector uses the real video.generate and video.getJob host APIs.\n\n```\nExplicit generation request\n        |\nValidate model parameters and app permission\n        |\nSubmit once -> retain job ID -> poll existing job\n        |\nDownload actual media bytes\n        |\nDecode and commit to My assets\n```\n\nAn app-level media grant is required. Selecting a video model alone does not establish permission, and submitting a job consumes the user's Anna CU.\n\nA recorded live text-to-video check submitted a five-second, 480p, 16:9 request. It returned a playable 832x480 video, approximately 5.184 seconds long, with H.264 video and AAC audio. The downloaded asset was stored and the existing timeline was unchanged.\n\nThat validates one concrete path. Image-conditioned generation and other model/resolution combinations need their own checks; the adapter's existence is not evidence that every provider combination works.\n\nStopping local waiting also does not imply that a remote generation job was canceled or refunded.\n\nA generated video might need another iteration. Committing it directly to the timeline would conflate “the provider returned a file” with “the user accepted the result.”\n\nCompleted video outputs are downloaded, decoded, and saved in My assets. Placement uses a separate inspected editing plan when the user requests it.\n\nThis separation preserves the original footage and makes recovery easier. Download failure, generation failure, and placement failure become distinct states instead of one ambiguous “try again” button.\n\nThe Anna music connector queries the official catalog:\n\n```\nllm.catalog({ serviceType: 'music_gen' });\n```\n\nThe selected model determines prompt length, lyrics support, instrumental mode, available formats, and whether duration is adjustable. A fixed-duration model receives no invented duration parameter. Lyrics go through the official lyrics field when supported.\n\nNo provider secret is embedded in the frontend; requests use the platform's permissioned host interface.\n\nThe current ChatCut music tool prepares a prompt and opens the official generation form. It does not silently submit a billed music job: the user clicks Generate in that form. Conversation-driven music preparation and beat-aware montage are working-draft features, not a promise that every published edition already includes them.\n\nThe implemented connector persists the exact request and an idempotency key before submission. An ambiguous outcome keeps its receipt; recovery looks for the same request rather than immediately inventing a new key.\n\nIf a successful job's signed download URL expires and returns 403, the connector refreshes the result through getJob. It does not call generate again.\n\nDownloaded audio is decoded to establish duration and waveform data, marked as music, and committed to My assets for explicit Music-track placement. Playback uses the downloaded object URL rather than a short-lived signed URL.\n\nThe repository documents build, contract, and local-browser checks for the music connector. Those checks do not establish live paid-generation audio quality, which still needs an authorized end-to-end run.\n\nThe montage work under development treats templates as starting points. Uploaded and generated music should feed the same analysis path: decoded waveform, beat and accent candidates, then a model-planned sequence of cuts.\n\n``` php\nMusic -> waveform and timing evidence\n                    |\nTemplate + source clips + user direction\n                    |\n              Cut plan\n                    |\nValidate times, source IDs, and available media\n                    |\n            Editable sequence\n```\n\nA constant BPM grid alone does not prove that the cuts follow the track's actual accents or sections. This is why analysis, planning, and validation are separate stages.\n\nChatCut can prepare export of the full timeline, a selected visual or overlay clip, or an explicit time interval. The visible export action starts rendering after the user clicks it.\n\nExports reuse the editor's composition pipeline, including the relevant captions, effects, and audio. This avoids maintaining a separate renderer for conversational edits.\n\nThe browser still does real work. Performance depends on the source media, browser capabilities, device, and composition complexity; cloud planning does not remove those constraints.\n\nStart with the [open-source repository](https://github.com/MartinDelophy/ai-video-editor), and the Anna implementation on the codex/anna-validation branch:\n\nThe [standalone browser editor](https://video-editor.ai-creator.top/) lets you explore Timeline Studio's editing foundation. ChatCut and the platform-hosted generation paths described here belong to the Anna edition; use the integration notes to understand their release and permission requirements.\n\nWhat interests me most about this architecture is the connection between an accessible conversation and a durable editing model. Users can ask for a change, inspect the result, undo it, and continue manually—all inside the same browser workspace.\n\nIf you build agent tools, browser media applications, or generation pipelines, I would welcome feedback and reproducible failure cases through the [repository](https://github.com/MartinDelophy/ai-video-editor).\n\n*Disclosure: I maintain the project discussed here. This article was drafted by an AI assistant using the current source code and implementation records.*", "url": "https://wpnews.pro/news/building-chatcut-conversational-video-editing-in-the-browser-with-cloud-models", "canonical_source": "https://dev.to/martindelophy/building-chatcut-conversational-video-editing-in-the-browser-with-cloud-models-43h9", "published_at": "2026-11-06 10:20:45+00:00", "updated_at": "2026-11-06 10:24:06.182321+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "generative-ai", "developer-tools", "ai-products"], "entities": ["ChatCut", "Timeline Studio", "Anna", "MartinDelophy", "GitHub"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/building-chatcut-conversational-video-editing-in-the-browser-with-cloud-models", "markdown": "https://wpnews.pro/news/building-chatcut-conversational-video-editing-in-the-browser-with-cloud-models.md", "text": "https://wpnews.pro/news/building-chatcut-conversational-video-editing-in-the-browser-with-cloud-models.txt", "jsonld": "https://wpnews.pro/news/building-chatcut-conversational-video-editing-in-the-browser-with-cloud-models.jsonld"}}