{"slug": "i-built-an-editable-timeline-for-ai-agents-skills-mcp-and-transactional-video", "title": "I Built an Editable Timeline for AI Agents: Skills, MCP, and Transactional Video Editing", "summary": "A developer built Timeline Studio, an open-source, local-first browser video editor that lets AI agents edit video through an inspectable, structured edit plan rather than one-off FFmpeg commands. The system uses an editable .timeline project as the source of truth, with a shared command registry and reducers behind CLI, local STDIO MCP, and browser WebMCP transports, plus a project-aware dry run (project.diff) and idempotent operation IDs to prevent duplicate edits and stale-plan conflicts.", "body_md": "An AI agent can generate an FFmpeg command. But what happens when the next request is: “Shorten the second shot, move the voiceover and its captions together, and give me a project I can keep editing”?\n\nThat is the problem I have been working on in **Timeline Studio**, an open-source, local-first browser video editor with an Agent Skill and a shared command layer.\n\nThis post focuses on the editing architecture: how an agent turns a request into an inspectable plan, how that plan changes project state, and where rendering still has explicit limits.\n\nA rendered MP4 does not preserve the decisions needed for the next edit: clip identity, source ranges, timing, transforms, and caption/audio relationships.\n\nTimeline Studio uses an editable **.timeline** project as the source of truth. In the local command workflow, the archive contains project.json and embedded media. Asset import probes necessary metadata, computes SHA-256 integrity information, and embeds the media bytes.\n\nThis gives both the user and the agent a concrete state to return to. The user can reopen the project in the editor. The agent can inspect it before making another change.\n\nThe model can decide that a voiceover should start at three seconds. The application should enforce whether the clip exists, whether the time is valid, and how linked captions follow it.\n\nThe architecture looks like this:\n\n```\nUser request + media evidence\n             |\n     Skill: analyze and plan\n             |\n     Structured edit plan\n             |\n CLI / local MCP / browser WebMCP\n             |\n Shared command registry and reducers\n             |\n Editable project + verified output\n```\n\nThe transports have different environments. The CLI operates on local archives. The local STDIO MCP adapter exposes that command runner to an agent. Browser WebMCP, when exposed by the host, operates on the open editor and commits through its normal history.\n\nThe important constraint is to reuse editing rules rather than implement a separate editor for each transport.\n\nHere is a simplified plan that moves an existing voiceover to project time 3s:\n\n```\n{\n  \"schemaVersion\": 1,\n  \"project\": \"/projects/input.timeline\",\n  \"baseRevision\": 12,\n  \"dryRun\": false,\n  \"operations\": [\n    {\n      \"id\": \"move-voice-001\",\n      \"type\": \"timed.move\",\n      \"track\": \"audio\",\n      \"clipId\": \"voice-1\",\n      \"start\": 3\n    }\n  ],\n  \"output\": {\n    \"project\": \"/projects/output.timeline\"\n  }\n}\n```\n\nThe revision and clip ID are illustrative: they must come from inspecting the actual project.\n\nEach field has a job:\n\nFor this audio move, still-linked captions move by the same delta. That behavior belongs in the editing rules.\n\nThe workflow is:\n\n``` php\ninspect -> plan -> validate -> diff -> apply -> inspect\n```\n\nThe CLI exposes it directly:\n\n```\nnpm run agent -- project.inspect /projects/input.timeline\n\nnode skills/edit-timeline-studio/scripts/validate_edit_plan.mjs \\\n  /projects/plan.json\n\nnpm run agent -- project.diff /projects/plan.json\nnpm run agent -- project.run /projects/plan.json\n```\n\nThe standalone validator checks transport shape, finite time fields, and duplicate operation IDs. It does not prove that a plan is valid for a particular project.\n\n**project.diff is the project-aware dry run.** It uses the same command registry and reducers as application, without writing an archive. It checks supported operations and preconditions, then reports project fields, clip additions/removals, changed fields, ordering, and duration changes.\n\nThis makes accidental scope expansion visible. A request to move narration should not quietly change the visual sequence duration.\n\nThree mechanisms matter here.\n\nIf an operation completes but its response is lost, retrying the same operation ID should not insert the same clip twice. Previously applied IDs are treated as no-ops.\n\nIf the agent inspected revision 12 and the user subsequently edited the project, a stale plan containing new operations must fail with a revision conflict. The agent needs to inspect again and reconsider the plan.\n\nA failed batch should not leave a partially edited output archive. Local CLI and MCP writes also require a new output path and reject overwriting the input or an existing output.\n\nThese are familiar application-engineering techniques. They become especially useful when the caller is an agent that may retry or reason from outdated state.\n\nA timestamp can refer to source media, a clip-local position, or the final project.\n\nFor a constant-speed clip:\n\n```\nsourceTime = sourceIn + (timelineTime - clipStart) * speed\n```\n\nWith a speed curve, source-time mapping must accumulate the changing playback speed over time.\n\nAn agent identifying an action at source time 8s cannot assume that the cut belongs at project time 8s. Trimming, splitting, keyframes, thumbnails, and effect analysis all depend on maintaining that mapping.\n\nTracks also have different semantics. Main visuals form a contiguous sequence; narration, captions, and overlays have explicit timing. Whether a duration change shifts other tracks depends on ripple mode, links, and track locks.\n\nThese rules need to be explicit application behavior.\n\nThe Skill describes the workflow: inspect material, plan the edit, choose an execution path, and verify the result.\n\nThe MCP adapter exposes concrete tools, including:\n\n```\ntimeline_project_inspect\ntimeline_track_inspect\ntimeline_clip_inspect\ntimeline_project_diff\ntimeline_project_apply\ntimeline_project_render\n```\n\nThe local adapter is a thin transport over the existing command runner. It does not duplicate timeline editing logic.\n\nTo install the Skill:\n\n```\nnpx skills add MartinDelophy/ai-video-editor \\\n  --skill edit-timeline-studio\n```\n\nTo explore the repository and local MCP server:\n\n```\ngit clone https://github.com/MartinDelophy/ai-video-editor.git\ncd ai-video-editor\nnpm install\nnpm run mcp\n```\n\nInstalling the Skill does not install the complete runtime. Local MCP needs the checkout and Node dependencies; media operations need their corresponding tools. The Skill includes a host dependency-check workflow.\n\nThe current local project.render path supports the documented portable Visuals, Voiceover, and Music subset. It uses FFmpeg to produce H.264/AAC MP4 and ffprobe to check dimensions, duration, and audio-track presence.\n\nIt rejects unsupported composition features rather than silently dropping visible captions, overlays, transitions, or effects. Richer compositions require the browser renderer.\n\nVerification needs several levels: inspect the project, reopen it and resolve archived media, check the output container, and actually play the important visual and audio boundaries. A valid project structure alone cannot prove that the final mix sounds right.\n\nUseful entry points in the [repository](https://github.com/MartinDelophy/ai-video-editor):\n\nThe [command reference](https://github.com/MartinDelophy/ai-video-editor/blob/main/skills/edit-timeline-studio/docs/command-reference.md) documents the current contract and limits.\n\nThe larger lesson for me is that agent reliability depends on the application interface: explicit state, constrained operations, reviewable changes, and verifiable outputs.\n\nIf you are building agent tools or working on browser media, I would welcome feedback on the architecture. Real editing failures and reproducible cases are particularly useful. You can explore the [source](https://github.com/MartinDelophy/ai-video-editor), [try the editor](https://video-editor.ai-creator.top/), or browse the [editing handbook](https://github.com/MartinDelophy/timeline-studio-handbook).\n\n*Writing disclosure: this article was drafted with AI assistance using the project's source code and documentation.*", "url": "https://wpnews.pro/news/i-built-an-editable-timeline-for-ai-agents-skills-mcp-and-transactional-video", "canonical_source": "https://dev.to/martindelophy/i-built-an-editable-timeline-for-ai-agents-skills-mcp-and-transactional-video-editing-15og", "published_at": "2026-11-03 07:20:35+00:00", "updated_at": "2026-11-03 07:22:45.237599+00:00", "lang": "en", "topics": ["ai-agents", "agent-protocols", "ai-tools", "developer-tools", "generative-ai"], "entities": ["Timeline Studio", "FFmpeg", "MCP", "WebMCP"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/i-built-an-editable-timeline-for-ai-agents-skills-mcp-and-transactional-video", "markdown": "https://wpnews.pro/news/i-built-an-editable-timeline-for-ai-agents-skills-mcp-and-transactional-video.md", "text": "https://wpnews.pro/news/i-built-an-editable-timeline-for-ai-agents-skills-mcp-and-transactional-video.txt", "jsonld": "https://wpnews.pro/news/i-built-an-editable-timeline-for-ai-agents-skills-mcp-and-transactional-video.jsonld"}}