{"slug": "removing-video-watermarks-entirely-in-the-browser-with-mi-gan-onnx-runtime-web", "title": "Removing Video Watermarks Entirely in the Browser with MI-GAN, ONNX Runtime Web, and WebGPU", "summary": "A developer built a browser-based video editor that removes watermarks and unwanted objects entirely on the user's device using MI-GAN, ONNX Runtime Web, and WebGPU, eliminating upload latency, infrastructure costs, and privacy concerns. The implementation handles multiple repair regions, moving watermarks, time ranges, progressive frame previews, and final video encoding without sending media to a server.", "body_md": "Watermark and unwanted-object removal is one of those features that looks simple in a model demo and becomes much more complicated when you put it inside a real video editor.\n\nThe obvious implementation is to upload the media to a GPU server, run an inpainting model, and return the result. That works, but it introduces upload latency, infrastructure cost, and privacy concerns—especially for large or unpublished videos.\n\nFor our browser video editor, we took a different approach:\n\nRun MI-GAN locally with ONNX Runtime Web and WebGPU, so images and videos never need to leave the user's device.\n\nThis article covers more than model inference. It explains the product-level work required for multiple repair regions, time ranges, moving watermarks, progressive frame previews, before/after comparison, cancellation, and final video encoding.\n\nUse object-removal tools only on media you own or are authorized to edit. Removing ownership or attribution marks from third-party content may violate licenses or platform rules.\n\nA server-side pipeline usually requires:\n\nBrowser-local inference moves the initial cost to downloading the model. Once the runtime and model artifacts are cached, repeated edits are much faster to start, and the original media remains local.\n\nOur implementation uses:\n\nImage repair is a single inference task. Video repair has a time axis, so the pipeline is substantially different:\n\n``` php\nSource media\n  -> user defines repair regions and time ranges\n  -> resolve regions active at the current timestamp\n  -> decode a video frame\n  -> build a mask\n  -> run MI-GAN inference\n  -> composite the repaired pixels over the source frame\n  -> commit the progressive preview\n  -> encode the processed frame\n  -> create a new media asset\n```\n\nThe model is only one stage. A production-quality experience also requires the canvas, playhead, processed-frame count, and progress indicator to share the same clock.\n\nIn image mode, the user draws one or more rectangles over the canvas. We store each rectangle in normalized coordinates so the selection remains correct across different preview sizes.\n\n``` js\nfunction createMask(width, height, regions) {\n  const canvas = new OffscreenCanvas(width, height);\n  const context = canvas.getContext(\"2d\");\n\n  context.fillStyle = \"#000\";\n  context.fillRect(0, 0, width, height);\n\n  context.fillStyle = \"#fff\";\n  for (const region of regions) {\n    context.fillRect(\n      region.x * width,\n      region.y * height,\n      region.width * width,\n      region.height * height\n    );\n  }\n\n  return context.getImageData(0, 0, width, height);\n}\n```\n\nEach region uses values between `0`\n\nand `1`\n\n. The same rectangle can therefore be mapped back to the full-resolution source instead of the CSS-sized preview.\n\nAfter inference, we do not immediately commit the result. The editor displays a draggable vertical comparison line:\n\n```\n<ComparisonView\n  before={sourceUrl}\n  after={repairedUrl}\n  position={comparePosition}\n  onPositionChange={setComparePosition}\n/>\n```\n\nThat distinction matters because inpainting does not recover a hidden ground truth. It synthesizes plausible pixels from the surrounding context, so users need a clear way to inspect the result before applying it.\n\nOur first video prototype repaired only the frame currently visible in the editor. It looked correct during a single-frame test but was obviously wrong for a video.\n\nA watermark exists for a time range, not for one frame. It may also move: for example, it can appear in the bottom-left corner for the first few seconds and later switch to the top-right.\n\nWe represent every repair region as an independent time-scoped object:\n\n``` js\nconst repairRegion = {\n  id: \"region-1\",\n  startTime: 0,\n  endTime: 4,\n  keyframes: [\n    { time: 0, x: 0.72, y: 0.82, width: 0.22, height: 0.10 },\n    { time: 4, x: 0.08, y: 0.08, width: 0.22, height: 0.10 }\n  ]\n};\n```\n\nThe rectangle between two recorded positions is interpolated:\n\n```\nfunction interpolateRegion(a, b, ratio) {\n  return {\n    x: a.x + (b.x - a.x) * ratio,\n    y: a.y + (b.y - a.y) * ratio,\n    width: a.width + (b.width - a.width) * ratio,\n    height: a.height + (b.height - a.height) * ratio\n  };\n}\n```\n\nThis supports fixed watermarks, watermarks that jump to another corner, and slowly moving overlays without creating a separate repair job for every frame.\n\nReal media may contain a logo and a text overlay at the same time. Both image and video workflows therefore need multiple selections.\n\nAt each video timestamp, we resolve all active regions and combine them into one mask:\n\n``` js\nconst activeRegions = regions\n  .filter((region) => {\n    return time >= region.startTime && time <= region.endTime;\n  })\n  .map((region) => resolveRegionAtTime(region, time));\n```\n\nThis data model also gives us a practical editing interface:\n\nCreating the model session or running inference on the main thread can freeze the entire editor. We keep a long-lived model session inside a Web Worker and transfer frame data to it.\n\n``` js\nconst session = await ort.InferenceSession.create(modelUrl, {\n  executionProviders: [\"webgpu\", \"wasm\"]\n});\n```\n\nWebGPU is preferred. If it is unavailable or initialization fails, we explicitly fall back to WASM and report the actual backend in the UI.\n\n```\nworker.postMessage(\n  {\n    type: \"repair\",\n    requestId,\n    imageBitmap,\n    regions\n  },\n  [imageBitmap]\n);\n```\n\nThe Worker and inference session remain alive for later requests. Model and runtime assets are stored in a versioned Cache Storage entry, so a second repair should never present itself as another model download.\n\nInpainting models commonly expect a fixed input size. Resizing and regenerating the entire video frame for a small watermark can unintentionally change faces, text, and fine texture outside the selected area.\n\nOur processing rules are:\n\n```\ncontext.drawImage(originalFrame, 0, 0);\ncontext.save();\ncontext.globalCompositeOperation = \"source-over\";\ncontext.drawImage(\n  repairedPatch,\n  patchX,\n  patchY,\n  patchWidth,\n  patchHeight\n);\ncontext.restore();\n```\n\nKeeping regions reasonably tight improves both performance and output stability.\n\nAn early version used three independent notions of time:\n\nThe result was confusing: the progress bar might show 40% while the preview was still near 25%.\n\nThe fix was to make the **committed visible frame the single UI clock**:\n\n``` js\nfor (let frameIndex = 0; frameIndex < totalFrames; frameIndex += 1) {\n  const frame = await decodeFrame(frameIndex);\n  const repairedFrame = await repairFrameIfNeeded(frame, frameIndex);\n\n  await onFrameCommitted(repairedFrame, frameIndex);\n\n  reportProgress({\n    completedFrames: frameIndex + 1,\n    totalFrames\n  });\n}\n```\n\nFrames outside every repair range must still call `onFrameCommitted`\n\n. Otherwise, the preview and playhead appear frozen whenever inference is skipped.\n\nWe also decode the progressive preview image and give the browser one paint before advancing the completed-frame count:\n\n```\nawait image.decode();\nsetPreviewFrame(image.src);\nawait new Promise(requestAnimationFrame);\nsetCompletedFrame(frameIndex + 1);\n```\n\nDuring processing, the left side now displays the source frame for the current timestamp while the right side displays the repaired frame for that same timestamp. The comparison line stays interactive throughout the process.\n\nReaching 100% repaired frames does not mean the output file is ready. We still need to initialize the encoder, encode the frames, and create the final media asset.\n\nWe expose these as distinct phases:\n\n```\n1. Preparing the model\n2. Repairing frames\n3. Loading the video encoder\n4. Encoding the repaired video\n5. Creating the new asset\n6. Complete\n```\n\nTrying to compress all of these into a single unexplained percentage makes the UI appear stuck at the end. A phase label tells the user what the browser is actually doing.\n\nFor long videos, cancellation is not optional. Changing the button text to \"Cancelling...\" is not enough.\n\nA real cancel path must:\n\n`VideoFrame`\n\nand `ImageBitmap`\n\nobjects.\n\n```\nif (abortSignal.aborted) {\n  throw new DOMException(\"Repair cancelled\", \"AbortError\");\n}\n```\n\nWe disable duplicate cancel requests immediately, but return to an editable state only after cleanup has completed. The original media remains unchanged until the user explicitly applies the new result.\n\nOur first UI placed all controls in the editor's right-side property panel. That panel quickly became too small for region drawing, time ranges, keyframes, preview, comparison, and progress.\n\nThe final product keeps **AI Repair** as a lightweight capability entry point and opens the actual work in a larger modal:\n\nThe mobile layout uses the same mental model in a vertical workspace. Applying closes both the repair modal and the properties drawer; cancelling closes only the modal.\n\nBrowser AI performance varies widely with GPU, resolution, codec, and browser implementation. Rather than promising a universal frame rate, we focused on these principles:\n\nThe first model load is still a cost, but cached repeat operations become a much better fit for an interactive editor.\n\nMI-GAN is lightweight and fast enough to be practical in a browser, but it is not perfect:\n\nFrame-by-frame inpainting does not guarantee temporal consistency. Optical flow, region tracking, and temporally aware restoration models are natural next steps.\n\nThe biggest lesson from building this feature was:\n\nSuccessful model inference is only the beginning. The real product is the editing flow that lets users inspect, trust, cancel, and apply the result.\n\nRepairing one image in a demo and shipping multi-region, time-aware, progressively previewed video repair are completely different engineering problems.\n\nThis implementation runs locally in the browser and inserts the result as a new media asset, leaving the original recoverable.\n\nIf you are experimenting with WebGPU, ONNX Runtime Web, browser-side media processing, or video-editor UX, I hope these implementation details save you a few iterations.", "url": "https://wpnews.pro/news/removing-video-watermarks-entirely-in-the-browser-with-mi-gan-onnx-runtime-web", "canonical_source": "https://dev.to/martindelophy/removing-video-watermarks-entirely-in-the-browser-with-mi-gan-onnx-runtime-web-and-webgpu-1jdi", "published_at": "2026-07-27 08:45:31+00:00", "updated_at": "2026-07-27 09:06:41.397484+00:00", "lang": "en", "topics": ["machine-learning", "computer-vision", "developer-tools", "ai-tools", "ai-infrastructure"], "entities": ["MI-GAN", "ONNX Runtime Web", "WebGPU"], "alternates": {"html": "https://wpnews.pro/news/removing-video-watermarks-entirely-in-the-browser-with-mi-gan-onnx-runtime-web", "markdown": "https://wpnews.pro/news/removing-video-watermarks-entirely-in-the-browser-with-mi-gan-onnx-runtime-web.md", "text": "https://wpnews.pro/news/removing-video-watermarks-entirely-in-the-browser-with-mi-gan-onnx-runtime-web.txt", "jsonld": "https://wpnews.pro/news/removing-video-watermarks-entirely-in-the-browser-with-mi-gan-onnx-runtime-web.jsonld"}}