cd /news/computer-vision/optical-flow-temporal-inpainting-eli… · home › topics › computer-vision › article
[ARTICLE · art-171461] src=dev.to ↗ pub= topic=computer-vision verified=true sentiment=· neutral

Optical Flow & Temporal Inpainting: Eliminating Video Watermarks Without Frame Degradation

A developer published a Python-based pipeline that removes watermarks from AI-generated video by reconstructing occluded regions with dense optical flow (Farneback and RAFT) and bidirectional temporal reference propagation instead of spatial blurring. The approach warps authentic background texture from neighboring frames into the masked watermark bounding box, avoiding the smudging and pulsating artifacts that frame-by-frame inpainting produces on moving 4K footage.

recommendation

When removing occlusions from video, borrow texture from adjacent frames using dense optical flow and bidirectional blending rather than applying spatial blur or per-frame inpainting.

more in playbook: Verifying model output
by read3 min views6 publishedNov 18, 2026

When modern diffusion-based video generators like Kling AI or Runway Gen-3 synthesize high-definition footage, they embed composite visual markers into the corner bounding boxes. For video editors, creative technologists, and archival engineers, the typical commercial solution is disastrous: Gaussian spatial blurring or bicubic interpolation.

Applying a spatial blur over a dynamic 4K video frame produces a smudged, pulsating smudge that immediately reveals human tampering to compression algorithms and human viewers alike.

In this guide, we dive into how to reconstruct occluded video frames cleanly using dense optical flow (Farneback and deep RAFT vectors) paired with bidirectional temporal reference propagation. We will look at why naive frame-by-frame inpainting fails, how temporal consistency is maintained, and walk through an executable Python implementation.

Traditional image inpainting (such as Navier-Stokes or Fast Marching algorithms) computes pixel replacements by propagating color gradients strictly from neighboring pixels within the same 2D image plane:

$$\Delta I(x, y) = \nabla I(x, y) \cdot \vec{N}(x, y)$$

In a static photograph, this is often sufficient. In video, however, the camera or background subjects are in constant motion. When you apply 2D spatial inpainting independently across 60 consecutive frames:

The solution is not to invent pixels from thin air, but to borrow the authentic, unoccluded background texture from neighboring temporal frames ($t-k$ and $t+k$). Because the camera or subjects pan across the canvas, the pixels occluded at time $t$ were almost certainly exposed and recorded a few frames earlier or later.

To transfer pixels from frame $t-1$ to frame $t$, we compute dense optical flow vectors $(\vec{u}, \vec{v})$. Optical flow assumes brightness constancy across infinitesimal time intervals:

$$I(x, y, t) = I(x + \Delta x, y + \Delta y, t + \Delta t)$$

By calculating the displacement vector for every pixel in the neighborhood of the watermark bounding box, we warp the unmasked texture from preceding and succeeding keyframes directly into the occluded region.

Here is an architectural breakdown of the bidirectional temporal pipeline:

[Frame t-2] ---- (Forward Flow) ---> \
                                      [Warped Accumulator] ---> [Alpha Weighted Blend] ---> Clean Frame t
[Frame t+2] <--- (Backward Flow) --- /

Below is a self-contained Python script implementing dense temporal flow reconstruction. It estimates motion vectors around the watermark quadrant and projects historical pixel data into the masked region.

import cv2
import numpy as np

def reconstruct_temporal_quadrant(
    prev_frame: np.ndarray,
    curr_frame: np.ndarray,
    mask: np.ndarray
) -> np.ndarray:
    """
    Reconstructs occluded pixels in curr_frame using forward optical flow
    borrowed from unoccluded regions in prev_frame.

    :param prev_frame: Reference frame (H, W, 3) uint8
    :param curr_frame: Target frame containing watermark (H, W, 3) uint8
    :param mask: Binary mask where 255 represents watermark pixels (H, W) uint8
    :return: Restored frame with temporal consistency
    """
    prev_gray = cv2.cvtColor(prev_frame, cv2.COLOR_BGR2GRAY)
    curr_gray = cv2.cvtColor(curr_frame, cv2.COLOR_BGR2GRAY)

    flow = cv2.calcOpticalFlowFarneback(
        curr_gray, prev_gray, None,
        pyr_scale=0.5, levels=3, winsize=15,
        iterations=3, poly_n=5, poly_sigma=1.2, flags=0
    )

    h, w = curr_gray.shape
    grid_x, grid_y = np.meshgrid(np.arange(w), np.arange(h))

    map_x = (grid_x + flow[..., 0]).astype(np.float32)
    map_y = (grid_y + flow[..., 1]).astype(np.float32)

    warped_prev = cv2.remap(
        prev_frame, map_x, map_y, 
        interpolation=cv2.INTER_LINEAR,
        borderMode=cv2.BORDER_REFLECT
    )

    mask_normalized = (mask.astype(np.float32) / 255.0)[..., np.newaxis]
    output_frame = (curr_frame * (1.0 - mask_normalized) + warped_prev * mask_normalized).astype(np.uint8)

    dilated_mask = cv2.dilate(mask, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3)))
    feather_boundary = dilated_mask - mask
    if np.any(feather_boundary):
        output_frame = cv2.inpaint(output_frame, feather_boundary, 3, cv2.INPAINT_TELEA)

    return output_frame

In production environments, optical flow warping fails under two specific scenarios:

Eliminating watermarks from high-fidelity AI-generated media is an exercise in signal processing and temporal coherency, not brute-force blurring. By pairing dense flow vector fields with temporal texture warping, developers can preserve grain structure, edge sharpness, and color fidelity.

To test these deconvolution and inpainting models on Kling AI and Sora footage without configuring a local PyTorch pipeline, check out the browser-based utility developed by the engineering team at Analytix9 via the Kling AI Watermark Remover portal.

── more in #computer-vision 4 stories · sorted by recency
── more on @kling ai 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/optical-flow-tempora…] indexed:0 read:3min 2026-11-18 · —