When modern diffusion-based video generators like Kling AI or Runway Gen-3 synthesize high-definition footage, they embed composite visual markers into the corner bounding boxes. For video editors, creative technologists, and archival engineers, the typical commercial solution is disastrous: Gaussian spatial blurring or bicubic interpolation.
Applying a spatial blur over a dynamic 4K video frame produces a smudged, pulsating smudge that immediately reveals human tampering to compression algorithms and human viewers alike.
In this guide, we dive into how to reconstruct occluded video frames cleanly using dense optical flow (Farneback and deep RAFT vectors) paired with bidirectional temporal reference propagation. We will look at why naive frame-by-frame inpainting fails, how temporal consistency is maintained, and walk through an executable Python implementation.
Traditional image inpainting (such as Navier-Stokes or Fast Marching algorithms) computes pixel replacements by propagating color gradients strictly from neighboring pixels within the same 2D image plane:
$$\Delta I(x, y) = \nabla I(x, y) \cdot \vec{N}(x, y)$$
In a static photograph, this is often sufficient. In video, however, the camera or background subjects are in constant motion. When you apply 2D spatial inpainting independently across 60 consecutive frames:
The solution is not to invent pixels from thin air, but to borrow the authentic, unoccluded background texture from neighboring temporal frames ($t-k$ and $t+k$). Because the camera or subjects pan across the canvas, the pixels occluded at time $t$ were almost certainly exposed and recorded a few frames earlier or later.
To transfer pixels from frame $t-1$ to frame $t$, we compute dense optical flow vectors $(\vec{u}, \vec{v})$. Optical flow assumes brightness constancy across infinitesimal time intervals:
$$I(x, y, t) = I(x + \Delta x, y + \Delta y, t + \Delta t)$$
By calculating the displacement vector for every pixel in the neighborhood of the watermark bounding box, we warp the unmasked texture from preceding and succeeding keyframes directly into the occluded region.
Here is an architectural breakdown of the bidirectional temporal pipeline:
[Frame t-2] ---- (Forward Flow) ---> \
[Warped Accumulator] ---> [Alpha Weighted Blend] ---> Clean Frame t
[Frame t+2] <--- (Backward Flow) --- /
Below is a self-contained Python script implementing dense temporal flow reconstruction. It estimates motion vectors around the watermark quadrant and projects historical pixel data into the masked region.
import cv2
import numpy as np
def reconstruct_temporal_quadrant(
prev_frame: np.ndarray,
curr_frame: np.ndarray,
mask: np.ndarray
) -> np.ndarray:
"""
Reconstructs occluded pixels in curr_frame using forward optical flow
borrowed from unoccluded regions in prev_frame.
:param prev_frame: Reference frame (H, W, 3) uint8
:param curr_frame: Target frame containing watermark (H, W, 3) uint8
:param mask: Binary mask where 255 represents watermark pixels (H, W) uint8
:return: Restored frame with temporal consistency
"""
prev_gray = cv2.cvtColor(prev_frame, cv2.COLOR_BGR2GRAY)
curr_gray = cv2.cvtColor(curr_frame, cv2.COLOR_BGR2GRAY)
flow = cv2.calcOpticalFlowFarneback(
curr_gray, prev_gray, None,
pyr_scale=0.5, levels=3, winsize=15,
iterations=3, poly_n=5, poly_sigma=1.2, flags=0
)
h, w = curr_gray.shape
grid_x, grid_y = np.meshgrid(np.arange(w), np.arange(h))
map_x = (grid_x + flow[..., 0]).astype(np.float32)
map_y = (grid_y + flow[..., 1]).astype(np.float32)
warped_prev = cv2.remap(
prev_frame, map_x, map_y,
interpolation=cv2.INTER_LINEAR,
borderMode=cv2.BORDER_REFLECT
)
mask_normalized = (mask.astype(np.float32) / 255.0)[..., np.newaxis]
output_frame = (curr_frame * (1.0 - mask_normalized) + warped_prev * mask_normalized).astype(np.uint8)
dilated_mask = cv2.dilate(mask, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3)))
feather_boundary = dilated_mask - mask
if np.any(feather_boundary):
output_frame = cv2.inpaint(output_frame, feather_boundary, 3, cv2.INPAINT_TELEA)
return output_frame
In production environments, optical flow warping fails under two specific scenarios:
Eliminating watermarks from high-fidelity AI-generated media is an exercise in signal processing and temporal coherency, not brute-force blurring. By pairing dense flow vector fields with temporal texture warping, developers can preserve grain structure, edge sharpness, and color fidelity.
To test these deconvolution and inpainting models on Kling AI and Sora footage without configuring a local PyTorch pipeline, check out the browser-based utility developed by the engineering team at Analytix9 via the Kling AI Watermark Remover portal.