# Is There a Reverse Video Search? How It Works, How to Build One

> Source: <https://dev.to/mixpeek/is-there-a-reverse-video-search-how-it-works-how-to-build-one-4g0e>
> Published: 2026-10-24 23:07:57+00:00

Yes. Reverse video search exists, but the big search engines do not offer it as a single upload box. The usual workflow is to pull keyframes out of the video and run each one through a reverse image search. Under the hood, a dedicated system does the same thing at scale: it samples frames or detects shot boundaries, turns each frame into a perceptual hash or an embedding, stores them in an index, and matches the frames of your query clip against that index. The best-matching frames also carry timestamps, so the answer comes back as "this file, at 01:42". Consumer tools search the public web's image index. If you want to search a video library you own, you build or buy the index yourself.

**Google Lens.** Google's [help page](https://support.google.com/websearch/answer/1325808) says you can search with an image by uploading a file, dragging it in, or pasting an image URL. Take a frame from the video, save it as an image, and upload it. The page does not mention video input, so the frame step is yours.

**Bing Visual Search.** Microsoft's [support page](https://support.microsoft.com/en-US/bing/using-bing-visual-search) describes searching the web with an image from the camera icon on Bing Images. It accepts uploads, pasted images and URLs. Same workflow: export a frame, upload it.

**TinEye.** TinEye [fingerprints the image you submit and compares it with its index](https://help.tineye.com/article/233-how-does-tineye-work), which it builds by crawling the web. Its help center says it finds [exact and altered copies, including cropped, color adjusted, resized, heavily edited or slightly rotated ones](https://help.tineye.com/article/235-can-tineye-find-similar-images-does-tineye-do-facial-recognition), and that it does not commonly return similar images of the same subject. Its help pages describe image search only.

**InVID-WeVerify.** This browser plugin was built for journalists and fact-checkers. Its [keyframe tab](https://www.invid-project.eu/tools-and-services/invid-verification-plugin/) takes a video URL or an uploaded file, segments it into keyframes, and lets you right-click a keyframe to search it on Google, Yandex, TinEye and Baidu. It automates the manual workflow above, so it is the quickest way to try it.

**YouTube Content ID.** Content ID does match video, but only for rights holders. [Google's help page](https://support.google.com/youtube/answer/2797370) says owners must hold exclusive rights to a substantial body of original material that is frequently uploaded to YouTube. They supply reference files, and every new upload is scanned against that library. A regular user cannot submit a clip and ask where it appears.

All four web options search an index of images. None of them has your video in it. That has practical consequences:

The pipeline has four steps.

**1. Sample frames.** The simplest approach is a fixed rate such as one frame per second. A more economical one is shot-based: ffmpeg's `select` filter exposes a `scene` score between 0 and 1, and the [documentation](https://ffmpeg.org/ffmpeg-filters.html#select_002c-aselect) says a threshold between 0.3 and 0.5 is generally a sane choice. Extract one representative frame per shot and store its timestamp.

**2. Choose a fingerprint.** Perceptual hashes (average, perceptual, difference, wavelet) from the [imagehash](https://github.com/JohannesBuchner/imagehash) library produce a short code, and the distance between two codes is a Hamming distance. They are cheap and handle re-encoding, resizing and mild color shifts. They are weak against heavy crops and flips. Embeddings from a vision model such as CLIP are larger and need a vector index, but they tolerate more edits and also support searching by meaning. Many systems run a hash for near-exact copies and embeddings for everything else.

**3. Index.** Store one row per sampled frame: video ID, timestamp, hash or vector. For hashes, a linear scan over a few million rows is fine. Past that, use a vector database or a BK-tree.

**4. Align in time.** Matching single frames gives noisy results, because unrelated frames sometimes collide. Real matches agree on an offset. If query frame at 3 s matches library frame at 63 s, and query frame at 4 s matches library frame at 64 s, the offset is 60 s in both cases. Count votes per (video, offset) and return the winner. That also gives you the start timestamp of the clip inside the source file.

Here is a small version using hashes and offset voting. It needs `ffmpeg` on your PATH, plus `pip install imagehash pillow`.

``` python
import subprocess, tempfile, glob
from collections import Counter
from PIL import Image
import imagehash

def frames(video, fps=1):
    """Return [(timestamp_seconds, phash)] sampled at `fps`."""
    with tempfile.TemporaryDirectory() as d:
        subprocess.run(
            ["ffmpeg", "-loglevel", "error", "-i", video,
             "-vf", f"fps={fps}", f"{d}/f_%06d.png"],
            check=True,
        )
        out = []
        for i, path in enumerate(sorted(glob.glob(f"{d}/f_*.png"))):
            out.append((i / fps, imagehash.phash(Image.open(path))))
        return out

# Build the index: one row per sampled frame.
index = []  # (video_id, timestamp, hash)
for vid in ["a.mp4", "b.mp4"]:
    index += [(vid, t, h) for t, h in frames(vid)]

def search(clip, max_dist=10):
    votes = Counter()
    for tq, hq in frames(clip):
        for vid, ts, h in index:
            if hq - h <= max_dist:          # Hamming distance
                votes[(vid, round(ts - tq))] += 1
    if not votes:
        return None
    (vid, offset), n = votes.most_common(1)[0]
    return vid, offset, n

print(search("query_clip.mp4"))  # e.g. ('a.mp4', 60, 14)
```

The third number is the vote count, which works as a confidence signal. The distance threshold of 10 on a 64-bit hash is a starting point, so tune it on clips you know are matches and clips you know are not. The first frame of the extracted images is labeled 0 seconds, which is approximate to within one sampling interval.

For production use, replace the linear scan with an index, add shot detection so static scenes do not produce hundreds of near-identical rows, and run a second embedding pass for mirrored or cropped copies.

*Disclosure: I work on Mixpeek.* Mixpeek indexes video in your own S3 or GCS bucket, splits it into scenes, and embeds frames. A retriever then returns the matching file and timestamp for an uploaded clip or image. We compare ten approaches in [Best Reverse Video Search Tools](https://mixpeek.com/curated-lists/best-reverse-video-search-tools), and the [reverse search tutorial](https://mixpeek.com/tutorials/reverse-search) walks through the setup.