I built an AI video clip finder that runs 100% in your browser — no uploads, no API, no GPU costs

A developer built ClipGG's AI Video Highlights tool, which runs entirely in the browser using Web Audio API and FFmpeg.wasm to find and extract video highlights without file uploads, server costs, or GPU requirements. The tool processes audio locally by decoding and downsampling, then scores segments based on RMS, zero-crossing rate, and peak volume with content-type-specific weights. It handles iOS limitations by pre-extracting audio to WAV format and supports fast export via stream copying for H.264/AAC MP4 files.

Every time I used Opus Clip or Vidyo.ai, the same thought hit me: I’m paying $20/month to upload my video to someone else’s server, wait in a queue, and hope their AI finds something useful. So I built an alternative that runs entirely in the browser. No file uploads. No subscriptions. No server costs on my end. The result is ClipGG’s AI Video Highlights tool — and in this post I’ll walk through exactly how it works technically. Finding highlights in a long video is genuinely hard to automate well. The expensive approach: transcribe with Whisper, feed text to GPT-4, profit. But that requires a backend, API costs, and user uploads. I wanted zero server involvement. That meant doing everything with browser APIs. The pipeline has four stages: js const arrayBuffer = await file.arrayBuffer // The file never leaves the device. // ArrayBuffer is passed directly to Web Audio API. I use OfflineAudioContext to decode audio faster than real-time, then downsample to 8000–11025 Hz before analysis. This reduces RAM usage from ~115MB to ~19MB for a 10-minute video. js // Decode in a Web Worker so the UI never freezes const tempCtx = new OfflineAudioContext 1, 44100, 44100 const audioBuffer = await tempCtx.decodeAudioData arrayBuffer // Downsample manually — OfflineAudioContext does NOT resample automatically function downsample channelData, originalRate, targetRate { const ratio = originalRate / targetRate const output = new Float32Array Math.floor channelData.length / ratio for let i = 0; i < output.length; i++ { const start = Math.floor i ratio const end = Math.min Math.floor i + 1 ratio , channelData.length let sum = 0 for let j = start; j < end; j++ sum += channelData j output i = sum / end - start } return output } For each 500ms window I compute: Then I do relative normalization so a quiet podcast and a loud gaming stream are scored fairly against themselves: js // Relative normalization — key insight const normalizedRms = seg.rms - globalMinRms / globalMaxRms - globalMinRms Different content types use different weights: | Mode | RMS | ZCR | Peak | |---|---|---|---| | Gaming | 0.20 | 0.35 | 0.20 | | Podcast | 0.50 | 0.05 | 0.20 | | Funny | 0.15 | 0.20 | 0.35 | | General | 0.30 | 0.20 | 0.25 | The selector groups high-scoring segments into zones, finds the peak moment in each zone, and centers a 30–90 second clip around it. A diversity radius of 12 seconds prevents three clips from covering the same moment. js const combinedSignal = seg.score ?? 0 + seg.energyChange ?? 0 2.0 + seg.volumePeak ?? 0 1.5 // Center the clip around the strongest combined signal, // not just the loudest sustained section Safari on iOS can’t decode video containers via AudioContext.decodeAudioData . It only accepts clean audio files. The fix: detect iOS and pre-extract audio with FFmpeg.wasm before passing it to the Web Audio API: js const isIOS = /iPhone|iPad|iPod/i.test navigator.userAgent if isIOS { await ffmpeg.exec '-i', 'input video', '-vn', '-acodec', 'pcm s16le', // WAV — guaranteed to work on all iOS versions '-ar', '44100', '-ac', '1', 'audio.wav' // Pass audio.wav to Web Audio instead of the original video } WAV/PCM is uncompressed and works reliably on every iOS version. AAC containers are not. Once highlights are found, FFmpeg.wasm cuts the clips: // Fast path: H.264 + AAC + MP4 = stream copy, no re-encoding // A 90-second clip exports in ~2–3 seconds await ffmpeg.exec '-ss', String clip.start , '-i', 'input', '-t', String clip.end - clip.start , '-c', 'copy', // copy bytes, don't re-encode '-avoid negative ts', 'make zero', '-movflags', '+faststart', outputName Non-standard formats MOV, MKV, AV1 get converted to MP4 first before the analysis pipeline runs. This also fixed all the “file won’t export” bugs from iPhone footage. OfflineAudioContext doesn’t resample. I assumed new OfflineAudioContext 1, length, 8000 would give me 8kHz audio. It doesn’t. You get whatever sample rate the source file has. Downsampling has to be manual. Transfer, don’t copy ArrayBuffers. worker.postMessage { arrayBuffer }, arrayBuffer transfers ownership with zero memory copy. Without the second argument you’re doubling RAM usage. -ss before -i for stream copy, after for re-encode. This one cost me an hour. For -c copy , seek before input for speed. For re-encoding, seek after input for frame accuracy. The tool is live and free at: 👉 https://clipgg.uk/en/ai-video-highlights https://clipgg.uk/en/ai-video-highlights Drop a video, pick a mode Gaming / Podcast / Funny / General , and get three highlight clips with timestamps in about 30 seconds. No account. No upload. Works on desktop Chrome, Firefox, and now iOS Safari too. Curious what others think about the audio scoring approach — would love feedback on the algorithm in the comments.