{"slug": "c-tool-that-extracts-text-from-scrolling-screen-recordings", "title": "C++ Tool That Extracts Text From Scrolling Screen Recordings", "summary": "A developer has released Palimpsest, a high-performance C++ CLI tool that extracts text from screen recordings of scrolling documents, deduplicating overlapping frames and outputting clean .txt or .docx files. The tool runs at 323 FPS on a laptop iGPU and achieves 94.56% accuracy on dense banking regulatory text. Its architecture uses a staged producer-consumer pipeline with lock-free ring buffers and a 'dwell gating' mechanism to only OCR stable frames, reducing 21,000 frames to just 5 for a 5-minute video.", "body_md": "*Most of the code was AI-generated from a detailed architecture prompt. I designed the pipeline, the AI wrote the C++. Here's what we built.*\n\nI had a problem. My dad takes 1.5 hour long banking regulation classes recorded as screen recordings of someone scrolling through RBI circulars and banking law documents. No download link. No PDF. Just a video of text scrolling past.\n\nI didn't want to type it all out manually.\n\nSo instead of spending 3 hours typing, I spent an afternoon designing a solution and prompting an AI to write the C++. The result is **Palimpsest** — a high-performance CLI tool that extracts text from screen recordings of scrolling documents, deduplicates overlapping frames, and outputs a clean `.txt`\n\nor `.docx`\n\nfile.\n\nIt runs at **323 FPS** on a laptop iGPU and gets **94.56% accuracy** on dense banking regulatory text.\n\nGitHub: [https://github.com/plexescor/Palimpsest](https://github.com/plexescor/Palimpsest)\n\nIf you've ever needed to extract text from a screen recording, you know the pain:\n\nExisting tools fail here. Generic OCR tools handle single images. Cloud video OCR services (ScreenApp, etc.) charge monthly subscriptions, cap video length, and you're uploading private content to someone's server. Manual transcription of a 1.5 hour recording is genuinely painful.\n\nThere was no good free, local, offline tool that specifically handled the **scrolling document** problem — deduplicating overlapping frame content and stitching it into a clean linear document.\n\nSo I built one.\n\n**Palimpsest** — a historical term for a manuscript page that has been scraped clean and rewritten over. Exactly what this tool does to a scroll recording.\n\nThis is where it gets interesting. The tool isn't just \"loop through frames and OCR each one.\" That would be slow and produce terrible results due to duplicate content across scroll frames.\n\nThe architecture is a strictly staged producer-consumer pipeline with lock-free SPSC (Single Producer Single Consumer) ring buffers between every stage:\n\n```\n[Decoder Thread]\n     ↓ SPSC RingBuffer<RawFrame, 128>\n[Frame Filter & Dwell Gatekeeper]\n     ↓ SPSC RingBuffer<FilteredFrame, 64>\n[Preprocessor Thread]\n     ↓ WorkQueue<PreprocessedFrame>\n[OCR Worker Pool — N Threads]\n     ↓ ResultQueue<OcrResult> (unordered)\n[Reorder Buffer]\n     ↓ ordered stream\n[Dedup & Stitch Thread]\n     ↓\n[Structure Recovery]\n     ↓\n[Output Writer — TXT or DOCX]\n```\n\nEvery queue is bounded. Backpressure propagates upstream automatically — if OCR workers slow down, the work queue fills, the preprocessor blocks, the frame filter blocks, the decoder blocks. RAM stays bounded regardless of video length. A 3 hour video uses the same memory as a 5 minute one.\n\nThe most important design decision in the whole tool is something I call **dwell gating**, and it's what separates Palimpsest from naive frame-by-frame OCR approaches.\n\nMost people's first instinct for this problem is: detect when the frame is changing (scrolling happening) and capture those frames. **This is completely backwards.**\n\nHere's why. The frames captured *during* active scrolling are:\n\nWhat you actually want are the frames where the educator **stopped scrolling and is talking about the content** — because those frames are:\n\nDwell gating works by tracking frame stability over time. A frame is only forwarded to OCR when it has been **continuously stable for --stable-dwell-ms milliseconds** (default 5000ms — five full seconds). During scrolling, the stability counter resets. During pauses, it accumulates. Only settled, stable, clean frames ever reach Tesseract.\n\nThe result: from 21,000 frames in a 5 minute 50 second video, only **5 frames** were forwarded to OCR in normal operation. Five perfect frames instead of thousands of garbage ones. 94.56% accuracy on the output.\n\nEven with dwell gating, consecutive captured frames overlap significantly. Someone scrolls down, pauses — you see the bottom half of section A and the top half of section B. Next pause — full section B. Next — overlap of B and C.\n\nPalimpsest handles this with **fuzzy suffix-prefix stitching** via rapidfuzz:\n\n`--overlap-window`\n\ncharacters, default 300)`--overlap-window`\n\ncharacters)`rapidfuzz::fuzz::partial_ratio`\n\nto find the overlap boundary`--overlap-threshold`\n\n(default 75): find exact boundary, append only new content`[GAP IN CONTENT]`\n\nmarker, append full frameA rolling **seen-lines history** handles cross-GAP deduplication — if someone scrolls back up and the same content reappears later, it's detected and skipped via fuzzy matching.\n\nTwo preprocessing backends, selected via `--backend`\n\n:\n\n**CPU (default)** — standard `cv::Mat`\n\npipeline, Gaussian blur, Otsu threshold, Hough deskew. Fully portable, zero dependencies beyond OpenCV.\n\n**OpenCL** — same pipeline but using `cv::UMat`\n\nthroughout, dispatched transparently to the iGPU via OpenCV's OpenCL backend. On startup, enumerates devices and prints which one was selected. Falls back to CPU silently if no OpenCL device found.\n\nOCR always runs on CPU regardless of backend — Tesseract's OpenCL path is unreliable and slower on integrated GPUs due to driver overhead.\n\nReal hardware: **AMD Ryzen AI 7 350** (8 cores, 16 threads), **Radeon 860M** iGPU (8 CU, 8GB shared), 16GB DDR5 @ 5600 MT/s, Arch Linux.\n\nVideo: **1920×1200, H.264 @ 6 Mbps, 60 FPS**, 21,000 frames (5 minutes 50 seconds).\n\n**Normal operation (dwell mode):**\n\n| Backend | Forwarded to OCR | Time | Avg FPS | Real-Time Factor |\n|---|---|---|---|---|\n| CPU | 5 | 1m 06s | 318 FPS | ~5.3× faster than real-time |\n| OpenCL | 5 | 1m 05s | 323 FPS | ~5.4× faster than real-time |\n\nIn dwell mode the bottleneck is purely the video decoder. CPU and OpenCL are identical because almost no frames reach preprocessing.\n\n**Preprocessing + OCR stress test (20,000 frames forced through full pipeline):**\n\n| Backend | Time | Avg FPS | Per-Frame |\n|---|---|---|---|\n| CPU | 2m 47s | ~120 FPS | ~8.3ms/frame |\n| OpenCL | 2m 41s | ~107 FPS | ~9.4ms/frame |\n\nCPU beats OpenCL in the stress test because the bottleneck is Tesseract (CPU-only) and OpenCL dispatch overhead costs more than the preprocessing savings at this scale.\n\n**Accuracy benchmark on banking regulatory text (RBI circulars, committee names, legal language):**\n\n**94.56% accuracy** on a real-world banking regulation document. The errors were: one line missed due to fast scroll (tunable with `--min-interval-ms`\n\n), one phantom character from compression artifact (`discipline`\n\n→ `disciplinec`\n\n), minor list prefix heuristic misfires. All fixable in a 5 minute review pass.\n\n```\n# Simplest run\n./build/bin/palimpsest recording.mkv\n\n# OpenCL backend + DOCX output + crop region\n./build/bin/palimpsest recording.mkv -b opencl -f docx --crop 20,60,1880,1100 -o output.docx\n\n# Config file (recommended for repeated use)\n./build/bin/palimpsest recording.mkv -c config.csv\n\n# Filter watermarks and recurring headers\n./build/bin/palimpsest recording.mkv --ignore ignore_phrases.txt\n\n# Print hardware info and exit\n./build/bin/palimpsest recording.mkv -b opencl --noshit\n```\n\nKey flags:\n\n`--stable-dwell-ms`\n\n— how long a frame must stay stable before OCR (default 5000ms)`--crop x,y,w,h`\n\n— exclude UI chrome, taskbars, window ribbons`--ignore`\n\n— strip watermarks and recurring boilerplate`--dump-images`\n\n— dump the exact frames that got OCR'd, for debugging`--bench-preprocess N`\n\n— stress test mode, force N frames through full pipelineFull flag reference and a documented `config.csv`\n\nare in the repo.\n\nNo vcpkg. No system package manager. One command builds everything from source:\n\n```\ngit clone https://github.com/plexescor/Palimpsest.git\ncd Palimpsest\n./compile.sh\n```\n\nCMake FetchContent pulls and builds:\n\nRuntime requirement: GStreamer or FFmpeg for video decoding (standard on any Linux with video playback), tessdata in `./tessdata/`\n\nor via `TESSDATA_PREFIX`\n\n.\n\nAlmost all of it.\n\nI designed the full pipeline architecture in conversation with Claude — every stage, the ring buffer design, the dwell gating concept, the dedup stitching algorithm, the backend abstraction interface, the DOCX XML structure. Then I wrote a detailed prompt describing the entire system and Claude generated the C++ implementation in one shot.\n\nThe AI wrote:\n\n`.docx`\n\nfiles`cv::UMat`\n\nWhat I contributed was the architecture, the dwell gating insight, the understanding of why motion-based extraction is wrong for this use case, the benchmark methodology, and the real-world test on actual banking lecture content.\n\nThe result was 94.56% accuracy on first real-world test. That's what happens when you understand the problem well enough to describe the solution precisely.\n\nIf you've ever had any of these problems:\n\nPalimpsest is free, runs locally, never uploads your content anywhere, processes any video length, and takes about a minute for a typical lecture recording.\n\n`--stable-dwell-ms`\n\nor set it to 0 for motion-threshold mode.\n\n```\ngit clone https://github.com/plexescor/Palimpsest.git\ncd Palimpsest\n./compile.sh\n./build/bin/palimpsest your_recording.mkv\n```\n\nDrop the output quality issues you hit in the comments — real feedback on diverse content types helps tune the defaults.\n\n*Built with C++17, OpenCV, Tesseract, rapidfuzz, miniz, {fmt}, CLI11. Named after ancient manuscript pages scraped clean and rewritten over.*\n\n*GitHub: https://github.com/plexescor/Palimpsest*\n\n", "url": "https://wpnews.pro/news/c-tool-that-extracts-text-from-scrolling-screen-recordings", "canonical_source": "https://dev.to/plexescor/c-tool-that-extracts-text-from-scrolling-screen-recordings-30f6", "published_at": "2026-08-24 05:32:18+00:00", "updated_at": "2026-08-24 05:42:45.485807+00:00", "lang": "en", "topics": ["developer-tools", "computer-vision", "artificial-intelligence"], "entities": ["Palimpsest", "Tesseract", "RBI", "GitHub", "ScreenApp"], "alternates": {"html": "https://wpnews.pro/news/c-tool-that-extracts-text-from-scrolling-screen-recordings", "markdown": "https://wpnews.pro/news/c-tool-that-extracts-text-from-scrolling-screen-recordings.md", "text": "https://wpnews.pro/news/c-tool-that-extracts-text-from-scrolling-screen-recordings.txt", "jsonld": "https://wpnews.pro/news/c-tool-that-extracts-text-from-scrolling-screen-recordings.jsonld"}}