{"slug": "i-am-tired-of-taking-screenshots-and-explaining-them-to-claude-code-while", "title": "I am tired of taking screenshots and explaining them to Claude code while working.", "summary": "A software developer built a macOS Swift app that captures cursor position, on-screen pointing gestures, and synchronized voice input to generate context briefs for AI coding agents like Claude Code and Cursor. The tool uses AXUIElementCopyElementAtPosition, SCScreenshotManager, and Vision OCR to identify what a user points at, binding pointing words such as \"this\" to cursor settles to resolve ambiguous references. The developer is considering a free BYO-key tier and a lifetime subscription around $40, with all screenshots, OCR text, and briefs kept local to the Mac.", "body_md": "Okay so a little context first,\n\nI work in an **SDE role** and work in multiple windows at once like i have many apps open in my mac.\n\nI faced a problem of **screenshotting** everything and explaining the context and their relations to *claude code* again and again like for little UI changes and such or any bug.\n\nI got this idea and started working on it , made a swift app which will track cursor and when your press a hotkey it starts only then and when you hold a hotkey it will draw over the areas your point with your cursor and you can keep talking while explaining what you need like you would to a real colleague. The voice timings come from whisper and they get attached with the cursor positions and screenshots and form a full event files.\n\nthose get stored as context everytime you do some actions like draw a circle , point to anything or something like that.\n\nand at the end we generate a context using those events that you can just drop to any AI agent of your choice like claude,cursor,codex etc. and it just starts working on it right off the bat.\n\nA bit on how it works\n\nThe hard part was never the screenshot. It's the word \"this\".\n\n\"Add a status key to this response\" means nothing on its own. It only works if you know what \"this\" was aimed at. So while a session runs I keep two streams: where the cursor settled, and what you were saying at the time. Then I have to figure out which settles were actually pointing at something and which were just my hand sitting on the trackpad. Cursor position alone can't tell you that.\n\nWhat worked was watching for the pointing words. If \"this\" or \"here\" lands on a settle, that settle was a pointing act. If a settle sits in the middle of silence, it was nothing. That binding is most of the app.\n\nFor the screenshot side I use AXUIElementCopyElementAtPosition against a system-wide element, which gives me whatever is under the cursor plus its accessibility label and window title. SCScreenshotManager crops the pixels and Vision runs OCR on the crop. So the brief ends up carrying the name of the thing you pointed at and not just a picture of it, which is usually what the agent was guessing at anyway.\n\nThere are no accounts. The trial is keyed to a hash of the machine's platform UUID rather than anything in preferences, so deleting a plist doesn't get you a fresh one. Screenshots, OCR text and the brief never leave the Mac. The audio is the only thing that goes anywhere and it's gone as soon as the text exists.\n\nHere is the producthunt page for more context on it:\n\nAnd below is the site to get it:\n\nI am thinking of providing a free version with BYO option and a lifetime subscription for around 40ish dollars. \n\nNeed some advice and people who face this problem and what do they generally use to solve it?", "url": "https://wpnews.pro/news/i-am-tired-of-taking-screenshots-and-explaining-them-to-claude-code-while", "canonical_source": "https://dev.to/maddy30445r/i-am-tired-of-taking-screenshots-and-explaining-them-to-claude-code-while-working-4bio", "published_at": "2026-09-17 18:21:24+00:00", "updated_at": "2026-09-17 18:53:06.081586+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "ai-products"], "entities": ["Claude Code", "Cursor", "Codex", "Whisper", "Vision", "Product Hunt", "Apple", "macOS"], "alternates": {"html": "https://wpnews.pro/news/i-am-tired-of-taking-screenshots-and-explaining-them-to-claude-code-while", "markdown": "https://wpnews.pro/news/i-am-tired-of-taking-screenshots-and-explaining-them-to-claude-code-while.md", "text": "https://wpnews.pro/news/i-am-tired-of-taking-screenshots-and-explaining-them-to-claude-code-while.txt", "jsonld": "https://wpnews.pro/news/i-am-tired-of-taking-screenshots-and-explaining-them-to-claude-code-while.jsonld"}}