{"slug": "show-hn-let-your-ai-agents-paint-big-arrows-boxes-and-text-on-your-screen", "title": "Show HN: Let your AI agents paint big arrows, boxes and text on your screen", "summary": "Developer Franz Zenhofer released bigarrow, an MIT-licensed macOS CLI and agent skill that lets AI agents draw click-through arrows, boxes and text on screen to point a human at a specific button or window without stealing focus or requiring any permission. The tool installs via `brew install franzenzenhofer/tap/bigarrow` and `bigarrow install-skill`, which teaches Claude Code (~/.claude/skills) and Codex (~/.agents/skills), and targets elements by label, coordinate, rectangle, mouse position, window or Peekaboo snapshot ID. It never clicks, types or captures anything, and is built as a single Swift binary requiring Xcode 16 or newer and macOS 14 or newer.", "body_md": "**big-arrow-on-the-screen** (`bigarrow`): one small macOS CLI and an agent skill. Click-through, never steals the focus, gone by itself. MIT.\n\nYour AI agent can refactor a monorepo, write a migration and explain monads, but when it needs you to click one button it prints *\"please click Allow in the dialog\"* into a terminal you are not looking at. `bigarrow` gives it a finger.\n\n```\nbigarrow point --element \"Allow\" --app \"System Settings\" --text \"Franz, click Allow\"\n```\n\nA big, friendly arrow with a sign appears on top of everything, points at the thing, and goes away again. It is click-through, it never steals your focus, it works on every display and every Space, and it needs **no permission at all** to draw. It is one small Swift binary. There is no daemon, no menu-bar icon, no account, no telemetry, and, we checked twice, no AI inside. It is an arrow.\n\nFair question. Arrows have existed since roughly the Paleolithic. Here is what changed: software agents now do real work on your Mac, and they keep hitting the same wall, **the part that only a human may do.**\n\n- **\"Click Allow.\"** macOS permission prompts, OAuth consent screens, \"Open with...?\" dialogs. The agent can find the button but must not, or cannot, press it for you. It can now point at it.\n- **\"Your turn.\"** 2FA codes, CAPTCHAs, passkeys, a payment confirmation, a signature, a legal checkbox. The things an agent should never click on its own behalf. It points, you decide, it continues.\n- **\"It's this window, not that one.\"** You have 14 Chrome windows. The agent knows which one it means:`--window \"Google Chrome:Pull request\" --raise` .\n- **\"I need you, and you're making coffee.\"**`--say` reads the sign aloud. Your Mac will literally call you back to your desk.\n- **Guided setups and onboarding.** Walk a human through a settings pane step by step:`start` , wait until they acted,`stop` , next step. Like a product tour, minus the product.\n- **Remote help.** \"No, the*other* gear icon.\" Point at it instead of describing it.\n- **Demos, screencasts, docs.** Highlight what matters while recording, or render the arrow straight into a PNG with`--png` for documentation.\n- **Debugging coordinates.** Not sure your Accessibility, screenshot or Peekaboo coordinates are right? Point at them and look.`--dry-run --json` tells you where it*would* point without drawing.\n\nStaged with a neutral demo dialog and recorded with the real `bigarrow` on a test Mac (`scripts/funny-scenes.sh`). The dialogs are fake. The feelings are real.\n\n| `--color green` | `--shape zigzag --color orange` | \n| `--color purple` | `--close-button` , because the human gets the last word | \n| three `start` s, one button, zero ambiguity | `--style box --corners sharp` , plus a lesson about macOS permissions | \n\nWhat it is not: a screen annotator for humans, a click bot, or a screenshot tool. It never clicks, types or captures anything. It only points. Deliberately.\n\n```\nbrew install franzenzenhofer/tap/bigarrow\nbigarrow install-skill          # teaches Claude Code (~/.claude/skills) and Codex (~/.agents/skills)\n```\n\nFrom source: `swift build -c release` (Xcode 16 or newer, macOS 14 or newer), binary at `.build/release/bigarrow`.\n\n```\nbigarrow point --element \"Allow\" --app \"System Settings\" --text \"Franz, click Allow\"   # by label\nbigarrow point --at 760,500 --text \"Franz, click HERE\"                                 # by coordinate\nbigarrow start --window \"Safari:Inbox\" --raise --text \"This window\" && bigarrow stop   # until stopped\n```\n\nThree ways an arrow ends, pick your level of commitment:\n\n| Time limit | `bigarrow point ... --duration 10` (default 8 s) | \n| Start and stop | `bigarrow start ...` returns at once;`bigarrow stop` (or`stop --all` ) removes it | \n| The human closes it | `bigarrow point ... --close-button` puts a clickable X on the sign | \n\nTargets: `--at X,Y`, `--rect X,Y,W,H`, `--mouse`, `--window App[:title]`, `--element Label --app App`, `--peekaboo ID --snapshot see.json` (from Peekaboo's `see --json`). Coordinates are global top-left logical points, the space Accessibility, CGWindowList and Peekaboo report. `--display N` makes `--at` and `--rect` relative to one display.\n\n`bigarrow front --app X` or `point --raise` brings the target's app to the front first, because pointing at a window hidden behind your terminal is a special kind of unhelpful. `bigarrow elements --app X` lists what `--element` can match. `bigarrow doctor` shows permissions, who owns them, and your displays.\n\nEvery command takes `--json`. Exit codes: 0 ok, 2 bad input, 3 target not found, 4 permission missing. Agents love exit codes. Humans tolerate them.\n\nIt is an arrow, so we spent an unreasonable amount of time on how it looks.\n\n- `--shape bend|straight|zigzag` (zigzag for when it is*really* urgent)\n- `--style arrow|ring|box` ; rings and boxes are border-only, so you still see what is under them\n- `--size S|M|L` ,`--corners round|sharp`\n- `--color red|orange|yellow|green|teal|blue|purple|pink|black|white|#RRGGBB` ; light colours automatically get a dark outline and text\n- `--follow` moves with a window or element,`--until-click` ends on a click on the target,`--say` speaks the sign\n- Several arrows at once keep their signs out of each other's way (the HN shot above is five independent `bigarrow start` calls)\n\nThe shaft grows out of the sign through a flared joint that never runs into a rounded corner. `scripts/gallery.py` renders every combination offscreen and zooms into every joint ([junctions](https://github.com/franzenzenhofer/big-arrow-on-the-screen/blob/main/docs/images/junctions.png)), because a seam at the joint was, apparently, unacceptable.\n\n**Does it need Screen Recording or Accessibility?**\nDrawing needs neither. `--element`, `elements`, `--until-click` and `front --window` use Accessibility, which macOS grants to the app that runs your shell (Terminal, iTerm2, Ghostty, VS Code, Claude), never to `bigarrow` itself. `bigarrow doctor` names that app, and exit code 4 tells the agent exactly what to ask you for. `--window App:title` reads window titles, which macOS 26 hides without Screen Recording; `--window App` alone needs nothing.\n\n**Will it steal my focus while I'm typing?**\nNo. That was the hardest bug in the project: `NSApplication.run()` quietly activates a process that has no terminal, so detached arrows grabbed the focus. `bigarrow` pumps events itself instead, and the tests check that the frontmost app never changes.\n\n**Can I click through it?**\nYes, everywhere except the optional X, which is its own tiny panel that also never takes the focus.\n\n**Multiple displays? Full-screen apps? Stage Manager? Spaces?**\nYes, yes, yes, yes. Displays left of or above the main one (negative coordinates) included. Unplug a display while an arrow is on it and the arrow politely leaves. See the [verification matrix](https://github.com/franzenzenhofer/big-arrow-on-the-screen/blob/main/docs/verification/multi-display.md).\n\n**How much CPU does a pulsing arrow cost?**\n1.4 % measured on a CI runner. Core Animation does the work in the render server.\n\n**Does `--element` work inside web pages?**\nIn Electron apps, yes. In Chrome, only when Chrome runs with `--force-renderer-accessibility` (or VoiceOver is on); Chrome ignores the usual request to expose page content, verified on Chrome in October 2026. Chrome's own toolbar always works. Otherwise point at the page's coordinates, which the skill explains.\n\n**Why not just use [some screen annotation app]?**\nThose are for humans drawing on screens. This is for programs pointing at things, from a shell, with exit codes. Twenty-six tools were checked before writing a line ([research](https://github.com/franzenzenhofer/big-arrow-on-the-screen/blob/main/docs/research)). None did this.\n\n**Is it AI?**\nNo. It is the least intelligent part of your AI stack, and proud of it.\n\n- 76 automated tests: geometry, placement, joint smoothness, a golden image, recorded window-server, Accessibility and Peekaboo 4.9.0 fixtures, and tests against the real window server (window level 1000, clicks pass through, focus never moves, detach and stop timing). CI runs them on macOS 15; they also passed on macOS 26 and macOS 27.\n- 17 behaviour checks on a clean runner ([visual.yml](https://github.com/franzenzenhofer/big-arrow-on-the-screen/blob/main/.github/workflows/visual.yml) ): real clicks on the X,`--until-click` ,`--follow` ,`--raise` ,`--say` , full-screen apps, Stage Manager, a Space switch, a second display, a 2x display, unplugging a display mid-arrow, CPU. The demo GIF above is recorded by the same workflow, on a desktop with nothing personal on it.\n- A fresh agent given only the skill and \"show Franz where the Reload button in Chrome is\" found it by label and built the right command ([transcript](https://github.com/franzenzenhofer/big-arrow-on-the-screen/blob/main/docs/skill-tests/2026-10-08-chrome-reload.md) ). It also found a bug, which is now a test.\n\nThe skill in `skill/big-arrow/` works for both Claude Code and Codex (one `SKILL.md`, Agent Skills format, plus `agents/openai.yaml` for Codex). It tells the agent when to point, how to pick a target, to write a full sentence on the sign, to add `--say` when you are probably not looking, and to `stop` once you have acted.\n\n`docs/plan/PLAN.md` (goal, architecture, risks), `docs/plan/TICKETS.md` (generated from `docs/plan/tickets.json`), `docs/decisions/`, `docs/research/` (verified facts with links), `docs/verification/`, `docs/skill-tests/`, `CHANGELOG.md`.\n\nPeekaboo's visualizer ([https://github.com/openclaw/Peekaboo](https://github.com/openclaw/Peekaboo)) and Nameplate ([https://github.com/steipete/Nameplate](https://github.com/steipete/Nameplate)) by Peter Steinberger showed the overlay window recipe and the agent-skill packaging. Neither draws a pointing arrow with a label, which is the gap this project fills. `bigarrow` reads Peekaboo's `see --json` as an optional target source.\n\nMIT. Point responsibly.", "url": "https://wpnews.pro/news/show-hn-let-your-ai-agents-paint-big-arrows-boxes-and-text-on-your-screen", "canonical_source": "https://github.com/franzenzenhofer/big-arrow-on-the-screen", "published_at": "2026-10-08 11:33:54+00:00", "updated_at": "2026-10-08 11:47:07.387392+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "ai-products"], "entities": ["bigarrow", "Franz Zenhofer", "Claude Code", "Codex", "Peekaboo", "macOS", "Swift", "Homebrew"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-let-your-ai-agents-paint-big-arrows-boxes-and-text-on-your-screen", "markdown": "https://wpnews.pro/news/show-hn-let-your-ai-agents-paint-big-arrows-boxes-and-text-on-your-screen.md", "text": "https://wpnews.pro/news/show-hn-let-your-ai-agents-paint-big-arrows-boxes-and-text-on-your-screen.txt", "jsonld": "https://wpnews.pro/news/show-hn-let-your-ai-agents-paint-big-arrows-boxes-and-text-on-your-screen.jsonld"}}