{"slug": "spoke-at-yurutech-beer-bash-lt-2026-10-letting-ai-drive-a-mac-with-computer-use", "title": "Spoke at Yurutech Beer Bash LT 2026.10 — Letting AI Drive a Mac with Computer Use", "summary": "Tadashi Shigeoka presented a five-minute lightning talk, \"Letting AI Drive a Mac with Computer Use,\" at Yurutech Beer Bash LT 2026.10 on Friday, October 2, 2026, in Kumamoto City, Japan, demonstrating Codex using Computer Use to build a cheers slide in Keynote. Shigeoka described Computer Use as a repeating see-act-verify loop in which the AI captures a screenshot, clicks, types or scrolls, then re-observes the screen, and he argued the verify step matters most because AI judgment is not deterministic — the same screen can be flagged as a bug on one run and pass on another. He positioned Computer Use for exploratory QA on unfamiliar screens, with suspected bugs returned with reproduction steps and screenshots for human verification, while existing end-to-end tests keep verifying fixed conditions.", "body_md": "# Spoke at Yurutech Beer Bash LT 2026.10 — Letting AI Drive a Mac with Computer Use\n\n[Tadashi Shigeoka](https://codenote.net/en/author/tadashi-shigeoka/)· Fri, October 2, 2026\n\nI gave a lightning talk at [Yurutech Beer Bash LT 2026.10](https://techplay.jp/event/999690). My talk, “Letting AI Drive a Mac with Computer Use,” packed into five minutes the core loop behind an AI that watches the screen and operates apps, together with a live demo where [Codex](https://learn.chatgpt.com/docs) builds a cheers slide in Keynote.\n\nThe content reworks three earlier posts on this blog into a five-minute beer-bash LT: [Notes on choosing a Computer Use agent for macOS and Windows](https://codenote.net/en/posts/computer-use-agents-macos-windows-selection-notes-2026/), [Notes on handing QA testing to an AI agent](https://codenote.net/en/posts/ai-tester-agent-gpt-astra-computer-use/), and [What Does CUA Stand For? — Sorting Out Computer-Using Agent, Computer Use, and Computer-Use Agents](https://codenote.net/en/posts/computer-using-agent-cua-meaning/).\n\n## Event Overview\n\n- Name: Yurutech Beer Bash LT 2026.10\n- Date: Friday, October 2, 2026, 19:00–21:00 JST\n- Location: Kumamoto City, Japan\n- Format: Beer-bash-style lightning talks\n\n## My Talk: “Letting AI Drive a Mac with Computer Use”\n\nThe talk introduced [Computer Use](https://learn.chatgpt.com/docs/computer-use), which I use regularly in Codex, with a beer-bash-appropriate motif: asking the AI to prepare the toast.\n\n### What Is Computer Use\n\nComputer Use is a mechanism where an AI watches the screen and operates an app. Clicks, text input, and scrolling, the things we usually do with a mouse and keyboard, get handed to the AI. When you describe the goal in natural language, such as “build a cheers slide in Keynote,” the AI drives the actual app to get there.\n\n### The Basic Loop: See, Act, Verify\n\nUnderneath, Computer Use is a repetition of three steps.\n\n| Step | What happens | \n|---|---|\n| See | Pull a screenshot or UI-element information to understand the current screen state | \n| Act | Perform a click, text input, or scroll | \n| Verify | Observe the screen again and decide the next move | \n\nThe observation surface varies by implementation; some setups add UI metadata on top of screenshots. The step that matters most is verify: checking the screen again after each action. I come back to it with a concrete example later in “After You Act, Check the Result.”\n\n``` php\nflowchart LR\n  U[\"User instruction\"] --> P[\"See<br/>Capture screenshot\"]\n  P --> R[\"Act<br/>Click, type, scroll\"]\n  R --> V[\"Verify<br/>Observe the screen again\"]\n  V -->|\"More to do\"| P\n  V -->|\"Done or need confirmation\"| E[\"Hand back to user\"]\n```\n\n### Pick Based on What It Can Touch\n\nNot every Computer Use touches the same surface. Some stay inside the browser; others go all the way into OS-level apps.\n\n| Scope | Can operate | \n|---|---|\n| In-browser | Web pages, web apps | \n| OS-level apps | Native apps like Keynote and Finder | \n\nThe pick depends on what you actually want to do: open a web page, or drive a desktop app like Keynote. For this demo, I used a Codex setup that can operate macOS native apps. For a per-product comparison, see my earlier post [Notes on choosing a Computer Use agent for macOS and Windows](https://codenote.net/en/posts/computer-use-agents-macos-windows-selection-notes-2026/).\n\n### A Dev Use Case: Exploratory QA\n\nThe use case I find most interesting on the development side is exploratory QA. You hand the agent an open-ended goal like “play with this new screen and tell me where you got stuck,” and let it explore.\n\n| Role | What it is good at | \n|---|---|\n| Computer Use | Take a goal and explore an unfamiliar screen | \n| Existing E2E | Repeatedly verify a fixed set of conditions | \n\nAI judgment is not deterministic: the same screen can be flagged as a bug on one run and pass on another. So suspected bugs come back with reproduction steps and screens attached, and a human verifies. Existing E2E keeps its job of verifying fixed conditions on repeat; Computer Use takes the role of touching unfamiliar screens. The design discussion is written up in [Notes on handing QA testing to an AI agent](https://codenote.net/en/posts/ai-tester-agent-gpt-astra-computer-use/).\n\n### After You Act, Check the Result\n\nThe failure that is easiest to miss with Computer Use is “the input happened, but the screen does not add up.” Even when text makes it into Keynote, the sentence can be long enough to overflow the box. Do not end at “input completed”; include “look back at the screen” as part of the completion condition.\n\nFor this demo, I made the completion condition explicit with three checks: exactly one slide, the text “みなさん、乾杯！🍻” entered as is, and no text overflowing its box. Say up front what “done” looks like.\n\n### Demo: Asking Codex to Prepare the Toast\n\nThe main-content closer was a demo where Codex builds a single Keynote slide reading “みなさん、乾杯！🍻” (“Cheers, everyone! 🍻” in Japanese). The two things I asked people to watch for were that Japanese text gets typed correctly, and that the final slide gets re-inspected after being produced.\n\nDuring the demo, I let the see / act / see-again loop play out live. I had a backup screenshot of the finished slide, captured earlier on the same Mac, in case we ran out of time, but the live run finished within the LT window and the backup stayed in the deck unused.\n\n### Wrap-Up\n\n- Computer Use is a loop: see, act, verify, then repeat\n- Pick in-browser or OS-level based on what you actually need to touch\n- After acting, confirm the result from the screen itself\n- On the development side, start with exploratory QA layered on top of existing E2E\n\nThe closing line I ended on: “Every operation we do on a PC, now handed off to AI.” Start with the small, everyday operations you keep repeating.\n\n## I Joined the After-Party Too\n\nAfter the main program, I joined the after-party organized by willing attendees. When you normally work fully remote, meeting fellow engineers in person (not through a screen) is rare, and we were able to dig into operational topics: how each company is folding Computer Use and AI agents into their work, where they draw the line and hand control back to humans. The topics I did not have time to cover on stage got their own slow discussion over drinks.\n\nThank you to the organizers and to everyone who attended.\n\nThat’s all from speaking at Yurutech Beer Bash LT 2026.10 and running the Codex-prepares-the-toast demo, from the Gemba.", "url": "https://wpnews.pro/news/spoke-at-yurutech-beer-bash-lt-2026-10-letting-ai-drive-a-mac-with-computer-use", "canonical_source": "https://codenote.net/en/posts/yurutech-beerbash-20261002/", "published_at": "2026-10-02 14:58:39+00:00", "updated_at": "2026-10-20 04:46:07.803558+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "artificial-intelligence"], "entities": ["Tadashi Shigeoka", "Yurutech Beer Bash LT 2026.10", "Codex", "Computer Use", "Keynote", "Kumamoto City", "macOS"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/spoke-at-yurutech-beer-bash-lt-2026-10-letting-ai-drive-a-mac-with-computer-use", "markdown": "https://wpnews.pro/news/spoke-at-yurutech-beer-bash-lt-2026-10-letting-ai-drive-a-mac-with-computer-use.md", "text": "https://wpnews.pro/news/spoke-at-yurutech-beer-bash-lt-2026-10-letting-ai-drive-a-mac-with-computer-use.txt", "jsonld": "https://wpnews.pro/news/spoke-at-yurutech-beer-bash-lt-2026-10-letting-ai-drive-a-mac-with-computer-use.jsonld"}}