You’re staring at the PR from your coding agent, mouse poised over the merge button. CI is green. The changes seem reasonable.
BUT.
Will this database migration finish quickly on a larger database? Is this added field in the API going to break existing callers? Will those CSS edits cascade to way more than you expected? Will the frontend render consistently across browsers?
And are the tests actually reproducing the issue your users reported, or are they testing a fiction composed of mocks and stubs?
What matters is whether it works for your users in production.
#
In ye olden days, your QA team (those 15 people over there, supporting 5 developers) would spend a month building a test environment and writing a binder full of manual test cases. You’d toss a build over the wall and hear back the following week.
I struggled to get that kind of help from agents. They’d either do everything behind the scenes without showing their work, or hand me the binder and make me do everything.
#
I need to play-test these diffs
At least in my experience with Claude Opus 5.5 high and Codex GPT-6.1 Sol xhigh, you say “let’s play-test this”, and they get it.
For a reported bug, choose whether to reproduce it, verify the fix, or do both. For other changes, check the intended behavior.
- They set up the starting state: files on disk, a half-imported library, or whatever the scenario needs.
- They ask you to confirm that the setup matches what you want to test.
- They say “click this” or “run this script” and explain what you should see. Read the script first to check that it does what you expect.
- You report what happened, and together you check whether it matches the expected result.
Pro tip: for mobile testing, ask your agent for a QR code linking to a URL your phone can reach. The skill below includes this step when you’re testing in a mobile browser.
#
But in case they don’t get it, I added a play-test skill to PhotoStructure’s coding plugin that makes this explicit.
In Claude Code, run:
/plugin marketplace add photostructure/coding-skills
/plugin install coding@photostructure
For Codex, run these commands in your terminal:
codex plugin marketplace add photostructure/coding-skills
codex plugin add coding@photostructure
Use /coding:play-test in Claude Code or $coding:play-test in Codex when you think everything’s pretty baked. You can play-test before a commit, alongside a gavel review, or while reviewing a pull request.
#
From the hundreds of times I’ve used this with agents, I’ve seen the full spectrum of outcomes:
- Holy crap, the agent nailed it .
- Ah, most of the way there , but this UX needs polish.
- Hmm, we need to address these two other scenarios I just discovered.
- Oh dear, this is all quite wrong .
Most of the time my play-testing reveals issues. Every bug I can fix before it bothers a user is a huge win.