I've been using Claude Code as a pair programmer, debugger, and feedback collector while I playtest my game, OnTrack. The setup has worked so well that I ended up building a small custom skill to make it scale and in this post I'll take you through how it works.
(That last DevBlog was in 2024 (!), but I have continued to work on the game — with AI my pace and progress has quickened, even if overall I've less time since
[becoming a dad].)
Pausing development to playtest #
After a period of development, I like to do a playtest. I resist fixing things, and instead I play the game for an hour, noting down bugs, thoughts and suggestions as I go.
I used to do this in a Trello board, but I recently decided to try using Claude as my bug collecting coworker and collate feedback in a markdown file. This worked really well! One of the benefits of using Claude is that it was able to detect similar feedback, or duplicates, or re-write old feedback to incorporate new points that were related. It also grouped them as it went under different categories such as UI, mechanics, or "quick wins".
The problem of large Markdown files #
As I then followed these playtesting sessions with fixes, I asked Claude to mark feedback as complete by moving it into a separate document. In one hour of playtesting I would easily pick up on 50+ issues I wanted to fix (the game is far from done!) and one day I realised that the Markdown file with incomplete items was 3,000 lines long, and the one with completed items was 4,000 lines long!
Claude was managing this fine - but I could see that as time grew, this list would become hard to manage, eat up more of my tokens, and become inefficient. Asking Claude to list the outstanding issues, or all the issues under a particular category, required it to read the entire Markdown file, even if I was only asking about a subset.
The solution: SQLite #
I needed a way to store this data in a more structured form where Claude could query it more efficiently — for example, getting all the titles of bugs without having to read the entire description, or listing all items under a particular category, or marking items as done and never having to parse them again unless explicitly asked.
The table is straightforward — each row has a title
, detail
, priority
, status
, category
, project
, and a done
flag. That last one is the key bit: completed items are never deleted, just hidden from the default list
output, so there's a full history if you ever want it.
So, I made a feedback skill that does just that and teaches Claude how to log, process and track ideas and feedback from the playtesting session.
At its core is a Deno CLI that talks to an SQLite database. This database lets Claude query feedback items and add new ones.
Why Deno? Honestly, because I wanted to try building something in it rather than always reaching for Node. But Node (or any other runtime/language) would be fine here. Plus, Claude wrote it anyway :)
Rather than one database per project, I keep all the data in a single file (that is in my .gitignore
). When the AI looks up or adds items, it must tag them with a --project
flag.
Here is the AI listing projects (routemaster
is the old, internal name for OnTrack):
run list --project routemaster
And then looking at detail for a particular item. Here is another reason that using an AI for this feedback shines - as you give it feedback it can also analyse your code and system to make some potential implementation notes as you go:
run show 32[32] #024 — No visual confirmation when player performs an action (toast notification system)Project: routemasterPriority: highStatus: openCreated: 2026-04-22 07:08:06---Observation: When the player builds a station, finalises a route, places a bridge, or purchasesan upgrade, there is no transient visual confirmation. The action succeeds silently. Routecreation in particular is confusing — after going through the full finalise modal flow there isno confirmation that the route was actually created.Analysis: A notification system already exists. Logger (src/log.ts) is a singleton EventTargetthat stores LogMessage objects and dispatches NewLogMessage events. LogOutput(src/views/log-output.ts) subscribes to it and renders a dismissing list — whenremove-old-messages attribute is set, messages auto-expire after 10 seconds. Currently used forcontract completions and town growth.
And adding an item (you can also edit an existing item by ID which looks nearly identical):
run add --project routemaster \ --title "Train detail window: journey progress bar disappears when train is delayed" \ --detail "When a train is running late (shown in red), the journey progress bar in the train detail window stops rendering entirely. The delayindicator and the progress bar appear to share state and one clobbers the other. Reproducible by letting any train fall behind schedule by morethan 2 stops." \ --priority high \ --category bug
And finally, marking an item as complete:
run done 75
The feedback SKILL.md file #
Because the CLI is intuitive and well documented, the actual skill is fairly straightforward — it teaches Claude the CLI commands and adds some guidance on how to use them well. The most impactful part has been telling it to check for duplicates or related items before storing new feedback. I found I'd often describe the same underlying issue in two different ways without realising, and Claude catches that.
A couple of the other hints worth calling out:
Always— titles and IDs use far fewer tokens than full item details, so the skill tells Claude to get the lay of the land before diving into specifics. I didn't want it to have to view all the details of each task until I need it. If I made it list the detail by default, I wouldn't be saving any tokens over the Markdown file approach.list
beforeshow
Infer the project from context— rather than asking me every time, Claude figures out which project we're working in from the conversation and only asks when it's genuinely ambiguous.
CLIs + skills #
What surprised me most is how much the structure helps beyond the token savings — being able to ask "what are all the high priority bugs in the UI category?" and get a fast, accurate answer genuinely changes how you work through a backlog. Feel free to grab the skill from GitHub and let me know if you build something similar.