I Kept Feeling Like I Was Wasting My Time — While Designing a Cross-Model Controlled Experiment

A developer analyzed 1,427 of their own AI prompts from six weeks and found a sharp gap between their actual skill growth and their self-perception. Despite designing a cross-model controlled experiment and issuing red-team orders to AI, they repeatedly expressed self-doubt and fear of wasting time. The analysis revealed a fivefold increase in adversarial/verification vocabulary and a jump in prompt length, indicating significant technical growth that the developer failed to recognize.

I pulled 1,427 of my own prompts from six weeks with AI. I meant to see "how did my way of asking change." The sharpest thing wasn't the capability curve — it was that the curve and how I saw myself were a full tier apart. Six months ago I wrote a piece called "Three Months, 1,604 Prompts: What Did AI Trade With Me?" That time I scanned "what I handle most." This time I wanted to look at something uglier: how do I ask — and did it change over six months? I pulled every prompt I typed myself between June 15 and July 23 — 1,427 of them stripping tool outputs, system messages, the fat-fingered interrupts — cut them into six time windows, and measured them window by window. The numbers were clear. What actually stopped me was the person standing next to the numbers. I tagged each prompt with a few categories of vocabulary, sorted by time: | Period | Adversarial / verify / root-cause | Meta / governance / method | Delegation / automation / batch | |---|---|---|---| | Mid-to-late June | 4% | 13% | 14% | | Late June | 9% | 28% | 34% | | Early July | 21% | 36% | 34% | In under six weeks, "make the AI push back, verify this, find the root cause" grew fivefold; "talk about method, governance, systematizing" nearly tripled. My average prompt length also jumped from just over 200 characters to around 1,000 — I'd started writing the kind of long, context-first "strategy prompt." If you only look at that table, it's an inspiring story: someone with no engineering background, in six months, going from "fix this bug for me" to "work backwards from my git scars to the pain most worth preventing." But I'm not here to write an inspirational post. Early morning, July 18, I typed this: "Having the ability doesn't mean I've actually productized it… I don't have an SOP or a cold-start flow that can cold-start in one day and ship an MVP in three… nothing is pushing me forward. Facing it head-on relies entirely on my anxiety, so I keep opening new sessions and asking, over and over." A little earlier, July 17: "It's hard not to feel like no one would pay for my service, that other people's is better." Earlier still, June 25: "Now that AI is this powerful and everyone can do things easily on their own — what am I even doing? Am I just wasting my time?" These aren't cherry-picked extremes. Sentences like this show up again and again across those 1,427 prompts — late at night, fishing for reassurance, feeling like I go deep on single points but can't connect them into a loop, feeling like others do effortlessly what I have to grind for. I'd often, in the same prompt, pour out a stack of self-doubt and then ask a genuinely hard technical question. I always thought I knew what I was doing. Laid open, it turned out my assessment of myself was frozen six months in the past. This is where the gap is sharpest. Right around the days of "am I just wasting my time," my prompt log has these: I designed a cross-model controlled experiment with my own hands. On July 10, to verify whether a methodology "skill" actually made the model smarter, I asked: "Can you open a sandbox or subagent right here and have haiku run it? And sonnet? If they'd be contaminated by this project's claude.md, tell me and I'll paste it manually." I didn't even notice — someone with no statistics training, no engineering background, instinctively knew to isolate the variable worried the project config would contaminate the experiment , to run a control group with skill vs. without , to cross-check across different models . That's experimental design. And I felt like I was wasting my time. I started giving the AI red-team orders. On July 21, I told it to attack a defense I'd just built myself: "Using a non-homologous model, ask 'what does this lens itself miss? Under what conditions would it give false reassurance?' — I want it to attack, not endorse." I settled on a "trust the scar, not my account" extraction method. In early July, I wanted to capture a frontier model's judgment into a reusable skill. I didn't ask it "how do you think" — I knew that would get a beautiful but empty answer. I told it to work from my git history: "Git is the crystallization of scars: a repeated fix = a pain that was never prevented, that keeps recurring." I even started using the AI to recalibrate my own perception. By July 23, I wasn't asking "how" anymore, I was asking: "What's the real value I provide? ? Please recalibrate me." Giving red-team orders, designing controlled experiments, telling the difference between "self-report" and "behavioral evidence," turning around to have the AI correct my own positioning — these are not a beginner's questions. This is someone who knows what he wants and knows how to force the AI to give up the real answer. That person and the one fishing for reassurance at 2am were the same me, the same week. I set out to write a nice growth curve. By the time I got here, I'd changed my mind. What's actually worth writing down is the seam between self-assessment and actual judgment. Because I'm almost certain that if you're also a solo operator building things with AI, you have this seam too. At night you feel like you're faking it, chasing someone else's taillights; by day you're doing things you don't even realize are hard. Three things I learned: One: your gut feeling about yourself is the least reliable instrument. My read on "what I'm doing" lagged my actual ability by six months. If I'd gone and looked at the record earlier instead of going by feel, I'd have saved myself a lot of anxious nights. So now I periodically pull my own conversations and look — not out of vanity, but to calibrate. Feelings lie; the record doesn't. Two: "I go deep on single points but can't connect them," I said this to myself so many times it became an excuse instead of a diagnosis. The record shows that by July I was already running my first real client case, doing end-to-end dry runs, wiring scattered things into a flywheel. It's not that I can't connect — it's that I kept using "I can't connect" to block myself from seeing how much I already had. Three: professional capability isn't shown by bragging, it's shown by laying it open. In this whole piece I never said "I'm good." I just put my own prompts side by side — the ones fishing for reassurance, next to the ones giving red-team orders. The gap speaks for itself. That's more convincing than any "I'm a senior AI collaborator," because it's real, and it doesn't even hide my own awkwardness. If you use Claude Code or a similar tool, your conversation history is sitting in jsonl files on your machine Claude Code keeps them under ~/.claude/projects/ . Spend half an hour writing a script to pull your own prompts from the past few months, and look at two things month over month: Then — and this is the most important step — put those numbers next to the assessment of yourself in your head. If they match, congratulations, you know yourself well. If they don't, if they're a full tier apart like mine, then you owe yourself an apology. You've come further than you think. The data for this comes from a script I ran over my own 171 sessions and 1,427 prompts from June–July. The method is the same as that piece six months ago, "Three Months, 1,604 Prompts" — except this time I didn't stop at the numbers; I also looked at the self-doubting me standing right next to them. 本文原載於我的部落格： I Kept Feeling Like I Was Wasting My Time — While Designing a Cross-Model Controlled Experiment