AI Thinks this Post is Terrible (but I’m writing it anyway)

Expected Parrot co-founder John used the company's conversational research agent to rank 25 blog post topics, fitting a Bradley-Terry model to pairwise comparisons after noting that AI raters tend to over-praise everything. The AI-ranked list placed a planned post on morphological analysis dead last, though the author says he will still write it. The agent used Firecrawl for background research and generated AI personas to evaluate the topics, with the full transcript shared at expectedparrot.com.

I took a break on blogging this summer to go heads-down on a bunch of new Expected Parrot features and research ideas. We have a lot to talk about—so much that I was feeling a bit overwhelmed in what, exactly, to focus on. I needed some way to: 1. Map all the new features to good blog post topics 2. Rate those topics by likely reader interest Naturally, I thought “maybe AI could help?” I used the following prompt with our conversational research agent www.expectedparrot.com http://www.expectedparrot.com : The full report is here https://www.expectedparrot.com/research-agent/share/rzMe4hfqb8d75tbFwxCg7Q which has my transcript. As it’s designed to do, it asked me a bunch of clarifying questions: The research agent also used Firecrawl https://www.firecrawl.dev/ to do a bit more research about the company and what we’re doing. This research was then used in the creation of the AI Agent personas that evaluated the topics: As I was working with it, I remembered that AI’s asked to rate things are way too nice and will claim to read everything, click everything, buy everything, etc. What I needed to do was do some forces comparisons so I asked it to try to ranking questions and then fit a Bradley-Terry https://en.wikipedia.org/wiki/Bradley%E2%80%93Terry model model. This was the end result: 25 blog topics rank-ordered by score. See the transcript for details and note that you are reading a last-from-the-bottom example: Also note that dead last is a blog post idea I have on using AI for morphological analysis <https://en.wikipedia.org/wiki/Morphological analysis problem-solving which I thought would be a banger. I actually will still write that one—AI isn’t the boss of me—but I’m definitely also more inclined to write the top performers. It’s not terribly important given the stakes of this post/decision, but I want to flag that you can see precisely how the end report was generating, starting from the code that generated the options and the code that generated the personas being used: Now, you might say “John - how do you know this is what people want?” The truth, is, I don’t and I could imagine the AI answers differ from what some high-quality market research would reveal. But the bigger truth is that I don’t have the money, time or inclination to do serious market research on this. The status quo was just to go with my gut. Instead, I can do this which is I think quite a bit better and more interesting for less money. Anyway, can’t wait to tell you more about Morphological Analysis PS - the agents really didn’t like the Morphological analysis idea