Four weeks ago, I had no idea what an AI agent was. Now I'm building one that researches market trends before writing about them, synthesizes information from three independent sources, and produces work that scores 96/100 on my eval system.
The change didn't come from a new model or a fancy framework. It came from stopping my agent from writing blind.
When I first built ShipStack's article factory, it was simple. I'd prompt Claude: "Write an article about AI agents and multi-agent orchestration." Claude would write. It was fine. Coherent. On-brand.
But it was hollow.
I realized what was happening: my agent was writing from pattern matching. It knew what an article about AI agents should look like because it had seen thousands. But it didn't know what was actually happening in the market right now. It didn't know that Cursor just hit $9.9B valuation with Agent Mode as the headline feature. It didn't know that enterprise leaders are abandoning 60% of AI projects because their data isn't ready. It didn't know that accuracy compounds exponentially—if each agent action hits 85%, a 10-step workflow only succeeds 20% of the time.
It was writing confidently about things it didn't actually understand.
That's the gap between a chatbot and a real agent. A chatbot answers questions. An agent investigates before it acts.
I started noticing something in my own process. When I write about something I actually understand, the work is sharper. More specific. I reference concrete numbers, real tools, actual timelines. When I'm writing from half-memory, it's generic. Filler. Safe.
So I asked myself: what if my writing agent worked the same way?
Instead of "Write an article about X," the prompt became:
The research agent runs independently. Error handling per source. If one fails, the others still work. Claude Haiku synthesizes the raw results into a clean brief—background noise removed, signal amplified. The brief gets injected into the writer's context before the first word is written.
First article written with research scored 96/100 on eval.
Before research: "AI agents are transforming business automation."
After research: "Cursor's Agent Mode hit 8 parallel agents and $9.9B valuation. NVIDIA's GTC 2026 saw agentic frameworks draw the largest attendance, signaling enterprise deployment momentum."
One is a claim. The other is evidence.
The brief gives the writer ammunition. Real stats. Real context. Real angles. The writing doesn't have to be cautious anymore because it's grounded in something verifiable.
Here's what shocked me: the research agent found problems that nobody is talking about, even though they're critical.
Accuracy compounding is a perfect example. Everyone talks about 85% per-action accuracy as a win. Almost nobody mentions that this cascades to ~20% success in 10-step workflows. The brief highlighted this as an "angle nobody's exploring." The article could then address it directly.
A writer without research writes from memory gaps. A writer with research writes from knowledge gaps—and those are infinitely more valuable.
When I read the 96/100 article, I didn't just feel it was better. I could point to why. The piece mentioned three validated statistics. It cited specific products and company valuations. It acknowledged real problems with real consequences. The eval system rated it higher because the work was verifiable.
That's the real shift. The agent isn't smarter. But it's more honest.
I'm not going to pretend this is rocket science. It's not. But it's also not trivial, and I had to think through some real problems.
async def research_topic(topic: str) -> dict:
"""
Research a topic across three independent sources.
Returns structured brief with background, current discussion, gaps, and stats.
"""
sources = [
{"name": "Brave Search", "func": search_brave},
{"name": "DuckDuckGo", "func": search_duckduckgo},
{"name": "Wikipedia", "func": search_wikipedia}
]
results = {}
for source in sources:
try:
results[source["name"]] = await source["func"](topic)
except Exception as e:
results[source["name"]] = {"error": str(e), "data": None}
brief = await synthesize_with_claude(
results,
sections=[
"background",
"what_is_being_discussed_now",
"gaps_and_underexplored_angles",
"key_stats_and_data_points"
]
)
return brief
The critical decisions:
Independent error handling: If Brave Search fails, DuckDuckGo still runs. If Wikipedia times out, the brief still synthesizes from two sources. I learned this the hard way—the first version failed if any source failed. Production taught me otherwise.
Parallel execution: All three search queries run at the same time using asyncio.gather()
. Sequential would take 3x longer. In production, speed matters because every second of latency is a second the user waits.
Structured synthesis: The brief isn't just raw search results dumped together. Claude is instructed to organize findings into four specific sections. Background is history and context. "What's being discussed now" is current momentum and trends. "Gaps" is the angle—what everyone's talking about versus what's actually critical. "Key stats" is the ammunition. This structure forces clarity.
I need to be honest about the downside: this costs more tokens.
Each research cycle uses Haiku (cheap) for searches and synthesis, then Sonnet (more expensive) for actual writing. A single article now pulls maybe 15,000-20,000 tokens where it used to pull 8,000-10,000. It's not dramatic, but it adds up across the article factory.
What I found: the 20% increase in token cost produces articles that score 15-20% higher on eval. The math works. But only if you're actually shipping and measuring. If you're just trying to sound smart, research doesn't matter.
Measure before and after: I didn't quantify article quality until after I built this. If I were starting over, I'd eval the old system, then the new one, so I'd have concrete proof. (I got lucky—the 96/100 score validated it retroactively.)
Automate source selection: Right now I hardcoded three sources. But different topics benefit from different research. A technical deep-dive needs StackOverflow and GitHub. A market analysis needs Crunchbase and SEC filings. A framework update needs the official docs. Future version should route the query to the right sources automatically.
Build a feedback loop from eval back to research: If an article scores 72/100 because it's missing recent data, the research agent should know that. Right now research and writing are separate. Next step is making them iterative.
This is bigger than article writing.
I'm seeing this pattern across all six of ShipStack's pipelines. When agents have access to real-time context—whether that's current market data, your inbox status, your GitHub repos, or your company priorities—they make better decisions. When they're operating on stale mental models, they fail quietly.
The Morning Brief pipeline needs current date and recent priorities (stored in memory) to actually be useful. The Inbox Zero processor needs to know which senders matter historically. The Repo Triage system needs to know what's actively shipping versus abandoned. Right now memory is an island—only the article factory reads from it. Next is connecting memory to all five other pipelines.
Research before action. Memory-informed decisions. Real-time context.
That's the move from agent-as-tool to agent-as-actually-useful.
I want to push this further. Instead of research happening once per article, I want continuous background research running on topics I care about—AI agents, agent engineering, multi-agent orchestration, memory architecture. The agent saves interesting findings to memory automatically. When I sit down to write, the brief isn't built from scratch—it's pulled from three weeks of background research plus fresh daily snapshots.
That's still hypothetical. But I'm building toward it.
Four weeks ago I thought building an AI agent meant connecting APIs and writing prompts. Now I understand it's about giving agents the ability to think before they act.
A chatbot without research writes confidently about things it doesn't know. An agent with research writes carefully about things it does.
The difference is measurable. It's in the eval scores. It's in the specificity of the output. It's in the actual value delivered.
I'm one month into this. I'm still figuring out what's possible. But I know for certain: the best agent isn't the one that can generate text fastest. It's the one that can verify reality before speaking.
That's worth building toward.