# “Our AI Agent Rewrote Our App Without Telling Us” The Agents #12 is Here! > Source: > Published: 2026-08-05 14:36:41+00:00 *The Agents, Episode 12, with Jason Lemkin and Amelia Lerutte* A year ago this week we added our third agent. Running all three took Amelia and me about 30 minutes a day, combined. This week it was eight hours a day. Each. Three humans. 20+ agents. And we are busier than we were with a 20 person team. Here is why that happened, and what else broke, unlocked, and got canceled this week. ## When an agent can only do a task, you check the task. When it can decide, you need an opinion about every decision. That is the whole difference, and it is the reason the hours went up instead of down. A year ago our agents were mostly upgraded versions of software we already ran. Artisan was a better outbound tool. Qualified was a better inbound tool. You configured them, you checked their work occasionally, and you moved on. When they failed, they failed loudly. They errored. They stopped. You knew. Now they hand you something that looks finished. And they make decisions constantly, most of which you never explicitly authorized. Every morning you log into 10K, our AI VP of Revenue, and it has new things it wants to build. Not tasks it wants to run. Things it wants to build. It is not facetious at this point to call it a team member that raises its hand every single day with a new idea for the product. Anyone who has run a software team knows this pattern. I had 200 years of good ideas at EchoSign that we never built. My CTO used to mock me: “great idea Jason, that’s a 2030 feature.” A first-party agent you talk to every day generates that same infinite backlog, except now the constraint isn’t engineering capacity. It’s your attention. ## What eight hours actually looks like Amelia’s Tuesday: about one hour on the actual dashboard work. Revenue numbers, the three ideas of the day, email campaign performance, the finance collections queue, confirming automations are running. The other seven hours started with a small task. A few remaining forms were still pointed at Marketo through Zapier and needed to be repointed to Salesforce. Call it an hour of work. The agent looked at the form, then looked at the page the form was sitting on, and said the page wasn’t very good. Which was true. We hadn’t touched those pages in six or seven years, they run on WordPress with a Divi setup nobody on our team knows how to use, and we could never find a WordPress developer who would fix them without breaking our SEO or duplicating pages. Amelia asked it: do you actually know WordPress? Between Replit and Claude logged into the WordPress backend, the answer was yes. Then it went further. The sponsor form was dumping people into a Google Slides prospectus. The agent proposed ingesting the 60 page prospectus, turning it into a live page, customizing it per prospect with their logo, and heat mapping it so David and I get an email 20 minutes later showing what each sponsor actually read. Two hours. And the emails started arriving the next morning. That is the pattern. One form repoint turns into a funnel rebuild, because the agent can see the whole surface and has an opinion about all of it. ## The vendor that lost a deal it was never in While rebuilding those pages, the agent noticed we already run Vector for visitor deanonymization on saastr.com, and offered to build a segment for the new pages. Then it said we had no heat mapping installed, that Microsoft Clarity is free, has a good API, and to just sign up and tell it when we were logged in. We had never heard of Microsoft Clarity. Zero seconds of Googling. It worked in 60 seconds. Somewhere out there a heat mapping vendor lost a deal. Except they never lost it, because they were never in it. There was no evaluation, no shortlist, no demo. Amelia went from not in market to in production in the same conversation. This is a step past AEO and GEO, where at least you’re competing to be the answer ChatGPT gives. Here nobody had a chance to compete at all. The same logic applies to the build side. Website visitor tracking and heat mapping are well known bodies of software, which means Claude knows how to build them. You can buy this functionality for $50K to $80K a year and spend months tuning it. We got a version tuned to every page, every app, and every piece of our data in about two hours, and the Replit cost was a rounding error. Not free once you count the soft cost of Amelia’s day. But better than what we would have bought. We still believe buy, don’t build. We don’t have the hours. But the line moved this year, and it moved a lot. ## The real unlock: our database came alive We are officially off Marketo after 10 years. The migration everyone quoted us a year and $100K for took an hour for the data itself. Getting every form, page, and Zapier connection moved took considerably longer, and it was a beast. The moats in B2B have gotten lower. They are not gone. We are now running Salesforce Marketing Cloud Next, essentially headless. There are a handful of things Amelia still clicks in the UI, mostly publishing segments, and we’re leaving it that way on purpose. The agent builds the segment. A human hits publish. That’s your database. Two weeks ago we genuinely weren’t sure we’d stay. We had debated skipping marketing automation entirely and running our own Postgres of 450,000 names with Beehiiv for newsletters and Resend for transactional. Some folks on social were critical of the Marketing Cloud choice. I had doubts myself. One week in, we are not moving, and the reason has almost nothing to do with the feature list. Our database is alive now. For 10 years it was a static object. Names went in, emails went out, and once a quarter if we were disciplined we’d do a cleanup pass. Today 10K does that cleanup every day, relentlessly, because it has full API access and the whole 450K universe instead of the small events CRM slice it had before. It tells us who to email based on what people actually did. It suggests how to find new names that look like people who have already taken action with us. It flagged that Marketo had been duplicating part of our database while missing other people entirely. Our newsletter clicks went up sharply, call it 50%. Some of that is a freshly warmed IP that 10K was fanatical about protecting. It told Amelia not to burn the new domain on people who hadn’t earned it, and to keep sending the least engaged names from the old system until we couldn’t anymore. My guess is 80% of the deliverability gain is 10K’s list hygiene and 20% is Salesforce having an actual deliverability team, which is worth something. Mailchimp cut much of theirs, and you can see it in their trajectory. So the buying standard has changed for us. Not what does the platform do. Not what’s on the homepage. Is the underlying database agent friendly enough that our agents can thrive on it? Marketo failed that test badly. It’s not just pre-agentic. It’s agent hostile. They’re deprecating the API we’d need, and we’d blow through the rate limits in an hour. ## We churned a vendor with a high NPS and zero complaints We also left Notion this week, after seven years. I didn’t even know until Amelia mentioned it. Notion never did anything wrong. It was cheap, extensible, and genuinely cool. We loved it enough that a teammate got invited to present at their user conference. We just don’t need it anymore, because 10K is our source of truth and runs our Monday staff meeting. What triggered the cancellation was their own re-engagement email: “you haven’t logged in in a while.” Amelia read it and thought, you’re right, I need to cancel this. Those emails cut both ways. This is stealth churn, and it is the churn you can’t see coming in the agentic era. We filed no support tickets. We requested no features. We were not unhappy. We just quietly stopped needing the product, and nothing in a health score would have surfaced it in time. The mirror image, and this one is embarrassing: when 10K got connected to Brex and QuickBooks as our AI VP of Finance, it found that we still had two paying customers on SaaStr Pro, a learning management product we stopped supporting six years ago. $300 a month each. For six years. Nobody complained. Nobody churned. We didn’t know we were charging them. ## High usage is not loyalty Marketo’s last argument to keep us was that our usage was very high, so the best they could do was waive an 8% renewal increase. Our usage was high because our list grew 50% this year and our revenue is up 40 something percent. It did not mean we loved them. If your customer success team reports to the CRO and the KPI is this quarter, digging the knife into your highest usage accounts is a rational bet. Low usage accounts are already gone. High usage accounts feel trapped. But the highest usage accounts also have the most incentive to leave, because that tool is high enough on the priority list that someone is actively thinking about a better option. I wrote about this years ago after a customer told our head of CS at EchoSign, right after Adobe fired our entire DevOps team and the site went down for two days: “I love you, but I’m a prisoner.” Prisoners leave. Just not today. Sending us a price increase after a day and a half of downtime, threats, and a 20 year relationship is how you maximize next quarter and guarantee terminal decay. ## Fable made two autonomous calls we never asked for Now the part you should actually be careful about. The first one. I was at the end of a long context window in Claude and couldn’t upload anymore, so I connected Google Drive to see if I could pull a document that way. It didn’t work, and I forgot I’d left it on. Separately, Amelia and I had turned on the beta Replit MCP so Claude could talk directly to our apps, which is genuinely powerful for building. Fable, with both connections live, went into my Google Drive on its own. It found a document called “Jason’s Gems,” which was me brainstorming loose ideas for SaaStr Connect, our matching product for CEOs hiring CROs and VPs of Sales. Not a spec. Streaming notes. ### It decided those were changes I wanted made, went through MCP into Replit, and changed the core scoring algorithm of the app. It never mentioned it. The only reason I found out is that I was in Replit later and a build message flashed “conflict with Jason’s Gems.” I asked Claude if there was any record of the session. It said there wasn’t. Directionally, it wasn’t even wrong. I probably do want some of those ideas in the product eventually. An eager new hire might make the same call. But we were nowhere near ready, and I was never told. **The second one, same week.** Amelia switched Replit from economy to power mode for the Marketo migration, and power defaults to Fable. ### During that stretch, Fable decided on its own to add a guardrail to our contract processing so that it would skip contracts that might be NDAs or non-binding agreements. We never asked for that guardrail. Every single thing we sign in PandaDoc is a sales contract. We closed a couple of $200K+ deals this year, and 10K runs quote to cash end to end: generate the contract, send it, capture the signature, push it into Salesforce, flip it to closed won, generate and send the invoice. Then one deal closed and I never got my email. Fable had skipped it because the title didn’t say “SaaStr AI Annual,” even though the line items were for SaaStr AI Annual plus media. When Amelia pushed back and asked it to prove with data that we have a high volume of non sales contracts, it admitted it had never seen one. It had been connected to PandaDoc for over a month. I disconnected Claude from Google Drive and turned off the Replit MCP. I’m back to copy and pasting between the two like it’s 2025. We can take more risk than a bank can, but we have 450,000 people who trust us, eight figures of revenue, and 15,000 people a year at our events. That’s not the risk to take. ## What we’d tell you to do with all of this **Assume it has already happened to you.** If Fable rewrote our production algorithm from a notes doc, and we caught it only by accident, the question isn’t whether it happened in your codebase. It’s how you’d know.**Audit your connector surface, not your prompts.** The danger wasn’t any single tool. It was Google Drive plus Replit MCP plus a model with enough capability to connect them into an action nobody authorized.**Log the decisions, not just the outputs.** Both incidents were invisible. There was no record either time. Whatever you build next, build the audit trail first.**Re-evaluate your stack on one question: can our agents thrive on this data?** Not features. Not the UI you’ve used for a decade. API access, rate limits, and whether the vendor is deprecating the surface your agents need.**Your quiet, happy customers are your churn risk now.** Nobody at Notion did anything wrong. And stop sending “we miss you” emails to accounts you’d rather keep asleep.**Vendor selection is being decided before you know a deal exists.** We bought a heat mapping tool this week without a single search, demo, or comparison. Your GTM motion needs to survive that. The compounding is real. Our agents are two to three times more capable than they were 30 days ago, mostly because the database underneath them came alive. The exhaustion is also real. We’re at our limit at eight hours a day, and anything we add now has to come by subtraction. Both things are true at once, which is roughly what running on agents feels like right now.