cd /news/ai-agents/visa-tested-22-ai-shopping-agents-23… · home › topics › ai-agents › article
[ARTICLE · art-171733] src=linas.substack.com ↗ pub= topic=ai-agents verified=true sentiment=↓ negative

Visa tested 22 AI Shopping Agents & 23% bought the wrong thing 🤖🛍️; Nubank’s Q3 2026 earnings: half the profit growth came from taxes, the 12% margin that’s really 10% & what NU stock is worth 🏦💜

Visa's Agentic Commerce Benchmark tested 22 AI shopping agents built on models from Google, Meta, OpenAI, Anthropic and eight other labs, finding that 23.1% of runs ended with the shopper buying a product that did not meet their needs despite a 57.2% headline pass rate. The newsletter analysis notes that these purchases would clear checkout normally, with authenticated payments and tokenized cards, and that no party has yet agreed on liability when a shopper disputes an agent's mistaken purchase.

recommendation

Evaluate shopping agents on consistency across repeated runs, not just pass rate, and test whether pricier models actually reduce wrong-purchase rates before upgrading.

more in playbook: Evals and testing for LLM apps
by read2 min views1 publishedNov 18, 2026
Visa tested 22 AI Shopping Agents & 23% bought the wrong thing 🤖🛍️; Nubank’s Q3 2026 earnings: half the profit growth came from taxes, the 12% margin that’s really 10% & what NU stock is worth 🏦💜
Image: Linas (auto-discovered)

Hey Everyone,

Good morning & happy Wednesday! Today, we’re breaking down Visa’s new Agentic Commerce Benchmark (why the best AI models still fail & why a more expensive model doesn’t fix that, the consistency figure that matters more than the 57.2% headline pass rate, who’s liable when an agent buys the wrong item, what Visa’s next move means for agent builders, merchants, issuers & investors, & what to watch next + bonus deep dives into Visa’s latest financials, and How to Build a One-Person Company on Grok Bot inside), and diving deep into Nubank’s Q3 2026 earnings, a record $1.1B profit from Latin America’s largest digital bank where nearly half the growth came from a lower tax rate (why NU’s 12% lending margin is really 10%, why loan losses are outrunning the income that carries them, why NU stock still trades like a 19% tax rate is here to stay, & the 3 numbers to watch next + bonus deep dives into How AI Agents Are Going to Change the Future of Banking, How Nubank Ships AI Agents in Days, Not Weeks, and deep dive into Anthropic’s IPO inside). So let’s jump straight into the interesting stuff 🌶️

Visa tested 22 AI Shopping Agents and 23% bought the wrong thing 🤖🛍️ #

Following the agents 🤖 Visa V 0.00%↑ recently tested 22 AI shopping agents built on models from Google, Meta, OpenAI, Anthropic, and eight other labs. In 23.1% of all runs, the shopper ended up buying a product that didn’t meet their needs 😬

In a live deployment (via say Grok Bot, Claude Duet, and others here), each of those purchases would go through checkout like any other. The payment is authenticated, the card is tokenized, and the transaction settles. Nothing in the payment flow shows that the agent misunderstood the shopper, and nobody has yet agreed on who pays when the shopper disputes the charge.

If you read it that way, Visa’s Agentic Commerce Benchmark is basically an early estimate of the dispute bill agentic payments will run up. It also hints at where Visa wants to sit when that bill arrives. Let’s dive deeper into this to see why the best AI models still fail, and why a more expensive model doesn’t fix that either. We’ll also look at the consistency figure that matters more than the 57.2% headline pass rate, who is liable when an agent buys the wrong item, and what Visa’s next move means for agent builders, merchants, issuers, and investors.

If agents already buy through your app, your store or your cards, start with the liability section.

── more in #ai-agents 4 stories · sorted by recency
── more on @visa 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/visa-tested-22-ai-sh…] indexed:0 read:2min 2026-11-18 · —