I sent an AI agent to buy from real stores. Here's what actually breaks.

A developer built AgentiQA, an open-source tool that sends AI agents through real e-commerce checkout funnels to test agent-readiness. Testing revealed that many stores fail at critical steps like login walls, JavaScript-heavy interactions, and non-standard form fields, even when their markup is technically correct. The tool uses a text-only DOM snapshot and a payment-field safety guard to prevent agents from entering sensitive data.

AI agents are starting to shop for people. ChatGPT has an operator that clicks around the web. Perplexity ships a "buy" button. Anthropic and others are wiring agents into real checkout flows. If you run an online store, some fraction of your future customers won't be humans clicking — they'll be agents acting on a human's behalf. Which raised a question I couldn't answer for my own projects: can an AI agent actually complete a purchase on this store? Every "agent-readiness" tool I found just lints your markup — does your robots.txt allow agent crawlers, do you have structured data, that kind of thing. Useful, but it answers a different question. Clean markup doesn't mean an agent can navigate your funnel any more than a valid HTML resume means you can do the job. The only way to know is to send an agent through and watch where it dies. So I built AgentiQA https://github.com/OmkarPalika/agentiqa . It's open source MIT . Here's how it works and — more interestingly — what I found when I pointed it at real stores. Static checks stdlib only, no API key : does robots.txt allow agent user-agents Claude-User , ChatGPT-User , OAI-SearchBot , PerplexityBot , is there JSON-LD Product / Offer data, OpenGraph tags, a reachable sitemap. This is the cheap "is the door unlocked" pass. The live shopper agent : Claude driving a real headless Chromium browser through the actual funnel — find a product, add to cart, reach checkout — recording milestones as it goes. This is the part that answers the real question. No screenshots, no vision model. The agent reads a text snapshot of the DOM: the page URL and title, every interactive element indexed by a number, and the visible text. Roughly: URL: https://store.example/ TITLE: All Products INTERACTIVE ELEMENTS: 0 <a href=/product/blue-top Blue Top 1 <button Add to cart 2 <input type=text name=search Search ... VISIBLE TEXT truncated : Blue Top Rs. 500 In Stock ... The agent gets a tiny toolset — read page , click element id , type text element id, text , goto url , record milestone stage — and loops: read the page, decide, act, read again. Text-only is cheaper than vision, deterministic, and it's roughly what an agent shopper actually operates on anyway. Any tool that drives a browser through checkout has to never touch payment data. You cannot rely on a system prompt for that — prompts are suggestions, and a model under pressure to "complete the task" can rationalize its way past one. So the refusal lives in the executor, not the instructions. Before any keystroke, the field is checked: PAYMENT FIELD RE = re.compile r"card|cvv|cvc|cc-|expir|security.?code|pan\b", re.IGNORECASE def is payment field el - bool: for attr in "name", "id", "autocomplete", "placeholder", "aria-label" : v = el.get attribute attr if v and PAYMENT FIELD RE.search v : return True return False If the agent tries to type into anything that looks like a payment field, the tool returns a hard REFUSED and tells it to stop and summarize. It also never creates accounts and never submits a final order — it stops at the checkout page. The model's judgment is a second layer, not the only one. The obvious way to drive this is the Anthropic API, which costs ~$1–2 per full audit. But for a solo dev validating an idea, I didn't want a per-run bill. So there's a second driver that speaks to the claude CLI the Claude Code subscription over a small JSON protocol — one action per turn, resumed across steps: python -m agentiqa https://your-store.com --driver cli subscription, no API key python -m agentiqa https://your-store.com --driver api API key, faster default: api if ANTHROPIC API KEY is set, else cli Same browser loop, same safety guards — just a different way to get the model's next move. It means you can run the whole thing on a subscription you already pay for. Here's the honest part, and it wasn't what I expected going in. I ran the benchmark against a set of real, public demo stores — the kind built to be tested against, with genuine cart and checkout flows. My assumption was that agents would faceplant everywhere. They didn't. On clean, functional stores, the agent checked out fine. Product → cart → checkout, no drama. One "failure" in my batch turned out to be a dead server throwing a Cloudflare SSL error — not an agent problem at all. The failures that are real are dumber and quieter than "the agent is dumb": None of these show up in a markup linter. And here's the kicker I found: a store scored 1/4 on static readiness and the agent still checked out perfectly. Static score did not predict agent success. The behavioral test and the markup test measure different things — you need the behavioral one. pip install -r requirements.txt playwright install chromium free static check, no API key: python -m agentiqa https://your-store.com --no-agent full run with the live agent: python -m agentiqa https://your-store.com You get an HTML report: a verdict banner did the agent complete checkout? , a prioritized fix list, the funnel, and the full step-by-step transcript of what the agent did. There's also a batch mode that runs a list of stores and emits an aggregate leaderboard. Only run it against stores you own or have permission to test. Agentic commerce is early. Right now the interesting finding is that the gap between "works for humans" and "works for agents" is made of small, invisible things — and nobody's funnel-testing for them yet. The tool is open source; I'd genuinely like to know what breaks on your store, because my sample is small and demo stores are easier than messy production ones. Repo, demo GIF, and the full report format: https://github.com/OmkarPalika/agentiqa https://github.com/OmkarPalika/agentiqa If you run an e-commerce store and want to know whether an AI agent can buy from you — or you just want to see an agent narrate its way through a checkout and give up at a broken button — point it at your site and tell me what you find.