{"slug": "the-checkout-button-an-ai-agent-built-in-30-minutes", "title": "The checkout button an AI agent built in 30 minutes", "summary": "Kameleoon reported that its Prompt-Based Experimentation (PBX) agents helped Dutch drugstore chain Kruidvat build and configure a sticky checkout-button test in about 30 minutes, producing a 1.3% lift in checkout starts and a 1.0% lift in conversion, both at p < 0.01. Kruidvat's two-person CRO team had identified in Contentsquare that large baskets pushed the main checkout button below the fold, and used PBX Build to generate the front-end code and PBX Configure to set the audience, goals and metrics, with a CRO specialist reviewing the output before launch. Kameleoon did not publish sample size, allocation, absolute rates or revenue effect for the test, which it says ran on sites drawing millions of visits a week.", "body_md": "## Experiment summary\n\n[Kruidvat](https://www.kruidvat.nl/) turned a cart friction signal into a launch-ready sticky checkout test with Kameleoon’s PBX agents. Kameleoon reports 1.3% more checkout starts and 1.0% higher conversion, both at p < 0.01.\n\n- Kruidvat’s two-person CRO team saw that large baskets pushed the main checkout button below the fold.\n- Behavioural data informed the hypothesis, while two AI agents generated the front-end variant and configured the audience, goals and metrics.\n- Kameleoon says the test was ready in about 30 minutes and reached significance within a few weeks.\n- Kameleoon reports a 1.3% lift in checkout starts and a 1.0% conversion lift, both at p < 0.01, on sites it says draw millions of visits a week. The page does not publish sample size, allocation, absolute rates or revenue effect.\n- *Hacking Growth* treats friction as the easier lever and recommends a 99 percent confidence level, which a p value below 0.01 matches.\n\nThe more products a customer adds to a basket, the further the main checkout button can slide down the screen. At [Kruidvat.nl](https://www.kruidvat.nl/), shoppers with larger baskets had to scroll before they could continue.\n\nThe Dutch drugstore chain had already found the friction in Contentsquare. The harder part was turning that observation into a clean experiment when two CRO specialists were responsible for more than [100 tests](https://industrycontents.com/ai-experimentation-workflow-fyxer/) a year across three ecommerce sites and two mobile apps. Kameleoon’s page gives that volume as more than 100 experiments a year in the text and 250 or more in a summary box.\n\nA human team already had the insight and the hypothesis. The useful question was whether AI could remove enough build and setup work to let a small growth team test the idea quickly without handing the rollout decision to the model.\n\n## The button fell below the basket\n\nAccording to [a case study Kameleoon published](https://www.kameleoon.com/customer/as-watson), Kruidvat’s CRO team combined behavioural evidence with business rules, analytics, technical knowledge and earlier experiment learning. The cart analysis showed a specific failure mode. As baskets grew, the main action moved below the visible area.\n\nThe hypothesis was narrow enough to test. If the checkout control stayed visible while a shopper moved through the cart, more people should start checkout and more visits should convert. The treatment was a sticky call to action, not a wholesale cart redesign.\n\nSean Ellis and Morgan Brown reduce this kind of problem to a formula in *Hacking Growth*: conversion rate equals desire minus friction. They argue that removing friction is usually easier than raising desire, which makes it the lower hanging fruit, and they tell teams to read their funnel report for large drop-off points. A checkout button pushed off screen is friction in its plainest form.\n\n## Two agents handled the mechanical work\n\nKruidvat used an internal AI workflow to turn the brief into a prompt for Kameleoon’s Prompt-Based Experimentation system. PBX Build generated the front-end code for the sticky button. PBX Configure helped set the audience, goals and performance metrics. A CRO specialist reviewed the output before the experiment could go live.\n\nPeter Harsevoord, Kruidvat’s CRO manager, says in the case that PBX lets his team build and prepare an experiment in minutes without compromising test quality. Kameleoon puts the path from identified opportunity to launch-ready experiment at about 30 minutes, including a final quality check by a CRO specialist. The source does not publish the old build time for the same treatment, so the 30 minute figure is a case duration. It still marks a concrete change in how a two person team gets from observation to a launch-ready test.\n\nThis is where prompt based experimentation becomes more than [code generation](https://industrycontents.com/ai-pricing-page-experiment-algolia/). A generated variant without audiences, exposure rules and outcome events is only a mock-up. PBX connected the treatment to the measurement layer, and a human quality check stayed between generation and traffic.\n\n## A few weeks produced a small commercial signal\n\nKameleoon reports that the sticky button test reached statistical significance within a few weeks. Checkout starts increased by 1.3%, and conversion increased by 1.0%. The company reports p < 0.01 for both metrics.\n\nThe two measures tell a coherent story. A persistent control should first make it easier to begin checkout. If the extra starts are useful rather than accidental, some of that movement should survive through purchase. Both reported metrics moved in the expected direction.\n\nThe public case gives no baseline checkout-start rate or conversion rate, so the percentages read as relative lifts and cannot be turned into extra checkouts per thousand visits. It also gives no orders, revenue per visitor, margin or average basket value.\n\nSmall lifts need large samples. In an example *Hacking Growth* borrows from Andy Johns, a 5 percent lift on a 3 percent baseline needed 72,300 visitors per variant, about 72 days at that team’s traffic. Required sample size grows with the inverse square of the lift, so a 1 percent lift needs roughly 25 times the traffic of a 5 percent lift at the same baseline. Reaching p < 0.01 within a few weeks therefore points to heavy cart traffic, which fits Kameleoon’s description of millions of visits a week across the sites. The page does not give the cart’s share of those visits.\n\nThe reported significance level also matches the playbook. Ellis and Brown recommend a 99 percent confidence level over the more common 95 percent, since at 95 percent about one winner in 20 can be a false positive, against one in 100 at 99 percent.\n\nCalling this an AI checkout experiment is accurate because the AI workflow produced and configured the treatment. The measured intervention is the sticky button, so the result speaks to a human UX hypothesis tested quickly, and the case does not compare AI-designed and human-designed UX.\n\nThe book also offers a way to size the result. At GrowthHackers, a test that moved the newsletter sign-up invitation from the bottom of the home page to the top carried an impact score of 4, an ease score of 9 and a confidence score of 8, and sign-ups rose 700 percent. The authors tell the story to show that expectations often miss, and that low-scoring ideas deserve a look. A sticky checkout button is the same kind of easy visibility fix and returned 1.0% here. Visibility changes can pay out very differently, which is why the book calls the ICE score a guide to relative priority.\n\n## What the case leaves open\n\nThe source calls the work an experiment and reports statistical significance, which implies a comparison against the existing cart. It does not describe the traffic split, randomisation unit, persistence rules for returning shoppers, sample size, number of conversions or stopping rule.\n\nA large, high traffic retailer can detect a small difference quickly, and a low p value says the difference is unlikely to be chance. Absolute rates and counts would add its commercial size, along with any novelty effect or device split.\n\nThe generated implementation may also have changed spacing or behaviour around the button. A screenshot of control and treatment, plus the interaction specification, would show exactly what customers saw.\n\nKameleoon supplied the PBX agents and published the customer story. [Peter Harsevoord](https://nl.linkedin.com/in/peterharsevoord), CRO manager at Kruidvat, is named and quoted, and he appears on Kameleoon’s 30 June 2026 webinar about the workflow. Kruidvat has not released the experiment dataset or its own statistical report. *Hacking Growth* lists what such a test summary holds: variants and audience, key metrics, dates, hypothesis, sample sizes, confidence and power, and confounders. Publishing the Kruidvat test in that shape would settle the open points.\n\n## The lever is a bounded automation loop\n\nA team can copy the operating method without building a machine learning function. Start with one observed customer friction, write the hypothesis and primary metric before prompting, let the agent generate only the bounded variant and configuration, then require a human review before exposure.\n\nThe important design choice is where automation stops. Kruidvat’s people kept the insight, the hypothesis and the quality check. The system absorbed mechanical work around variant creation and experiment setup. That boundary lets a small team increase testing capacity without letting an agent turn every plausible idea into production behaviour.\n\nThe next experiment should measure the workflow as well as the checkout treatment. Randomly assign comparable briefs to the AI-assisted process or the normal process, then compare time to launch, engineering minutes, defects found in QA and proportion of tests that reach a valid decision. Commercial lift still belongs to each individual customer-facing experiment.\n\nFor checkout conversion optimisation, the follow-up customer test should publish the control and treatment screens, allocation, sample, absolute conversion rates and confidence intervals. Revenue per visitor can be the primary business metric, with checkout starts, completion, average order value, page errors and return rate as diagnostic or guardrail measures. That follows the book’s advice to track the downstream metrics an experiment touches, since a gain in one can cost another.\n\n## Learning speed is the result worth keeping\n\nKruidvat’s reported 1.0% conversion lift is modest beside many AI case study headlines, and the operating lesson holds up at that size. A two person team found a familiar leak, produced a treatment quickly, kept a human checkpoint and waited for observed customer behaviour.\n\nThe more transferable gain is a shorter route from a CRO specialist’s observation to a launch-ready test. The case does not quantify how many additional tests the team could run. The sticky button may be the winning interface, and the workflow behind it is the part other teams can reuse: context first, then code and configuration, with customer behaviour as the judge of the result.\n\n| Experiment detail | What the reporting shows | \n|---|---|\n| Constraint | Large baskets pushed Kruidvat’s main checkout button below the fold, while two CRO specialists managed more than 100 tests a year. | \n| Hypothesis | Keeping the checkout action visible would increase checkout starts and completed conversions. | \n| Intervention | Kameleoon PBX agents generated the sticky CTA variant and helped configure its audience, goals and metrics, followed by human QA. | \n| Reported result | Kameleoon reports 1.3% more checkout starts and 1.0% higher conversion, both at p < 0.01, after a few weeks. | \n| Open questions | Sample size, allocation, absolute rates, revenue effect and stopping rule are not published. | \n\n## Sources and disclosure\n\n- Kameleoon’s [AS Watson and Kruidvat customer story](https://www.kameleoon.com/customer/as-watson)\n- Kameleoon’s [30 June 2026 webinar page](https://www.kameleoon.com/_vai_/ngM2gi3xnIAKzZFg/fr/webinar/uk-20260630-from-insight-to-impact-ai-powered-retail-at-as-watson) naming the participating practitioners and workflow\n- Kruidvat CRO manager [Peter Harsevoord on LinkedIn](https://nl.linkedin.com/in/peterharsevoord)\n- AS Watson’s [official description of its data and AI teams](https://185-anniversary.aswatson.com/innovation/a-team-of-data-experts)\n- Sean Ellis and Morgan Brown, *Hacking Growth* , on friction, sample size, confidence levels and test summaries\n\nKameleoon supplied the PBX tooling and published the performance figures, and its customer story page carries no publication date. Neither Kruidvat nor AS Watson has published the underlying allocation, raw outcome counts or statistical analysis.\n\n## FAQ\n\n## What did Kruidvat test?\n\nIt tested a sticky checkout call to action after behavioural analysis showed that large baskets pushed the existing button below the fold.\n\n## Where was AI used?\n\nKameleoon’s PBX Build agent generated the front-end variant, while PBX Configure helped set the audience, goals and metrics. A CRO specialist approved the setup.\n\n## What result was reported?\n\nKameleoon reports a 1.3% lift in checkout starts and a 1.0% lift in conversion, both at p < 0.01, after a few weeks.\n\n## Who published the result?\n\nKameleoon supplied the system and published the case. The sample, absolute rates, allocation and revenue impact are not public.\n\n## Why start with a sticky button?\n\nHacking Growth treats friction as an easier lever than desire, and a visible checkout button is a low effort fix to a drop-off point the team had already observed.\n\n## What can another growth team copy?\n\nGive an AI agent one bounded experiment brief, connect it to the approved measurement setup, keep human QA before launch and let a controlled business metric decide rollout.\n\nCompare more measured experiments in the [Signal Index](https://industrycontents.com/ic-lab/growth-signal-index/), browse the [Growth Signal archive](https://industrycontents.com/category/growth-signal/), or use the benchmark tools in the [IC Lab](https://industrycontents.com/ic-lab/).", "url": "https://wpnews.pro/news/the-checkout-button-an-ai-agent-built-in-30-minutes", "canonical_source": "https://industrycontents.com/kruidvat-ai-checkout-experiment/?utm_source=rss&utm_medium=feed&utm_campaign=editorial", "published_at": "2026-11-12 11:00:00+00:00", "updated_at": "2026-11-12 11:23:17.606598+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "generative-ai"], "entities": ["Kameleoon", "Kruidvat", "Prompt-Based Experimentation", "PBX Build", "PBX Configure", "Peter Harsevoord", "Contentsquare", "Hacking Growth"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/the-checkout-button-an-ai-agent-built-in-30-minutes", "markdown": "https://wpnews.pro/news/the-checkout-button-an-ai-agent-built-in-30-minutes.md", "text": "https://wpnews.pro/news/the-checkout-button-an-ai-agent-built-in-30-minutes.txt", "jsonld": "https://wpnews.pro/news/the-checkout-button-an-ai-agent-built-in-30-minutes.jsonld"}}