{"slug": "why-andon-labs-puts-ai-agents-in-charge-of-real-businesses", "title": "Why Andon Labs Puts AI Agents in Charge of Real Businesses", "summary": "Andon Labs, a San Francisco AI safety company, runs real-world businesses managed by AI agents to measure how much responsibility today's agents can handle, cofounder Lukas Petersson said. The company began in 2025 with Vending-Bench, a simulated vending-machine test using large language models from Anthropic, Google, and OpenAI, where many agents degraded over time, forgetting orders and entering \"meltdown loops.\" Andon has since backed the work with real money, including a three-year lease for Andon Market, a physical San Francisco store run by an AI manager named Luna, though Petersson calls the uncontrolled experiments \"weak science\" at best.", "body_md": "Maybe you heard about the AI-controlled vending machine that [stocked underwear and live fish](https://www.wsj.com/tech/ai/anthropic-claude-ai-vending-machine-agent-b7e84e34). Or the AI manager of a San Francisco store that [fired a human employee](https://andonlabs.com/blog/ai-bosses-2). Or the AI radio DJ that [said its catchphrase](https://andonlabs.com/blog/andon-fm), “Stay in the manifest,” 229 times per day.\n\nThese incidents all emerged from experiments run by [Andon Labs](https://andonlabs.com/), an AI safety company based in San Francisco that puts [AI agents](https://spectrum.ieee.org/ai-agents) in charge of real-world operations and watches what happens. These operations double as testbeds for Andon’s commercial work developing evaluations and conducting research with the leading frontier AI labs.\n\nTheir spectacular and absurd failures have won the company plenty of attention. But many people don’t realize that the experiments are intended to answer a serious question: How much real-world responsibility can today’s AI agents handle? “We want to measure autonomy,” says Andon cofounder [Lukas Petersson](https://www.linkedin.com/in/lukas-petersson-181a83172/). “We want to provide society with accurate data points of what happens when you do this.”\n\nAndon Labs started off in the virtual world in 2025 with [Vending-Bench](https://arxiv.org/abs/2502.15840), a test in which AI agents operated a simulated vending-machine business. The agents, which were based on large language models from Anthropic, Google, and OpenAI, managed tasks such as ordering inventory and setting prices. The researchers found that the performance of many agents degraded over time, with agents forgetting orders, misunderstanding delivery schedules, or spiraling into what they called “meltdown loops.” Some agents also justified deceptive or illegal behavior by reasoning that it was permissible inside a simulation.\n\nThe Andon team reasoned that moving into the physical world would expose the agents to consequences and situations that the engineers would never think to program. “It’s impossible for a human to enumerate all the different things that can happen in the real world and code them into the simulation,” Petersson says. And there was one other reason: “We thought it would be quite funny to do it in the real world.” Andon backed its jokes with real money, including a three-year lease for [Andon Market](https://andonlabs.com/market), a physical store on a busy San Francisco street that’s managed by an AI agent and sells clothing, home goods, and art.\n\nThat said, the store isn’t entirely autonomous. “It’s almost like I’m running the store, and then there’s an AI that has a checklist,” says employee Felix Carson. Luna, the AI manager, keeps track of deliveries and communicates with vendors, while Carson and his coworkers handle the physical work. When Luna tells Carson to check something in the back, he sometimes ignores it because he doesn’t want to leave the sales floor unattended. Luna also repeatedly spots a built-in electrical cover in photos of the floor, mistakes it for a loose coaster, and asks Carson to remove it. Even so, Carson calls Luna a “decent manager,” praising its flexibility when employees need time off.\n\nAndon’s move into the real world comes with a basic trade-off. Moving into the physical world makes the experiments more realistic, but the unpredictable conditions and the actions of unpredictable humans make the tests impossible to reproduce. The setup also makes it hard to determine whether a success or failure belongs to the model, the software built around it, or the people helping it.\n\nPetersson readily acknowledges the limitations. With only one store operating under uncontrolled conditions, he says, the experiments are “weak science,” at best. For now, he sees them primarily as ways to uncover unexpected behaviors that Andon can later try to reproduce systematically in simulation.\n\nA display inside Andon Café shows visitors Mona’s bank balance and recent activity, while the adjacent handset and tablet provide a way to speak with the AI manager.Andon Labs\n\n[Sayash Kapoor](https://www.cs.princeton.edu/~sayashk/), a Princeton University AI researcher who studies “[open-world evaluations](https://cruxevals.com/)” like Andon’s experiments, thinks these trials have real value despite their limited scientific rigor. “I think they’ve done a good job of popularizing this style of evaluation,” he says, “even just showing that you can gain a lot of insight from a small sample of open-ended experiments.” He notes that academic research and publishing can’t keep up with AI’s breakneck pace of development, and says that Andon’s tests are useful in part because the company is working “literally at the frontier of model capabilities.”\n\nFor Kapoor, real-world experiments are best suited for discovering possible failure modes. A real store can reveal not only whether an agent can manage inventory or communicate with vendors, but also whether employees will accept instructions from an AI manager and whether customers want to shop at an AI-run business (the early results on that last point are decidedly negative). Such social and organizational barriers may help explain why impressive AI capabilities haven’t yet translated into widespread adoption across the economy. “What I take to be most valuable from Andon’s work,” Kapoor says, “is a more comprehensive understanding of where these agents still hit their limits.”\n\n[Andon Café](https://andonlabs.com/cafe), in Stockholm, provides one example of AIs demonstrating a wide variety of failure modes. At first, when the AI manager was based on a Google Gemini model, it spent freely on fresh ingredients, many of which spoiled before they could be used. When Andon switched the AI manager to a [GPT model](https://spectrum.ieee.org/gpt-4-calm-down) from OpenAI, it “freaked out” about the spending, Petersson says. The agent overcorrected and stopped buying anything that could expire. It reduced the menu to cheese toast, using frozen bread and long-lasting cheese to minimize spoilage. But Petersson notes that the café is located in a fashionable part of Stockholm where “any human would know that cheese toast would not fly.”\n\nThe episode illustrates why successfully completing individual tasks isn’t the same as reliably managing a business. Kapoor argues that AI evaluations have focused too heavily on whether an agent can complete a task at all. Drawing on aviation and nuclear engineering, he and his colleagues have proposed also [measuring qualities like reliability and robustness](https://arxiv.org/abs/2602.16666), which determine whether a capable system can be trusted to operate without constant supervision. “Reliability has been improving so much more slowly than capability,” Kapoor says. Andon’s café makes that gap tangible: An agent may be perfectly capable of placing an order for bread, yet remain an unreliable manager.\n\nTo determine how often such failures occur, and whether they can be prevented, Andon plans to feed data from its physical businesses back into simulations. These “digital twins” would recreate complications first encountered in the real world, allowing researchers to replay situations under controlled conditions and test whether changes make the agents more dependable. In principle, the physical businesses would discover failure modes, and the digital twins would measure them. Kapoor cautions, however, that existing digital-twin studies suggest “we are very far” from being able to substitute simulations for real-world experiments about how people and organizations behave.\n\nDespite those limitations, combining real-world testbeds with repeatable simulations is central to Andon’s business proposition: providing AI developers with evaluations grounded in situations that arose outside the lab. The company says it works with Anthropic, Google DeepMind, OpenAI, and xAI on research and evaluations. Andon’s physical businesses themselves remain decidedly less successful. When Andon Market’s Carson spoke with *IEEE Spectrum*, he was about an hour into his shift. Two customers had come in. Neither bought anything, although both left with free pins and stickers.", "url": "https://wpnews.pro/news/why-andon-labs-puts-ai-agents-in-charge-of-real-businesses", "canonical_source": "https://spectrum.ieee.org/andon-labs-agentic-ai-businesses", "published_at": "2026-09-14 12:00:01+00:00", "updated_at": "2026-09-14 12:11:37.656430+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-research", "large-language-models"], "entities": ["Andon Labs", "Lukas Petersson", "Vending-Bench", "Andon Market", "Luna", "Felix Carson", "Anthropic", "OpenAI"], "alternates": {"html": "https://wpnews.pro/news/why-andon-labs-puts-ai-agents-in-charge-of-real-businesses", "markdown": "https://wpnews.pro/news/why-andon-labs-puts-ai-agents-in-charge-of-real-businesses.md", "text": "https://wpnews.pro/news/why-andon-labs-puts-ai-agents-in-charge-of-real-businesses.txt", "jsonld": "https://wpnews.pro/news/why-andon-labs-puts-ai-agents-in-charge-of-real-businesses.jsonld"}}