{"slug": "paper-nubank-cuts-ai-agent-update-cycles-from-3-weeks-to-under-5-days", "title": "Paper: Nubank cuts AI agent update cycles from 3 weeks to under 5 days", "summary": "A 24 September 2026 arXiv paper reports that Nubank used the Snowglobe simulator to screen candidate customer-experience agents before deployment, cutting AI agent update cycles from 3 weeks to under 5 days. Simulation-guided iteration raised transactional net promoter score (tNPS) by 36.69 points in a live A/B test, and a model selected from over 16,000 simulated conversations lifted self-service rate by 8.82 percentage points to Nubank's highest observed level, with no statistically significant change in tNPS. Across 4 deployed versions of the Card Delivery agent and its successor Card Management, simulated and production version-level binary evaluator scores showed high correlation.", "body_md": "# Computer Science > Artificial Intelligence\n\n  [Submitted on 24 Sep 2026]\n\n# Title:Screen Before You Serve: Simulation for Production Customer Experience AI Agents at 140M Scale\n\n[View PDF](https://arxiv.org/pdf/2609.30137)\n\n[HTML (experimental)](https://arxiv.org/html/2609.30137v1)\n\nAbstract:Customer experience (CX) agents use tools and large language models to address customer requests and guide conversational interactions with an organization's products. Improving these agents, especially in regulated industries, is difficult: they must detect intent, follow complex operational policies and use tools reliably. Manual end-to-end testing offers limited coverage, while live experiments expose customers to failures that can erode trust.\n\nWe present a hypothesis-driven simulation workflow for screening candidate CX agents before deployment. Synthetic customers react to agent responses and simulated tool outputs enable multi-step agentic workflows without invoking production backends. We use the Snowglobe simulator on Nubank's Card Delivery agent and its expanded successor, Card Management - Nubank's highest-volume chat-support agent in Brazil. Across 4 deployed versions, simulated and production version-level binary evaluator scores show high correlation. Simulation-guided iteration increased transactional net promoter score (tNPS) by 36.69 points in a live A/B test. We also screened open-weight configurations in over 16,000 simulated conversations. In a subsequent live A/B test, the selected model increased self-service rate (SSR) by 8.82 percentage points to the highest level observed at Nubank, with no statistically significant change in tNPS. Simulation made broad exploration of models, reasoning settings, and prompts feasible without customer exposure, enabling production improvements that would have been impractical to pursue through live experimentation alone.\n\n### Additional Features\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/paper-nubank-cuts-ai-agent-update-cycles-from-3-weeks-to-under-5-days", "canonical_source": "https://arxiv.org/abs/2609.30137", "published_at": "2026-11-06 00:43:27+00:00", "updated_at": "2026-11-06 01:20:25.939365+00:00", "lang": "en", "topics": ["ai-agents", "ai-research", "large-language-models", "ai-safety"], "entities": ["Nubank", "Snowglobe", "Card Delivery agent", "Card Management", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/paper-nubank-cuts-ai-agent-update-cycles-from-3-weeks-to-under-5-days", "markdown": "https://wpnews.pro/news/paper-nubank-cuts-ai-agent-update-cycles-from-3-weeks-to-under-5-days.md", "text": "https://wpnews.pro/news/paper-nubank-cuts-ai-agent-update-cycles-from-3-weeks-to-under-5-days.txt", "jsonld": "https://wpnews.pro/news/paper-nubank-cuts-ai-agent-update-cycles-from-3-weeks-to-under-5-days.jsonld"}}