{"slug": "sydney-researchers-uncover-a-hidden-limit-to-ai-predictability", "title": "Sydney researchers uncover a hidden limit to AI predictability", "summary": "University of Sydney researchers led by Professor Pulin Gong and PhD student Andrew Ly found that neural networks' training outcomes are organized into fractal \"riddled basins,\" so arbitrarily small differences in initial parameters can send training to different solutions even in fully deterministic settings. In one larger image-recognition network, making starting settings 10 billion times more precise produced no measurable improvement in reproducibility, and changing the starting settings by a single bit sent training to a different internal configuration in about 48 percent of cases, based on roughly 500,000 trained networks. The study, published in Nature Machine Intelligence, attributes AI unpredictability to the underlying mathematics of learning rather than hardware randomness or training-data order, a finding the researchers say could inform methods for making systems more consistent.", "body_md": "Researchers at the University of Sydney have identified a fundamental mathematical reason why artificial intelligence systems can learn different behaviours even when trained under almost identical conditions, a discovery that could help improve the reliability and trustworthiness of AI technologies.\n\nThe study by [__Professor Pulin Gong__](https://profiles.sydney.edu.au/pulin.gong) from the [__School of Physics__](https://www.sydney.edu.au/science/schools/school-of-physics.html) and his PhD student Andrew Ly, has been published in [*__Nature Machine Intelligence__*](https://www.nature.com/articles/s42256-026-01336-3), one of the world’s leading journals in artificial intelligence research.\n\nAs AI systems become increasingly embedded in healthcare, business, research and everyday life, ensuring they behave consistently and predictably has become a major challenge. The new research suggests that some forms of AI unpredictability are not due to engineering flaws or inconsistencies computer hardware, but in the underlying mathematics of how neural networks learn.\n\n“Researchers have often attributed differences in AI behaviour to external sources of randomness, such as the order in which training data is presented or tiny numerical variations introduced by hardware,” Professor Gong said.\n\n“Our findings point to something deeper. Even when these factors are removed, extremely small differences in a model’s starting conditions can still lead to different outcomes.”\n\nThis research provides a new way of understanding the limits of predictability in AI, and how we might overcome them.\n\n##### Professor Pulin Gong\n\nSchool of Physics\n\nNeural networks, the technology behind today’s most advanced AI systems, learn by adjusting millions or even billions of internal parameters during training. The Sydney team investigated why networks sometimes arrive at different solutions despite using the same data, training procedure and deterministic computing environment.\n\nAndrew Ly said: “Imagine two AI systems trained to detect cancer using the same medical images and the same training procedure. Tiny differences in their initial parameters could lead them to learn different behaviours. Our research helps explain how such differences arise and might inform methods for making these systems more consistent.”\n\nUnderstanding the source of these differences could guide the design of more dependable systems and reduce the cost and uncertainty associated with developing large-scale models.\n\n### **AI training map is a fractal**\n\nTo explain the phenomenon, the researchers use the analogy of a map. Imagine every possible starting point for training as a location on that map, coloured according to the outcome it eventually produces. Regions leading to the same outcome form what mathematicians call a “basin of attraction”.\n\nThe team found evidence that these basins are organised as highly intricate fractals known as “riddled basins”. In such a landscape, every point lies arbitrarily close to points leading somewhere else. No matter how far you zoom in, the different regions remain interwoven.\n\nThe implication is striking. Increasing the precision of a network’s starting conditions may do little to improve confidence about where training will end up. In one larger image-recognition network, making the starting settings 10 billion times more precise produced no measurable improvement in reproducibility. Altering that network’s starting settings by the smallest amount a computer can register, a single “bit”, sent training to a different internal configuration in close to half (about 48 percent) of cases.\n\nTo test the idea, the researchers trained approximately 500,000 neural networks, analysing systems used for tasks including image recognition and text processing. Evidence consistent with “riddled basins” fractal structure was found across a wide range of network architectures, datasets and training methods.\n\nImportantly, the effect remained even in fully deterministic environments where repeated training runs would otherwise produce identical results.\n\n### Model helps explain different AI outcomes\n\nThe findings may help explain several longstanding observations in AI research, including why independently trained models often behave differently and why both large and tiny disturbances can sometimes have similar effects on outcomes.\n\nThe work also revealed a surprising trade-off between performance and reproducibility. The settings enabling networks to learn most effectively also made their outcomes harder to predict and reproduce. However, reducing the ‘learning rate’ – the size of the step a network takes each time it adjusts itself – made the results reproducible, but the networks learned less effectively.\n\nRather than viewing unpredictability as a problem that can be completely engineered away, the researchers argue it may represent a fundamental feature of how many neural networks operate. The study introduces new tools for measuring this sensitivity and points towards future approaches, such as redesigning network architectures to reduce the mathematical conditions that give rise to instability.\n\n“AI is becoming increasingly important across society,” Professor Gong said. “To build trustworthy systems, we need not only accurate predictions but also confidence that those predictions can be reproduced. This research provides a new way of understanding the limits of predictability in AI, and how we might overcome them.”\n\n### Main photo: 'Riddled basin' fractal\n\nEach coloured point represents a starting setup for an AI model: blue leads to one training outcome, orange to another. Together, they form a striking butterfly-like fractal pattern. Image: Pulin Gong/University of Sydney\n\n### **Research**\n\nLy, A. and Gong, P. **‘** Fractal basin geometry and the limits to predictability and reproducibility in deep learning’ (*[Nature Machine Intelligence](https://www.nature.com/articles/s42256-026-01336-3)* 2026). DOI: 10.1038/s42256-026-01336-3\n\n### **Declaration**\n\nThe authors declare no competing interests. The study was supported by the Australian Research Council.\n\n## Manual Name : Professor Pulin Gong\n\nManual Description : School of Physics\n\nManual Address :\n\n### Manual Addition Info Title :\n\nManual Addition Info Content :\n\nProfile image : /content/dam/people/academic-and-research-staff/associate-professor-pulin-gong.jpg\n\nManual Type : profile\n\n_self\n\nAuto Type : contact\n\n## Auto Addition Title :\n\nAuto Addition Content :\n\nAuto Name : true\n\nAuto Position : true\n\nAuto Profile image :\n\nAuto Phone Number : false\n\nAuto Mobile Number : true\n\nAuto Email Address : true\n\nAuto Address : false\n\nUUID :\n\n## Manual Name : Media enquiries\n\nManual Description : Marcus Strom, Media Relations Adviser (Science)\n\nManual Address :\n\n### Manual Addition Info Title :\n\nManual Addition Info Content :\n\nProfile image :\n\nManual Type : contact\n\n_self\n\nAuto Type : contact\n\n## Auto Addition Title :\n\nAuto Addition Content :\n\nAuto Name : true\n\nAuto Position : true\n\nAuto Profile image :\n\nAuto Phone Number : false\n\nAuto Mobile Number : true\n\nAuto Email Address : true\n\nAuto Address : false\n\nUUID :", "url": "https://wpnews.pro/news/sydney-researchers-uncover-a-hidden-limit-to-ai-predictability", "canonical_source": "https://www.sydney.edu.au/news-opinion/news/2026/11/06/sydney-researchers-uncover-a-hidden-limit-to-ai-predictability.html", "published_at": "2026-11-10 13:34:54+00:00", "updated_at": "2026-11-10 13:53:46.722391+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "neural-networks", "ai-research", "ai-safety"], "entities": ["University of Sydney", "Pulin Gong", "Andrew Ly", "Nature Machine Intelligence", "School of Physics"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/sydney-researchers-uncover-a-hidden-limit-to-ai-predictability", "markdown": "https://wpnews.pro/news/sydney-researchers-uncover-a-hidden-limit-to-ai-predictability.md", "text": "https://wpnews.pro/news/sydney-researchers-uncover-a-hidden-limit-to-ai-predictability.txt", "jsonld": "https://wpnews.pro/news/sydney-researchers-uncover-a-hidden-limit-to-ai-predictability.jsonld"}}