How Pixel photography became an AI proving ground

Google's Pixel phones have used AI and machine learning for over a decade to advance phone photography, with features like Night Sight, Magic Eraser, and Best Take, according to an exclusive interview with Isaac Reynolds, who has led the Pixel camera team for the past decade. Reynolds said progress has accelerated in the past two years due to generative AI and Gemini-class models, making previously hard problems easier. The Pixel 11 is expected to launch at the Made by Google event on August 12.

For over a decade, Google's Pixel phones have used AI and machine learning to leap forward in phone photography. The company has pushed the bleeding edge by using software and algorithms to make its phones produce images more and more like a professional camera. While there have occasionally been mixed results, we can't say that the Pixel camera hasn't advanced the boundaries of what's possible. Year after year, the devices have continued to launch new, updated, and refined features to expand and improve phone photography using AI models. In an exclusive interview with The Deep View, Isaac Reynolds, who has led the Pixel camera team for the past decade at Google, explained how his team has set its sights on tackling a series of "durable problems" in phone photography: - Dynamic range - Low-light performance - Zoom quality - Detail and sharpness - Background blur - Motion freezing - Artifacts - Timing "We chose durable problems for a reason," said Reynolds, " because no matter how good the technology gets, users always want more." As we prepare for the launch of the Pixel 11 https://x.com/madebygoogle/status/2086845666538520927 , let's do a quick recap of the AI and ML features that have launched on the Pixel phones across the generations: Pixel 2016 : Always-on HDR+ Pixel 2 2017 : Portrait Mode with ML depth estimation in a single lens Pixel 3 2018 : Night Sight, Super Res Zoom, Top Shot Pixel 4 2019 : Live HDR+, Dual Exposure Controls, computational astrophotography Pixel 5 2020 : HDR+ with Bracketing, ML-powered Portrait Light Pixel 6 2021 : Real Tone, Face Unblur, Motion Mode, Magic Eraser Pixel 7 2022 : Photo Unblur, Guided Frame, improved Super Res Zoom Pixel 8 2023 : Best Take, Magic Editor, Audio Magic Eraser, Video Boost Pixel 9 2024 : Add Me, Reimagine, Auto Frame, Night Sight Panorama Pixel 10 2025 : Camera Coach, Auto Best Take, diffusion-based Pro Res Zoom One thing Reynolds made clear was that progress has accelerated significantly in the past two years because of generative AI and the integration of Gemini-class models into the team's toolset. "Things that we thought were going to be really, really hard are actually really easy," said Reynolds. "I'm seeing examples of models that just work. We are able to scale them much harder, much further, much faster than we ever could have suspected… It breaks our planning process, because things are so much easier now than they were even a year or two ago… I'm looking forward to the next three years. It makes my job a little bit easier when all the things just magically work." Our Deeper View Naturally, one of the biggest questions becomes what's actually a photograph? If you can use generative AI to fix the light, make night appear like daytime, add a different sky into a landscape shot, add a person who wasn't in the shot into a group photo, and extend the canvas to add parts to an image that weren't originally there, then a photograph starts to become more like a work of art than a snapshot of reality. Of course, this question goes back to the beginnings of photography itself. Ansel Adams famously talked about how he would manipulate an image in the darkroom so that the photo didn't merely capture what he saw but would portray what it felt like to stand in a particular spot. Today's phones have AI capabilities that give us more and more tools to make art with photography, if we choose to look at it that way. And no phone pushes the boundaries of what's possible more than the Pixel. That makes it exciting to see what comes next, which we'll learn at this week's Made by Google event on August 12. I'll be onsite in New York and will report back the most interesting new developments.