Always-on AI agents turn infrastructure into a continuous learning loop
AI agent infrastructure is evolving to support systems that move continuously among inference, feedback and training.
Cognition AI Inc.’s Devin now assists throughout the software development lifecycle, from planning and writing code to reviewing it and responding to production problems. That expanding role requires AI agent infrastructure capable of supporting continual learning at scale, according to Silas Alberti (pictured, right), head of research and founding team at Cognition.
“Our runs are always on,” he said. “While we still ship releases, package them up a little bit, I think the reality is we’re always training. We’re always trying to find the next data and the next reward signals to improve our models.”
Alberti and Chen Goldberg (left), executive vice president of product and engineering at CoreWeave Inc., spoke with theCUBE Research’s Dave Vellante and John Furrier at the Fully Connected event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed continuous learning, distributed AI training, infrastructure reliability and CoreWeave Forge. ( Disclosure below.)*
Always-on training raises the reliability bar
Training and inference are increasingly intertwined, especially in reinforcement learning workloads that generate responses and use the results to improve a model. Cognition has distributed training across data centers in multiple countries and across continents, making uptime across thousands of graphics processing units essential, according to Alberti.
“If you run a big training run, you’re not just looking at the GPUs, but you also want uptime,” he said. “If just one replica goes down, the whole training run goes down. I think a very important metric is getting to this 99.99% reliability.”
New hardware also shapes Cognition’s research direction, according to Alberti. Early access to Nvidia Corp.’s Vera Rubin platform allows its researchers to study the system and adapt model architectures while pursuing better price-performance. “With each generation, price performance just goes up, so we can do more with the same amount of compute,” Alberti said. “Being early and actually being able to study the kernels and the dynamics of this new platform allows us to prioritize our research investments.”
How CoreWeave Forge connects the AI agent infrastructure loop
Forge, which was announced during the event, connects inference, observation, data curation, model improvement and evaluation. The platform includes Agent Lens for tracing agent activity, along with model distillation and reinforcement learning capabilities. Its RL Rollouts service can hot-load updated model checkpoints into a live deployment without redeploying the system.
“CoreWeave Forge is … tailored for those AI loops, those learning systems of how you build agents and connect all the different steps,” Goldberg said. “You don’t have to do everything at once. You can always start with serverless inference, find your model and start.”
For Cognition, the value of the continuous loop lies in using real-world experience to improve subsequent versions of Devin. The goal is to make the agent more capable across long-running software projects, according to Alberti. “Agents are not just planning the code, writing the code [and] reviewing the code, but also becoming real production maintainers,” he said. “If there’s a production issue, Devin can jump on it, respond to it. And by the time you wake up, there’s already a [pull request] and you can merge it.”
Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the Fully Connected event:
( Disclosure: TheCUBE is a paid media partner for the Fully Connected event. Neither CoreWeave, the sponsor of theCUBE’s coverage, nor other sponsors have editorial control over the content on theCUBE or SiliconANGLE.)*
Photo: SiliconANGLE
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
- 15M+ viewers of theCUBE videos , powering conversations across AI, cloud, cybersecurity and more
- 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network
Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/
About SiliconANGLE Media
theCUBE AIand theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.