ai/ml
VAST Data and lakehouse analytics supplier Cloudera have a strategic partnership to help keep GPUs processing AI workloads fed with data moving through a combined AI factory data pipeline,
Cloudera’s containerized data engineering, streaming, analytics, machine learning, and AI data services will be integrated with VAST’s AI Operating System (AI OS), its data services stack – DataSpace, DataBase, DataStore, DataEngine, AgentEngine, and InsightEngine – all layered on top of its DASE (Distributed and Shared Everything) storage architecture. AI OS, based on Nvidia’s AI Data Platform reference design, is available in the the on-premises, neocloud, GCP and Azure environments. A PolicyEngine and Tuning Engine were added to AI OS in February, with both due for release by the end of the year.
Jeff Denworth, Co-Founder at VAST Data, said: “Most enterprises already have the data they need for AI. The challenge is unlocking the value in data to create a continuous pipeline of AI inference, fine-tuning, and data analysis to build the next generation of intelligent applications. Together, Cloudera and VAST are helping customers build AI factories that connect data, intelligence, and infrastructure into a single operational platform for AI across hybrid environments.”
The two companies say VAST’s AI OS transforms latent enterprise data into activated AI-ready data, while Cloudera delivers data engineering, analytics, governance, and AI services on top. The two say they provide:
A unified AI factory architecture from raw data ingestion to model deployment
Consistent operations anywhere across data centers, private cloud, and public cloud
Elimination of GPU starvation through ultra-high-bandwidth, low-latency data pipelines
Significantly improved compute efficiency by ensuring GPUs operate at sustained utilization levels
Superior storage performance at scale for structured, unstructured, and multimodal datasets
Secure private AI environments with enterprise governance and compliance
Customers are told this means they can implement “production-grade AI systems that continuously transform enterprise data into actionable intelligence for training, inference, analytics, and agentic applications” applicable to normal, private and sovereign AI. They will be able to accelerate Apache Spark workloads with Nvidia cuDF to accelerate Cloudera Data Engineering workloads by allowing Spark jobs to leverage VAST’s data services via GPU-accelerated processing, supporting more AI workload optimization.
Abhas Ricky, Chief Strategy Officer, Chief Business Officer and GM, Applied AI at Cloudera, said: ” Enterprises are investing billions in GPUs, yet many struggle to achieve full utilization due to data bottlenecks. Our partnership with VAST eliminates GPU starvation and enables customers to build true AI factories—where data flows seamlessly from ingestion to insight.”
The joint Cloudera–VAST AI factory offering is available immediately through both companies’ enterprise sales teams and partner ecosystems. Learn more here.