If you're clinging to the idea that you can't build a great business on an open ecosystem in the AI era, then Databricks is rapidly making that line of thinking obsolete. The company started in 2013 as a platform built around the open-source engine for processing big data, Apache Spark, which the founders created at UC Berkeley. Today, it's a platform where companies organize their data and build AI on top of it. It's valued at $134 billion, and it's headed for a blockbuster IPO soon. Over 60% of Fortune 500 companies use the platform.
What may be most impressive and surprising is that even as the company has grown its customer base and scaled its revenue to the moon, it has never shed its commitment to open systems. If anything, it's accelerating its commitment to openness. It used the Databricks Data + AI Summit this week to launch an open stack for agentic AI.
That included a flurry of developments:
OpenSharing: This is a platform that allows organizations to share AI components across clouds and platforms, including AI models, agents, AI skills, and AI applications. The goal is to help organizations avoid lock-in on proprietary platforms.OpenSharingis an OSS project hosted by Linux Foundation.Omnigent: This is a meta-harness, or a harness for harnesses. In other words, it's a powerful agent for managing, monitoring, and controlling your various agents. And Databricks announced that it's makingOmnigentopen-source.DBRX for open LLMs:DBRXis an open-source LLM from Databricks that it claims outperforms all existing open-source models at launch, a big claim. The goal is to raise the bar for what organizations can build without relying on proprietary AI. Because the full model is open, teams can fine-tune it or use it as a blueprint to build their own custom models.
Jonathan Frankle, chief AI scientist, told The Deep View, "[Databrick founders] Ali [Ghodsi], Matei [Zaharia], Reynold [Xi], and all of our leaders … have worked really hard over the past decade plus to just make sure things are open… It forces us to be on the ball every single day, and it makes our customers feel really comfortable being with us, because they know that they're not trapped here. I think it makes it easier to win business, and it forces us to work really hard to keep business."
In a question and answer session with the press at Databricks Data + AI Summit on Tuesday, Ghodsi said, "Our roots are open-source. Our roots are community. So we want to make sure that we can actually offer that we are independent, we are multi cloud, we work on top of all the clouds, we work on top of all the AI’s. We partner with all of those vendors and we work with open-source. So we think this is an important function in the market, and we hope to provide that."
Our Deeper View #
The bigger question is whether Databricks' open ecosystem strategy is an outlier, or if the company is on to something. Could its approach have broader merit and point the way toward an emerging trend? With all of the big tech companies releasing their own agents and their own frameworks for managing agents, it's all getting very confusing and siloed. More interoperability and standard would allow organizations to take advantage of the best of breed in terms of models, harnesses, and other AI modules. That's especially valuable since new breakthroughs are happening at an incredible pace across the AI industry, and it's unlikely to slow down.