FILE type makes your documents, images, video, and other unstructured data native to your lakehouse and ready for AI.
by Michael Armbrust, Burak Yavuz, Dejan Krakovic and John Spencer Your data estate holds far more than structured tables, metrics, and transaction logs - it holds contracts, product images, call recordings, and video. AI can now turn that unstructured data into something you can query and analyze, but only if it is governed and managed alongside everything else.
Today we're announcing the beta of FILE type: a new column type that stores unstructured data as a native, governed column in your tables. With** FILE**, your unstructured data is AI-ready: queryable, secured, and managed right alongside your structured data, instead of living in a separate system. Benefits of FILE include:
FILE is an innovation we are driving in the open: we are working with the community to build support directly into Parquet and Delta Lake, so the entire ecosystem can leverage it. That keeps your multimodal data open and portable, so you're never locked into a single vendor or model provider.
Today, your data teams want to build applications on multimodal data to drive business impact:
FILE reduces friction through the entire lifecycle of multimodal data - from exploratory analysis, to model training, to serving data to agents in real-time. To demonstrate this, we will walk through one example, but you can swap in your own use case as you read along.
Let's start our example with a pointed question from the CEO of an autonomous-driving company:
Can we reduce the number of random stops? When our self-driving car stops for no apparent reason, passengers lose confidence in our service
The data needed to address this is unstructured, dashcam videos captured from each self-driving car. Here is how FILE makes that data AI ready so we can address the CEO’s request
Figure 1: Conceptual diagram demonstrating a FILE column storing videos
footage
**Below is example syntax for creating a table with a FILE column. **To try out the complete run through of a file-processing pipeline using FILE, see the accompanying documentation which includes example notebooks.
Figure 2: Diagram demonstrating processed video FILEs
frame
hazard
hazard = none
The payoff with using FILE is that you can easily work with gigabytes of video without taking a hit on query performance. Because the FILE column holds a lightweight reference, the engine pulls the actual bytes only at the step that needs them. Contrast that with encoding the raw binary in the table, where every operation drags the large binary through the engine’s memory and hinders performance.
Figure 3: Diagram demonstrating querying multimodal table
hazard
The agent can now accurately answer the question quickly because everything lives in one row: the original video frame as ground truth, the embedding, extracted insights like present hazards, and the trip metadata (speed, timestamp, etc.) all sit side by side.
We often see customers try to solve use cases like this by storing a URL string containing a path to the file in the table. However, those strings are governed by an entirely different system of broad, folder-level permissions. You now are forced to maintain two permission models for one dataset, and a row filter that protects a path does nothing to protect the video at the end of it.
We have solved this governance challenge by integrating FILE into Unity Catalog, where FILEs are also secured by row and column-level access controls as well as attribute-based access control (ABAC), ensuring the right people have access to your data.
Another problem that teams face today is coordinating the lifecycle of their unstructured data. If someone deletes a video in object storage, the table has no idea of the storage lifecycle policy and the table knows nothing about each other, so you're left with a reference that points to nothing.
With FILE, the video's lifecycle travels with its row in the dataset: delete a row, and the data and its reference stay in sync - no orphaned files, no compliance gaps. Now your teams can move quickly without worrying about right-to-be-forgotten requests (e.g. GDPR) and a manual scramble to find every copy of the data.
FILE now enables your unstructured data to benefit from the same open foundation, governance model, and AI stack you already use for structured data.
AI_PARSE_DOCUMENT
, AI_QUERY
, and your own UDFs directly over the FILE column to turn raw documents, images, and video into the structured columns your agents and analysts already query.FILE type is available now in Beta and we would love your feedback. **Read the **Beta documentation for the full list of what you can try out today and step-by-step guidance.
This is just the start for FILE type and we have an exciting roadmap of features coming soon that will accelerate your AI projects, including:
If you want to learn more about how we are optimizing AI workloads in Databricks, reach out to your Databricks account team. Subscribe to our blog and get the latest posts delivered to your inbox.