Provisioning for the Agentic Era: How Databricks Built a Self-Serve Infrastructure Vending Machine

Databricks built the Field Engineering Vending Machine (FEVM), a self-serve infrastructure provisioning system that gives each engineer an isolated environment in minutes, governed centrally and cleaned up automatically, to support its field engineering organization as it grew from under 1,500 to over 7,000 people. FEVM uses Databricks-native components and agents, with a React frontend and Python backend deployed via Databricks Apps, allowing use-case-based provisioning where engineers describe their task in plain English and receive a configured environment.

by Evan Pandya /blog/author/evan-pandya , Greg Wood /blog/author/greg-wood and Joel Thomas /blog/author/joelthomas At Databricks, our field engineering organization exists to help customers succeed. That means building demos, reproducing issues, testing features against real workloads, and being ready on short notice to show a prospect exactly what's possible on the platform. Never has this been more imporant than in the age of AI, where a single week can mean a fundamental shift in what is possible. As Databricks has grown, it has become more important- and more challenging- to enable our field organization in a way that's fast, governed and cost-aware. To solve that problem, we set out to build the Field Engineering Vending Machine, or FEVM for short. Like any good vending machine, the idea is simple: you say what you need, you get it, and when you're done, it goes away. Perhaps most importantly, FEVM is built with Databricks-native components at its core, and agents as first-class citizens. Just over three years ago, field engineering was under 1,500 people. A small number of shared workspaces covered most use cases, and manual maintenance kept things running reasonably well. But Databricks grew fast. GTM is now over 7,000 people, with field engineering making up a significant portion of that. Infrastructure that worked at one size needs a different design at another. Databricks workspaces are optimized for a small number of administrators managing many users, which makes sense for most enterprise deployments. Field engineering has different requirements, however: nearly everyone needs admin-level access to actually do their jobs. Configuring demo environments, testing preview features, running customer-specific scenarios — all of it requires the kind of control that admin access provides. At scale, coordinating work across shared workspaces creates real operational complexity. Multiple engineers working in the same environment during a critical demo moment can interfere with each other's work, and platform limits — on catalogs, Lakebase instances and concurrent workloads, for example — become an active concern rather than a background one. Cost attribution grows harder to track as usage increases and ownership becomes less clear. The observability problem also compounded these challenges. When something unexpected happened in a shared workspace, tracing what occurred, when and why required manual investigation. We needed a model that made every provisioning action transparent, auditable and attributable from the start. The insight that shaped FEVM was straightforward: what if every engineer could have their own isolated environment, provisioned in minutes, governed centrally and cleaned up automatically? When Databricks Apps became available, the opportunity was clear. We had been maintaining environments through a collection of individual jobs — one job per task, running separately and always slightly behind reality. Apps gave us a way to abstract all of that behind a single interface: tell us what you're trying to accomplish in plain english, and we'll figure out what infrastructure to provision. That became the core design principle for FEVM: use-case-based provisioning. You don't request "a workspace." You describe what you're trying to do, whether it’s building a demo for a financial services customer, reproducing a support issue, running a hackathon session, and you get an environment configured for that purpose. This is all abstracted with an MCP at its core, meaning that chat, external services, and agents are all served from a single point of control; with the rise of tools like Claude, this means that adding a simple .md skill file allows you to ask for a new environment on the command line in plain english and log in minutes later. More complex workflows become similarly trivial from a user perspective. The app is built on a React frontend with a Python backend, deployed via Databricks Apps. Terraform runs in the background to handle actual cloud resource provisioning across AWS, Azure and GCP. A state and configuration database - run on Lakebase - tracks every resource: what it is, who owns it, what it's for, when it expires. The Databricks Apps development experience itself was straightforward. The more complex work was integrating into the broader ecosystem: wiring up Git-based workflow launches, handling identity across corporate systems, and ensuring that automating administrative actions against real backend infrastructure stayed within our security boundaries. We operate under a number of security exceptions for exactly this reason and have been deliberate about designing for good stewardship at every step; new templates are vetted, hardened, and tested repeatedly to make sure they do not expose new threat vectors. When a field engineer opens FEVM, they're greeted with a search interface. They can browse a template catalog to find the right environment type — stable serverless on AWS, a multi-cloud setup, an environment with Lakebase autoscaling pre-configured, for example — and configure from there. They choose a cloud provider and region, describe what they're building, give it a name or accept a random one , and deploy. More recently, we’ve enabled agent-first workflows that allow users to leverage centrally published claude skills; this is an increasingly popular option that allows agents to automate away the UI and go directly to underlying FEVM APIs. This becomes especially powerful in multi-step and multi-tool workflows; for example, a user might tell their agent to spin up a new workspace, deploy several DABs that they’ve developed locally, and upload some data they have in S3, and then run a hydration script to populate a dashboard. This type of hyper-personalized setup would not be possible without a strong agent framework- and Databricks Apps makes this trivial. When a user or agent requests a new resource, behind the scenes the app fetches the appropriate terraform template, feeds it to a Git Runner, adjusts permissions, and then adds any user-requests “add-ons” such as Lakebase, notebooks, or pre-packaged assets hosted in a UC Volume.. Builder environments live for 90 days by default, with extension options available, while other resource types have configurable TTL. When provisioning completes, a Slack notification arrives. When the resource is approaching expiration, another notification goes out. When it's deleted, you're notified again. Transparency was a core design goal from day one: every lifecycle event is visible, and engineers always know the state of their resources. We also manage shared resources through the same interface. Standalone catalogs, for example, carry independent lifecycles from the workspaces they're attached to. Delete a workspace and the catalog persists; spin up a new workspace in the same region and the catalog reattaches automatically. This kind of resource-level lifecycle management matters because Unity Catalog and Lakebase both have hard platform limits, and with hundreds of engineers actively provisioning resources, centralized control is what keeps those limits from becoming a ceiling. An admin layer within the app lets our team manage global configuration — adjusting workspace limits by cloud, auditing active deployments, tuning defaults — directly from the UI. During BuildCon — one of our internal engineering events — FEVM handled close to 1,200 provisioning requests in a single day. Engineers across the organization spun up environments, did their work and let them expire, without coordination overhead or resource contention. We haven't hit a scalability issue to date, even at 5,000+ active users; as of writing, we’re managing over 2600 active deployments across 3 clouds That number matters to us not just as a stress test, but as a proof of concept. The hypothesis behind FEVM was that just-in-time, isolated provisioning could replace the friction of shared environments at scale. A single-day invocation count like that confirms the demand was always there and engineers just needed a tool that made it frictionless to act on. The broader vision we’re approaching at Databricks is an agent-first field engineering organization: an engineer describes a customer situation in plain language, an agent spins up an appropriately configured environment, retrieves relevant account context, attempts to reproduce the issue or build the demo, and returns a summary ready to take back to the customer. FEVM is the infrastructure layer that makes that possible. We've rearchitected the app twice to get here: redesigning the database schema, rebuilding the frontend, rethinking how we manage state. We've used AI to accelerate the coding, but stayed in the driver's seat on architecture throughout. The result is a system we trust to run at scale. A few principles that shaped this build: Build for Databricks on Databricks. Nearly everything in FEVM runs on the platform itself. This gives us firsthand insight into what works at scale and keeps us close to the product experience we're selling. We're focused on three things: expanding the natural language provisioning interface, shipping the MCP integration for tool-based access, and scaling support across the broader GTM organization. As agents take on more of the work field engineers do today, FEVM will become the resource layer those agents run on — not just a provisioning tool, but core infrastructure for an agentic workflow. Watch the breakout session from Data and AI Summit 2026 Provisioning for the Agentic Era: How Databricks Built a Self-Serve Infrastructure Vending Machine https://www.databricks.com/dataaisummit/session/provisioning-agentic-era-how-databricks-built-self-serve-infrastructure for more details on the FE Vending Machine. To learn more about building production-grade apps on Databricks, check out Databricks Apps https://docs.databricks.com/aws/en/dev-tools/databricks-apps/ , Lakebase https://www.databricks.com/blog/what-is-a-lakebase and Agent Bricks https://docs.databricks.com/aws/en/generative-ai/agent-bricks/ . Subscribe to our blog and get the latest posts delivered to your inbox.