Today we're open sourcing Modelplane, a control plane for AI inference. You install it in your own environment, and it operates your GPU clusters as one inference fleet: provisioning clusters, placing models across them, autoscaling replicas, caching weights, and routing requests through a single OpenAI-compatible gateway.
It's built on Crossplane, and it runs any model on any serving engine on any infrastructure, from a single GPU to disaggregated, multi-node serving.
The shift toward open inference# #
Open-weight models changed who runs AI. They can be post-trained, including with reinforcement learning, to compete with frontier models, and they put cost, governance, and data sovereignty back under your own control. So inference is moving outward, from the labs and hyperscalers that served everyone through an API to a much larger population of organizations running it on their own hardware: neoclouds turning it into a business, regulated and sovereign enterprises keeping it inside their own walls, and AI-native companies bringing their inference bills under control.
The open source community moved fast to meet this, with projects emerging at every layer: serving engines like vLLM, SGLang, and TensorRT-LLM; schedulers, gateways, and routers; and multi-node serving and weight distribution. And Kubernetes is emerging as the convergence point underneath all of it. The cloud-native community is making it a first-class platform for AI workloads, the major inference projects are standardizing on it, and neoclouds like Baseten and CoreWeave run their operations on it.
Inference is a fleet problem# #
Almost all of those open source projects, though, focus on serving inference within a single cluster, and inference almost never stays in one. Capacity is scattered across hardware types, providers, and regions, sovereignty and compliance pin workloads to particular places, and large clusters concentrate failure. So inference quickly grows into a fleet, and the hard problems move above the cluster: placing models across the available capacity, failing over across clouds and regions, routing by cost and sovereignty, provisioning capacity, and caching and moving weights, all managed as one inference platform.
The labs, the hyperscalers, and the managed providers have all built a system to do this, and they've all built it privately. The open one didn't exist.
Introducing Modelplane# #
This is why we built Modelplane. We've been watching Crossplane adopters build inference platforms across clusters and operate it at large scale, composing the clusters, the GPUs, the serving stacks, and the routing into their own control planes. We wanted to standardize those patterns, make them far easier to get started with, and contribute the result back to the community as open infrastructure.
Modelplane sits above your inference clusters and operates them as one inference platform, reconciling the whole fleet toward the state you declare. It does for the fleet what Kubernetes does for the cluster: platform teams and developers describe what they want as Kubernetes resources, and Modelplane composes the clusters, places the models, and exposes the endpoints to match.
Modelplane API#
The Modelplane API splits along the two teams that actually do this work, and fills in everything between them.
Platform teams define the inference platform. They describe the unified entry
point (an InferenceGateway
), the hardware shapes they offer (an
InferenceClass
), and the clusters that make up the fleet (an
InferenceCluster
), and Modelplane provisions and reconciles them:
ML and development teams deploy models. They declare a ModelDeployment
(the
model, its engine, serving topology, and a replica count) and a ModelService
to expose it as one
stable, OpenAI-compatible endpoint: That's the whole interface. Developers never name a cluster: they describe the hardware a replica needs and Modelplane's scheduler places it on a cluster and pool that fits, then hands off to that cluster's own scheduler and DRA to bind the GPUs. Engine flags pass through verbatim, so any container-based engine and any topology works without a change to Modelplane.
Modelplane capabilities# #
Once those resources exist, five things run continuously across the fleet:
Provisioning. Create clusters and GPU node pools on hyperscalers, neoclouds or bring your own on any Kubernetes, and install the serving stack on each.Scheduling. A two-level scheduler places each replica on a cluster and pool whose hardware matches using DRA and cel expressions, accounting capacity at the node level so it never overcommits.Autoscaling. Scale replicas through the standard Kubernetes scale subresource, sokubectl scale
and a KEDAScaledObject
work out of the box.Routing. One OpenAI-compatibleModelService
over many replicas, with weighted traffic for canary and A/B rollouts and optional fallback to an external provider.Caching. Stage model weights on cluster storage once, so serving pods read them locally instead of re-down on every start.
Modelplane stays neutral across models, engines, accelerators, clouds, and serving stacks, and composes them rather than replacing them.
We're building in the open# #
Modelplane is early, and we're releasing it at v0.1 so we can build it in the open, with the community. It's Apache 2, and we plan to donate it to a neutral open source foundation later this year, because a control plane that sits above the ecosystem should belong to the ecosystem.
If you're running inference yourself, planning to, or hosting accelerators (any vendor, any cloud, on-prem), we'd love for you to join us. Modelplane is live at
[github.com/modelplaneai/modelplane](https://github.com/modelplaneai/modelplane), with a getting-started
guide at [docs.modelplane.ai/getting-started](https://docs.modelplane.ai/getting-started/) that takes you from nothing to a
live endpoint in about 45 minutes. Come find us in [our Slack](https://slack.modelplane.ai),
open issues, contribute, and help us shape it.
Kubernetes became the standard control plane for compute. With Modelplane, we’re extending that same approach to AI inference across fleets of cloud, neocloud and datacenters.
Intelligence is becoming the most important thing we build, and it shouldn’t be something you can only rent. Everyone should be able to own theirs.
Help us build that future.