Moving Your Local Airflow to GCP for under $150 a month A technical team migrated its self-managed Apache Airflow orchestration from on-prem Linux boxes to Google Cloud Platform, building a self-managed Airflow VM plus a config service for under $150 a month after finding Google's managed Cloud Composer would cost roughly $350 a month. The design keeps Airflow from calling the config service directly, sharing only a versioned GCS bucket that doubles as change history, and secures the config app behind an HTTPS load balancer with Identity-Aware Proxy. My team runs ML training and prediction pipelines on Vertex AI. For a long time, the thing telling those pipelines when to run was Apache Airflow, installed by hand on a few on-prem Linux boxes. From there it orchestrated our hybrid cloud setup from one place. Next to Airflow sat our own configuration management system. It lets people change a pipeline's settings on demand, and every change is tracked and logged over time, so if a model behaves differently on Tuesday, we can see what changed on Monday. Both of these lived on Linux boxes we maintained ourselves, and those boxes had to go: our organization is phasing out its on-prem setup and moving fully cloud native. We are in the middle of that move right now. The on-prem pipelines are moving out to different clouds, and our own pipelines and compute are moving into GCP. With the work landing in GCP, putting the orchestrator there too was the natural answer. The goal was to make that move without ending up any less safe or less reliable than what we had. In this post I want to share how we did it, chapter by chapter. The code is in the self-managed-airflow-on-gcp repo, and everything below is a trimmed-down, working version of it. If you ask "how do I run Airflow on GCP?", the answer is Cloud Composer. It is managed Airflow: Google runs the scheduler, the database and the workers, and you drop DAGs into a bucket. We built it first. We stood up a Composer 3 environment with Terraform, pointed our DAGs at it, and it worked. The cost was the problem. With the pricing calculator, a small Composer environment came to about $350 a month . Composer 3 bills for compute units every hour the environment is alive, whether a DAG runs or not. Our pipelines do not need a cluster awake around the clock to schedule a handful of jobs, because the heavy work happens in Vertex AI anyway. My search was not done there. Our Airflow does not do the heavy work. It decides when work happens and hands it to Vertex AI or BigQuery. A scheduler like that fits on one small VM. So we built the same thing a second time: The estimate for this came in under $150 a month . About $20 of that is the HTTPS Load Balancer in front of our config app GCP charges a flat $0.025/hour for the first five forwarding rules, roughly $18/month, plus data . The rest is the VM, its disk, and Cloud NAT. That number is not the whole cost. With Composer, Google upgrades Airflow and patches the machine. On a VM, we do. We had both versions running side by side, and the team picked the self-managed one: we are a technical team that already ran Airflow ourselves, so the extra work was work we knew. Cheaper only counts if it is also safe and reliable, so the rest of this post is about how we got there. There are two components we deploy: config hq airflow-vm The part I like most in this design is that Airflow never calls config hq . The two only share a bucket. Our org policy disables Cloud Run's default run.app URL, so the service is deployed with default uri disabled = true and there is no hostname for a DAG to call anyway. Instead, config hq writes every save as a new object named config/