Apache Airflow 3.4.0: Task Loops, Dags in Any Language, and AI Agents Apache Airflow 3.4.0 shipped with Task Loops (AIP-111), a native bounded-iteration primitive that repeats a decorated task group up to a required max_iterations cap with an optional until predicate, plus the ability to author an entire Dag in TypeScript, Go, Java, or YAML instead of only Python. The release also matures the common.ai provider, which turns LLM calls and agents into ordinary Airflow tasks with human approval steps. Loops are handled at runtime only, so the Dag itself remains acyclic, and Airflow materializes each iteration lazily rather than creating a tail of skipped tasks. Apache Airflow 3.4.0: Task Loops, Dags in Any Language, and AI Agents We’re proud to announce the release of Apache Airflow 3.4.0 Where 3.3 gave your tasks memory and task bodies in multiple languages, 3.4 is about shape and reach . It brings Task Loops , a native loop primitive for workflows that repeat until they are done; the ability to author an entire Dag in TypeScript, Go, Java, or YAML rather than only Python; and a steadily maturing common.ai provider that turns LLM calls and agents into ordinary Airflow tasks, complete with human approval. 🎯 Release Highlights 📦 PyPI: https://pypi.org/project/apache-airflow/3.4.0/ https://pypi.org/project/apache-airflow/3.4.0/ 📚 Docs: https://airflow.apache.org/docs/apache-airflow/3.4.0/ https://airflow.apache.org/docs/apache-airflow/3.4.0/ 🛠️ Release Notes: https://airflow.apache.org/docs/apache-airflow/3.4.0/release notes.html https://airflow.apache.org/docs/apache-airflow/3.4.0/release notes.html 🐳 Docker Image: docker pull apache/airflow:3.4.0 🚏 Constraints: https://github.com/apache/airflow/tree/constraints-3.4.0 https://github.com/apache/airflow/tree/constraints-3.4.0 🔁 Task Loops AIP-111 : Workflows That Repeat Until They Are Done Some workflows do not have a fixed shape. You walk a tree of unknown depth, you refine an estimate until the error is small enough, or you run an agentic generate → score → refine loop until the output is good. Until now you reached for hard-coding a fixed depth or for Dags that trigger themselves. Airflow 3.4 makes bounded, sequential iteration a native concept. Before we go any further, it is worth addressing one thing: “How can you have loops in a Dag, given DAG stands for Directed Acyclic Graph”. The answer to that is by being very careful. Essentially, there is no dependency loop introduced, the tasks in the Dag still do form a cycle-free graph. The looping nature is handled at runtime only. Call .loop on a decorated task group to repeat it a number of times decided at runtime, where each iteration can build on the result of the last. It behaves like a do … while loop with a hard cap: python from airflow.sdk import dag, task, task group @dag tags= "example" def refine estimate : @task group def refine : @task def improve , loop : First pass starts from 1.0; later passes read the previous result. estimate = 1.0 if loop.index == 1 else loop.previous "estimate" return estimate + 2.0 / estimate / 2.0 @task def evaluate estimate : return {"estimate": estimate, "error": abs estimate estimate - 2.0 } evaluate improve def accurate enough , loop : return loop.result "error" < 0.000001 @task def finished : print "Refinement finished." refinement = refine.loop max iterations=10, until=accurate enough refinement finished refine estimate Key Capabilities - max iterations : a required upper bound, so a loop can never run away. - until : an optional predicate function that decides when to stop, or omit it entirely for a loop that runs a fixed number of times. Because the body runs before the check, it always runs at least once. - A loop context object inside the body and the predicate: loop.index the 1-based current pass , loop.previous the last iteration’s result , and loop.result for the predicate . - Lazy materialization : Airflow only creates the next iteration when the loop decides to continue, so you do not see a tail of skipped iterations. - Composable : loops work with .partial for body arguments, .override for group config, and even .expand to run mapped tasks inside a loop. A loop with a fixed count no until always runs exactly max iterations times and succeeds. If a loop reaches max iterations while its until predicate is still False , meaning it never converged, the final iteration is marked failed, so a failure to converge surfaces as a real failure. The Airflow UI gains dedicated loop views so you can see each iteration’s history at a glance. A few limits to know: loops cannot be nested, a loop cannot sit inside a mapped task group though mapped tasks inside loops are okay , and a whole loop cannot itself be mapped. For detailed usage, see the loops authoring guide https://airflow.apache.org/docs/apache-airflow/3.4.0/authoring-and-scheduling/loops.html . 🌐 Write Airflow Dags in TypeScript, Java and Go Airflow 3.3 let you write individual tasks in Java or Go while Python still owned the Dag. Airflow 3.4 takes the next step: you can define the entire Dag, including its schedule, tasks and dependencies, in TypeScript, Java or Go, with no Python file involved. All three SDKs are generally available. This is powered by AIP-85, the Dag Importer , a pluggable framework that teaches the Dag processor to read Dag sources beyond .py . Each format registers an importer. For a language SDK, the Dag processor runs the language’s runtime to read the Dags an artifact declares, and Airflow then treats them exactly like Python Dags. The same framework also powers an opt-in YAML importer. What’s supported Each SDK works in two modes, and one artifact can serve both: a Dag definition declares the whole Dag in the SDK’s language, which is what 3.4 adds, and a Python Dag with stub TaskHandler implements the @task.stub tasks of a Python Dag, including their TaskFlow arguments. A Dag definition supports Dag and task options under Airflow’s own names, one queue for all of a Dag’s tasks, TaskFlow inputs from task results and literals, order-only dependencies before / after , nested task groups, edge labels, if / else and switch / case branching, and triggering another Dag’s run. Task code can use Variables, Connections, XCom and the task state store. An artifact can hold several Dags, including Dags generated in a loop, and the Code view shows each Dag’s own source file. All of this works the same in TypeScript, Java and Go, and the TypeScript SDK adds human-in-the-loop tasks approval, hitl . The same Dag declared in three languages Each example below declares the same Dag, orders daily : - the extract task group counts orders for two regions, - summarize adds them up, - has orders runs load when there are orders, and alert when there are none, - pick report picks a full or a brief report, - trigger finance starts a run of the finance close Dag after load , on a labeled edge. The UI screenshot in each tab is of a separate real Dag authored with that SDK, not the orders daily snippet itself. TypeScript A handler takes one object of named arguments, and calling a task names its inputs. The calls you write are the dependencies, and the compiler checks each one. js import { Bundle, Dag, label, triggerDagRun } from "apache-airflow-ts-sdk"; const dag = new Dag "orders daily", { schedule: "@daily", tags: "orders" , queue: "typescript" } ; // A task group: its task ids are "extract.eu" and "extract.us". const extract = dag.taskGroup "extract" ; const eu = extract.task "eu", async : Promise