# Your ML Pipeline Isn't a DAG of Tasks. It's a Graph of Assets > Source: > Published: 2026-07-29 13:01:12+00:00 TL;DR —Airflow-style orchestration models ML pipelines as sequences of tasks that ran successfully, which is the wrong unit of abstraction for retraining, drift-triggered runs, and versioned artifacts. Asset-centric orchestration — treating datasets, features, and models as first-class, versioned objects with freshness contracts — matches how ML dependency graphs actually behave. The fix isn't picking Dagster over Airflow; it's changing what you orchestrate around. Every orchestrator you've used was designed to answer one question: did task B run after task A succeeded? Airflow made that question extremely reliable. Cron made it explicit. Dagster, Prefect, and their peers all inherited the same core primitive: a directed graph of tasks, triggered on a schedule or a sensor, executed in order. That primitive is correct for ETL. It is subtly wrong for ML. Task-based orchestration assumes two things. First, that "ran successfully" is the meaningful state to track. Second, that dependencies are about execution order, not about data state. Both assumptions hold for a nightly report: extract, transform, load, done. Nobody asks whether yesterday's extract is "stale" in some semantic sense — it either ran or it didn't. ML pipelines don't work that way. A feature table isn't just "computed" or "not computed" — it has a freshness contract relative to the model that consumes it. A model isn't just "trained" — it's trained against a specific snapshot of data, and if that snapshot silently drifts, the model is stale even though every task in the DAG shows a green checkmark. The task ran. The asset it produced is no longer trustworthy. Task-based orchestrators have no vocabulary for that gap, because they were never tracking the asset in the first place — they were tracking the run. The clearest symptom shows up in retraining logic. In a task-based world, retraining is usually bolted on as a cron job: retrain every Monday, or every N hours, regardless of whether anything meaningful changed. Teams then layer drift detectors on top, which fire alerts that a human has to translate back into a manual DAG trigger. The orchestrator and the actual reason for retraining live in two different systems that don't speak to each other. The underlying dependency isn't "time has passed." It's "the input distribution moved enough that the current model is no longer a valid function of it." That's an asset-state dependency, not a schedule. Modeling it as a cron job is a workaround for not having a graph that understands data state as a first-class node. Dagster's software-defined assets are the clearest mainstream implementation of the alternative model, but the idea matters more than the tool. Instead of orchestrating tasks that happen to produce data, you declare the data objects themselves — a feature table, an embedding index, a trained model — as nodes in the graph, each with a materialization function and, critically, a freshness policy attached to the asset rather than to a schedule. This sounds like a naming difference. It isn't. Once the asset is the unit of orchestration, you get properties that are painful to bolt onto task-based systems after the fact: **Staleness becomes queryable.** You can ask "is this model's training data older than its freshness contract allows" as a graph query, not a log-scraping exercise across run history. **Partitions become native.** ML data is almost always partitioned — by day, by customer segment, by data source — and backfills need to operate on specific partitions of an asset, not on entire pipeline runs. Asset graphs track partition-level materialization state; task graphs mostly don't, which is why backfilling three days of a feature table in Airflow often means re-running logic that was never designed to be partition-aware. **Lineage falls out for free.** If the model asset declares the feature asset as an upstream dependency, and the feature asset declares the raw ingestion asset as its upstream, you get artifact-level lineage without a separate metadata-tracking layer bolted on top of task logs. **Retraining triggers become declarative.** "Rematerialize this model when its upstream feature asset's freshness policy is violated" is a graph rule, not a cron expression plus an out-of-band alert. Pipelines-as-code was supposed to fix orchestration by making the graph reviewable, versioned, and testable like any other software artifact. It delivered on that promise for the mechanics of execution: retries, backoff, resource allocation, secrets, scheduling syntax. What it didn't fix is the abstraction underneath the code. You can write a beautifully tested, version-controlled Airflow DAG that still can't tell you whether the model it produced is semantically fresh, because "fresh" was never a concept the DAG was built to hold. Code quality and abstraction correctness are different axes. A well-engineered task graph is still a task graph. This is also why switching tools rarely fixes the underlying problem on its own. Teams that migrate from Airflow to Dagster expecting the asset model to appear automatically are often disappointed, because you can absolutely write Dagster code that models tasks instead of assets — the framework offers the better primitive, it doesn't force you to use it. The discipline required is explicitly identifying which nodes in your pipeline are data objects with a lifecycle worth tracking, versus which nodes are pure execution steps with no independent state worth remembering. Not everything needs to be an asset. A logging step, a notification, a cleanup job — those are genuinely tasks, and forcing them into an asset model adds ceremony without benefit. The useful design rule is narrower than "use asset-based orchestration everywhere." It's this: anything downstream consumers rely on for correctness — features, embeddings, model checkpoints, evaluation datasets — should be modeled as a versioned, freshness-aware asset, regardless of which orchestrator executes the code. Anything that exists purely to move the pipeline forward — retries, glue steps, orchestration control flow — can stay a task, because tracking its "state" independent of execution has no payoff. Airflow can be made to approximate this with datasets and asset-aware scheduling features added in recent versions, and plenty of teams get real value out of layering an external metadata or feature-store system on top of a pure task DAG. The tool matters less than the discipline of drawing that line explicitly, in code, before you build the graph — not discovering it three months later when someone asks why a model trained on three-week-old data passed every task in the pipeline.