For about a decade, "get more data" meant "crawl more pages." That instinct is quietly expiring, and most engineering teams haven't updated their mental model yet. Epoch AI's estimate is the number worth internalizing: the effective stock of quality- and repetition-adjusted public human text is roughly 300 trillion tokens, and frontier developers could consume it somewhere between 2026 and 2032. The subtlety matters more than the headline. Nobody is claiming humans stop writing. The claim is that effective stock — deduplicated, quality-filtered, not already seen fifteen times — is finite, and that additional passes over the same corpus don't create new human observations. A model can reread every Stack Overflow answer a hundred times and still not learn how your radiology department actually annotates a borderline nodule.
Meanwhile the legal side moved from theory to invoice. Anthropic's $1.5B settlement with authors got final court approval in July 2026 — the largest copyright settlement in U.S. history. The major music labels converted their suits against Suno and Udio into licensing deals. And as of August 2026, the EU AI Act's high-risk obligations are in full enforcement, which means Article 10 documentation: training data characteristics, sources, and rights clearances, on the record, for regulated deployments. Article 10 also treats synthetic data as legally equivalent to real data — same governance, same documentation, same penalties. You do not escape provenance requirements by generating your way around them.
Put those together and you get a shift that's structural, not cyclical: training data stops being something you acquire and becomes something you operate. A supply chain, with sourcing, QA, provenance records, and unit economics.
Here's what that actually looks like in practice, and what to do about it if you're the one who owns the pipeline.
After watching a lot of teams go through this, the shape that survives contact with production looks roughly like this:
Tier 1 — Licensed and public corpora. Cheap per token, broad, no domain edge. This gets you general competence. It is also the tier where everyone else is buying, so it produces zero differentiation. Treat it as commodity infrastructure.
Tier 2 — Synthetic augmentation. Excellent multiplier, terrible primary source. The industry pattern in 2026 is synthetic plus licensed, not synthetic instead of licensed. Over a third of Fortune 500 firms use synthetic data in production, and the ones doing it well use it to expand coverage around a real seed — perturbing known-good examples, generating edge cases in a distribution you've already mapped. Generating from an unanchored prompt just re-samples the generator's priors, which is how you get a model that is confidently average at everything.
Tier 3 — Commissioned human data. Expensive per unit, and the only tier that produces genuine differentiation. This is domain experts producing observations that do not exist anywhere on the public web: your failure taxonomies, your adjudicated edge cases, your reasoning traces through problems that took a human eleven years of practice to solve in four minutes.
The mistake I see most often is teams spending Tier 1 money on Tier 3 problems. They scrape 400GB of loosely-related domain text, fine-tune, get a 2% eval bump, and conclude that fine-tuning "doesn't work for us." Then someone commissions 3,000 expert-written examples with adjudicated disagreements and gets a 15-point jump on the metric that actually matters. The scarce input was never volume. It was judgment.
A concrete example. A team building a code review assistant had two million public PR diffs. Their model was good at flagging unused imports and bad at everything that mattered. Why? Public PR data is overwhelmingly trivial changes, because trivial changes are what get merged fast and openly. The hard reviews — the ones involving a concurrency bug that only manifests under a specific retry policy — happen in private repos, get discussed in a call, and leave almost no textual trace.
You cannot crawl your way to that data. It has to be produced. Someone senior has to sit down, look at a diff, and write out the reasoning: what they checked, what they ruled out, what made them uneasy. That output is the asset. It is also the reason reasoning and human-feedback data has become its own procurement category rather than a line item under "annotation" — preference rankings, agent trajectory corrections, tool-use validation, and SFT curation are all fundamentally the same act of capturing expert judgment in a form a model can learn from.
The same logic applies outside code. Clinical notes, claims adjudication, ADAS edge cases in rain at dusk — the valuable examples are, by construction, rare. Which means the sampling strategy matters more than the crawl budget. If you are still measuring your data program in gigabytes, you are measuring the wrong axis.
Whether you produce data in-house or commission it, these four things determine whether you get an asset or a liability.
1. A written spec with a disagreement protocol. Not "label whether the response is helpful." Define helpful, define the boundary cases, and — most importantly — define what happens when two qualified annotators disagree. In expert domains, inter-annotator disagreement is signal, not noise. It usually means your spec has an unresolved ambiguity, and finding it early is cheaper than discovering it in a 50,000-example delivery.
2. Multi-pass QA with real numbers. Single-pass labeling on expert tasks produces error rates that quietly cap your model's ceiling. Serious pipelines run independent passes plus adjudication and report per-batch pass rates. Anything above roughly 3-5% error on a nuanced task will show up in your evals as a mysterious plateau you'll spend a quarter chasing. Ask for the QA methodology and the pass-rate numbers before you sign anything; if a vendor can't produce them, that's your answer. It's the reason we publish our own triple-pass QA process and 99%+ pass rates rather than quoting a per-label price and hoping nobody asks.
3. Provenance metadata from day one. Source, license, generation method, annotator qualification tier, date, and — for synthetic — the exact seed and generator version. Under Article 10 you may have to produce this. Even without regulation, you will need it the first time you want to retrain while excluding one source, and retrofitting provenance onto a two-year-old corpus is a genuinely miserable engineering project.
4. A held-out set built by different people than the training set. Obvious, routinely violated. If the same team produced both, you are measuring annotation-style agreement, not capability.
Commissioned expert data is expensive. That is the whole point — the price reflects the scarcity of the judgment, and the judgment is what you're buying. But "expensive" has a wide range. A large share of the cost in expert data is coordination: recruiting qualified people, training them on the spec, running QA, handling adjudication. Where that operational layer sits changes the economics substantially without changing the qualification bar for the person doing the actual work.
That's the structural reason bilingual SME-led teams in places like Vietnam have taken a growing share of this market — typically 30-40% below US-based vendors for the same qualification tier and QA rigor. Domain expertise is globally distributed even when data budgets aren't. The thing to hold constant when comparing options isn't headcount cost, it's the QA methodology and the qualification of the person making the judgment call. If you're mapping out what a domain-specific data collection program would actually require, that's the axis worth pressure-testing.
The scrape-first era worked because the web was an accidental gift: a vast, free, pre-existing corpus nobody had to commission. That gift has been mostly unwrapped. What comes next looks less like crawling and more like manufacturing — specs, sourcing, QA gates, provenance records, cost-per-unit tracked against downstream eval lift.
That's less romantic than "we trained on the internet." It's also a discipline software teams are already good at. You've built supply chains before; you just called them build pipelines.
The practical takeaway: stop asking how much data you can get, and start asking which 5,000 examples would change your model's behavior on the failure mode that's actually costing you. Then figure out who has the expertise to produce them, and what it takes to verify they did it right.
I work at SyncSoft.AI, where we build data collection, annotation, human-feedback, and evaluation pipelines for AI teams. If you're working through any of this — especially the "which 5,000 examples" question — I'm happy to talk it through, no pitch required.