AI training has become a pretty strange market.
A few years ago, “AI training work” mostly sounded like labeling cats, cars, traffic lights, and whatever else somebody could put into a Mechanical Turk task.
Now we have people with actual software, science, finance, law, and engineering backgrounds being paid to create problems, judge answers, write rubrics, test agents, and figure out exactly how a model failed.
So, naturally, after spending billions trying to automate humans, we’ve created another industry where humans are hired to explain to the machines why they’re wrong.
Grand View Research currently estimates the AI training-dataset market at around $3.9B for 2026, with more than 20% annual growth projected through 2033.
Scale AI also says it signed well over $1B in new business last year, and its language has noticeably shifted toward evaluations, realistic RL environments, reliability, and expert data rather than just “give us more labels.”
That change is the part I find interesting.
As models get better, I suspect the valuable work moves away from simply producing answers and toward producing good tests: weird failures, reproducible bugs, adversarial cases, verifiers, repair tasks, and evidence that something was actually fixed.
I’ve been working on this from the software side with a project called FaultWright — previously FaultFoundry.
The core system is finished now. It creates controlled software failures, verifies that the intended failure actually happened, packages the evidence into a repair task, and then checks whether a model really repaired the original behavior.
I’m currently looking for a collaborator more on the deployment/customer/business side rather than someone to build the code with me.
But I’m also curious what people here think about the broader direction of this market.
Does human AI-training work eventually shrink as models improve, or does it just move upward into harder evaluation and domain-expert work?
And if synthetic data keeps increasing, who tests the synthetic data generator so we don’t end up with AI grading AI-generated homework using an AI-generated answer key?
I have a feeling the boring-looking evaluation layer may end up being considerably more important than the flashy model demos.