{"slug": "a-problem-with-rlvr-training-data", "title": "A problem with RLVR training data", "summary": "A former employee of an outsourced training provider for RLVR (Reinforcement Learning with Verifiable Rewards) data revealed that environments used to train AI models for computer use and MCP tasks were rushed, buggy, and failed to reflect real-world conditions, with workers encouraged to 'reward hack' around broken environments to maximize data volume. The source, who also worked in 3D synthetic data creation in 2023, said this rush to maximum volume is standard industry practice, and that such flawed training data is fed into models without clear indication of its synthetic nature.", "body_md": "Okay, so since I got laid off, I can actually explain a huge problem I saw from the inside with regard to industry practices on training models. I won't say specifically where I worked, but I worked at an outsource training provider that was focused on RLVR training data for computer use and mcp stuff.\nNearly all of the environments were rushed and vibecoded and failed to robustly reflect the real things they were based off. Both the scenario designers and models engaging with the scenarios for synthetic data gen were encouraged to work around the brokenness of said environments in order to get the procedurally verified reward confirmations. You know... they were *encouraged* to reward hack. On the human end, it was possible to mark an environment bugged, but greatly discouraged, as this reduced the volume of training data being produced. Instead, where possible, you were supposed to find the spots of the environment that weren't bugged and build scenarios around those, with the environment still bugged around you.\nFrom what I understand, this training data, with these problems, is fed into models without indication that its training/a fake environment other than the fact that names of softwares are changed to placeholders, but thing is, not *everything* is changed to placeholder names in these environments. The presence of placeholder/code names isn't universal and thus when a model accesses something in an environment that it shouldn't, the code names not being on it isn't a robust signal that that thing isn't part of the environment.\nI believe this *rush to maximum volume* is standard industry practice with these types of RLVR trainings as well, because maximizing volume has been an industry standard for years! It was the same standard applied to me and pushed on me despite my requests to slow down and focus on quality when I worked in 3d synthetic data creation as well, all the way back as far as 2023.", "url": "https://wpnews.pro/news/a-problem-with-rlvr-training-data", "canonical_source": "https://twitter.com/SkyeSharkie/status/2092122622834442581", "published_at": "2026-08-26 07:56:29+00:00", "updated_at": "2026-08-26 08:14:44.511065+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-ethics", "ai-policy"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/a-problem-with-rlvr-training-data", "markdown": "https://wpnews.pro/news/a-problem-with-rlvr-training-data.md", "text": "https://wpnews.pro/news/a-problem-with-rlvr-training-data.txt", "jsonld": "https://wpnews.pro/news/a-problem-with-rlvr-training-data.jsonld"}}