{"slug": "building-self-improving-rl-environments-that-scale-with-model", "title": "Building Self-Improving RL Environments That Scale With Model", "summary": "Rui and Michael built EdotEnv, a suite of self-improving reinforcement learning environments that scale with model capability, designed to benchmark LLMs on quant research tasks such as feature engineering, portfolio design, and backtesting. Testing state-of-the-art models revealed consistent weaknesses, including shallow search strategies and a tendency to give up when losing money. The environments use real market data and natural noise floors to foster transferable research skills, and the team has open-sourced a sample feature engineering task repo at github.com/MMcollab-dotcom/feature-engineering.", "body_md": "# Building Self-Improving RL Environments That Scale With Model\n\nRui and Michael built this after seeing firsthand how quant markets evolve: profitable strategies decay as more capital chases them, making new alpha increasingly difficult to extract. That's exactly the property we want in a benchmark.\n\nTheir environments give LLMs a realistic research pipeline: build predictive features, design portfolios, backtest strategies, and adapt to regime changes — all evaluated on out-of-sample data. Each task comes with purpose-built tools: a backtesting engine for testing features historically, an execution tool for live strategy deployment, and a final scoring mechanism that isolates specific skills like feature engineering.\n\nRunning SOTA models through these envs revealed consistent weaknesses: models prefer shallow broad searches over deep iteration, higher reasoning doesn't translate to better performance, and agents often give up when losing money instead of adapting their approach. The reward structure keeps evaluation honest — no LLM judges or human gatekeepers needed.\n\nThe key insight is that quant research is pure applied ML: long-horizon planning, continual learning, and iterative hypothesis testing. By embedding real market data and natural noise floors, EdotEnv creates transferable research skills rather than benchmark-specific tricks.\n\nThey've open-sourced a sample task repo for feature engineering at `github.com/MMcollab-dotcom/feature-engineering`\n\n, and plan to offer continuously evolving environments to AI labs focused on ML modeling, long-term planning, and autonomous research workflows.\n\nIf you're training agents on research tasks or building eval harnesses, this is worth examining — especially the reward design that stays meaningful even as capabilities advance.\n\n[Citadel Acquires Situational Awareness After AI Division Bleeds 5d ago](/en/news/4420/)\n\n[Next Maple-Preview: 20B MoE Hits 120 tok/s on iPhone →](/en/news/5019/)\n\n[a practical ChatGPT prompt guide](http://154.12.95.112/), with plenty of directly applicable cases.", "url": "https://wpnews.pro/news/building-self-improving-rl-environments-that-scale-with-model", "canonical_source": "https://promptcube3.com/en/news/5023/", "published_at": "2026-08-05 00:43:01+00:00", "updated_at": "2026-08-05 00:46:59.276701+00:00", "lang": "en", "topics": ["machine-learning", "ai-research", "ai-tools"], "entities": ["Rui", "Michael", "EdotEnv", "github.com/MMcollab-dotcom/feature-engineering"], "alternates": {"html": "https://wpnews.pro/news/building-self-improving-rl-environments-that-scale-with-model", "markdown": "https://wpnews.pro/news/building-self-improving-rl-environments-that-scale-with-model.md", "text": "https://wpnews.pro/news/building-self-improving-rl-environments-that-scale-with-model.txt", "jsonld": "https://wpnews.pro/news/building-self-improving-rl-environments-that-scale-with-model.jsonld"}}