Rewards: How RTA is Changing AI Training

Researchers introduced Rank-Then-Act (RTA), a framework that trains AI using video-derived ordinal signals instead of traditional rewards, achieving state-of-the-art results on benchmarks like PyBoy and MetaWorld. RTA employs a Vision-Language Model and Group Relative Policy Optimization to learn from frame sequences, potentially enabling more human-like and adaptable AI learning.

Rewards: How RTA is Changing AI Training Rank-Then-Act could revolutionize AI learning by ditching traditional rewards for video-derived signals. Here's why it matters. Artificial intelligence /glossary/artificial-intelligence has a knack for shaking things up, and the Rank-Then-Act RTA framework might just be the next big disruptor. Forget the usual reward systems in AI training /glossary/training . RTA’s approach is all about learning from video demonstrations without dangling those carrot-and-stick incentives in front of the model. It's a bold new direction that could redefine how we teach machines to think and act. What’s in a Rank? The RTA framework uses a Vision- Language Model /glossary/language-model VLM to act as a kind of instructor, guiding AI policies based on video input. But here's the kicker: it doesn't rely on typical reward signals. Instead, RTA employs a unique method called Group Relative Policy Optimization /glossary/optimization GRPO to shuffle frame sequences and allow the model to learn from visual cues. It’s like teaching the AI to put a movie back in order by looking at the scenes, not just by knowing the script. Instead of using the VLM as a direct reward source, RTA introduces a correlation-centered reward function. By measuring the Spearman rank correlation between what the AI predicts and the actual sequence, this framework provides a stable, scalable learning signal. The jobs numbers tell one story, but AI, the paychecks, our reward structures, are changing. Benchmark /glossary/benchmark Beating So, how does RTA stack up? It’s been tested on both discrete control benchmarks, like PyBoy’s Catrap and Kirby, and on continuous control tasks such as PointMaze and MetaWorld. The results are impressive. RTA not only matches but often surpasses previous methods. It’s like the AI equivalent of going from zero to hero, showing that it can adapt across different environments without missing a beat. This isn't just some flashy tech gimmick either. By using video-derived ordinal signals, RTA opens up possibilities for broader applications and cross-task adaptability. Ask the workers, or in this case, the AI models, not the executives. They'll tell you this method delivers results. Why Should You Care? Here’s the big question: why does this matter? RTA pushes us to rethink how we train AI. Traditional methods hinge on explicit rewards, think of it like training a dog with treats. But what happens when the treats run out? RTA’s approach suggests that AI can learn just as effectively, if not more so, by understanding sequences and context. It’s a step towards more human-like learning. We don’t always need rewards to learn. sometimes, understanding the story is enough. Automation isn’t neutral. It has winners and losers, and RTA could be a breakthrough in finding the winners in AI development. In a world that often rushes to embrace automation, RTA asks us to pause and consider: are we teaching machines the right way to learn? Automation is reshaping the workforce, so why not reshape how we teach machines to think? Get AI news in your inbox Daily digest of what matters in AI. Key Terms Explained Artificial Intelligence /glossary/artificial-intelligence The science of creating machines that can perform tasks requiring human-like intelligence — reasoning, learning, perception, language understanding, and decision-making. Benchmark /glossary/benchmark A standardized test used to measure and compare AI model performance. Language Model /glossary/language-model An AI model that understands and generates human language. Optimization /glossary/optimization The process of finding the best set of model parameters by minimizing a loss function.