{"slug": "generative-value-classifier", "title": "Generative Value Classifier", "summary": "A new arXiv paper (2610.22497v1) proposes a \"generative value classifier\" that reframes Q-learning as predicting how likely an action is under each possible return rather than directly predicting the return of a state-action pair. The authors report the return-conditioned critic, which can share the actor's architecture and is supervised by every action plus a squared-error loss anchoring Q-values to the bootstrapped target, outperforms leading discriminative methods including distributional RL on both offline and online reinforcement learning tasks.", "body_md": "arXiv:2610.22497v1 Announce Type: new \nAbstract: Q-learning is premised on learning a function to predict the value of a state-action pair, with the goal of using such a function to guide policy improvement. Most approaches learn this function from scratch using scalar or distributional return targets. This discriminative approach leaves another source of supervision for learning representations unused: the actions themselves. Rather than directly predict the return of a state-action pair, we ask the reverse question: how likely is the action under each possible return? This perspective provides a critic whose generative component is a return-conditioned policy, which can share the actor's architecture. Training this policy supervises the critic with every action rather than with returns alone, and a squared-error loss anchors the critic's Q-value to the bootstrapped target. We evaluate our approach on both offline and online RL tasks and show gains over leading discriminative methods, including distributional RL methods.", "url": "https://wpnews.pro/news/generative-value-classifier", "canonical_source": "https://arxiv.org/abs/2610.22497", "published_at": "2026-10-19 04:00:00+00:00", "updated_at": "2026-10-19 04:19:27.670680+00:00", "lang": "en", "topics": ["machine-learning", "artificial-intelligence", "ai-research"], "entities": ["arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/generative-value-classifier", "markdown": "https://wpnews.pro/news/generative-value-classifier.md", "text": "https://wpnews.pro/news/generative-value-classifier.txt", "jsonld": "https://wpnews.pro/news/generative-value-classifier.jsonld"}}