cd /news/generative-ai/rethinking-visual-reasoning-in-text-… · home › topics › generative-ai › article
[ARTICLE · art-170301] src=arxiv.org ↗ pub= topic=generative-ai verified=true sentiment=↑ positive

Rethinking Visual Reasoning in Text-to-Image Reward Modeling

A data-centric training intervention that converts existing visual reasoning datasets into preference pairs sharing the same image but differing in prompt produces a Pareto improvement in both visual understanding and preference prediction for text-to-image reward models, according to arXiv paper 2611.17585v1. The resulting reward model improves text-to-image generation on challenging compositional prompts in GenEval by an average of 4.0% for inference-time best-of-n selection and 5.5% for RL post-training, outperforming HPSv3, which was trained on 6x more preference pairs. The authors found that widely-used reward models perform poorly on targeted visual reasoning evaluations, and that preference fine-tuning erodes visual reasoning even from strong vision-language backbones, with continual-learning baselines failing to mitigate the steep tradeoff between visual reasoning and preference accuracy.

by read1 min views1 publishedNov 16, 2026

arXiv:2611.17585v1 Announce Type: new Abstract: Reward models are a core component of modern text-to-image systems, serving as the objective for inference-time selection and RL-based post-training. A key prerequisite is visual understanding: to judge prompt adherence and ground aesthetic preferences, a reward model must bind attributes, resolve relations, and count reliably. We find that widely-used reward models perform poorly on targeted visual reasoning evaluations. Even when starting from strong vision-language backbones, preference fine-tuning erodes these capabilities, and continual-learning baselines fail to mitigate a steep tradeoff between visual reasoning and preference accuracy. We propose a data-centric intervention that injects visual reasoning supervision into preference learning by flipping the pairing axis. To complement conventional human preference data (paired images under a shared prompt), we convert existing visual reasoning datasets into preference pairs that share the same image but differ in prompt, and train on the mixture. This yields a Pareto improvement in both visual understanding and preference prediction. Across inference-time best-of-n selection and RL post-training, the resulting reward model improves text-to-image generation on challenging compositional prompts (GenEval) by an average of 4.0% and 5.5% respectively, outperforming HPSv3 which was trained on 6x more preference pairs.

── more in #generative-ai 4 stories · sorted by recency
── more on @arxiv 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/rethinking-visual-re…] indexed:0 read:1min 2026-11-16 · —