Web agents' world models learn to predict, not to choose: a rebuilt training objective lifts task success

A paper posted 2 September by a team including Kelvin Li, Leonid Karlinsky, Rogerio Feris, Trevor Darrell and Roei Herzig argues that LLM web agent pipelines train world models with supervised next-state prediction to reproduce pages faithfully, while the process reward model ranker actually needs predictions that discriminate between alternative actions. Their predicted-state matching objective trains the world model to distinguish the true resulting state from states other actions would reach, using a branching dataset built from WebArena Go-Browse trajectories. The method beats supervised next-state prediction on their held-out benchmark, improves process-reward action ranking on WebPRMBench, and raises end-to-end task success on WebArena-Lite, drawing researcher endorsements on SciRate within hours of posting.

LLM web agents increasingly pick actions by sampling candidate moves, predicting the resulting page states with a world model, and letting a process reward model rank the options. A paper posted 2 September by a team including Kelvin Li, Leonid Karlinsky, Rogerio Feris, Trevor Darrell and Roei Herzig argues that pipeline hides a flaw: world models are trained with supervised next-state prediction to reproduce pages faithfully, but the ranker actually needs predictions that discriminate between alternative actions. Their predicted-state matching objective trains the world model to tell the true resulting state apart from states other actions would reach, using a branching dataset built from WebArena Go-Browse trajectories. It beats supervised next-state prediction on their held-out benchmark, improves process-reward action ranking on WebPRMBench, and raises end-to-end task success on WebArena-Lite. The paper drew researcher endorsements on SciRate within hours of posting.