EarthRecord: From Pixels to Land Change with Multimodal Agents over Four Decades of Satellite Imagery
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
Multimodal agents can now analyze four decades of raw satellite imagery to answer land-change questions, but evaluations of 12 agents reveal they often rely on prior knowledge or altered data rather than actual changes on the ground. This matters because it highlights a significant limitation in current LLM capabilities when applied to complex Earth observation tasks. Shipping LLMs for such tasks may require additional development to ensure they accurately map changes from raw imagery.
EarthRecord tests 12 multimodal agents on 238 land-change questions over 56 sites using four decades of raw Landsat, MODIS, VIIRS, and Sentinel-2 data, requiring agents to handle cloud masking and cross-sensor harmonization themselves. The key failure mode is not tool use but grounding: agents can complete many workflows, yet when one satellite stream is altered, about half of their correct answers follow the corrupted data rather than real ground change. For production Earth-observation agents, this means end-to-end completion metrics are unsafe; you need sensor-perturbation and cross-instrument consistency tests before trusting outputs as physical change.