World Models

A deep learning practitioner argues that training world models on sanitized data is counterproductive, as exposing models to unfiltered reality at scale is necessary for unexpected capabilities to emerge.

A world model trained on a sanitized world is a contradiction. My deep learning career taught me: at low data, curate carefully. At scale, stop protecting models from reality. That is where unexpected capabilities emerge. https://t.co/1fO6faqNZV