Any 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale

A research paper titled "Any 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale" argues that scene-level 3D generation has been dominated by 2D multi-view or video diffusion models that operate in a 2D latent space, which the authors say introduces two fundamental issues, the first being that representing 3D scenes via 2D views leads to significant representation redundancy. The paper proposes a 3D-grounded representation that encodes a scene in roughly 1K tokens. The work matters to researchers building scalable 3D scene generation systems because it targets the inefficiency of 2D-latent approaches.

Scene-level 3D generation has long been dominated by 2D multi-view or video diffusion models. Typically, these 2D-based approaches perform scene generation in a 2D latent space, which introduces two fundamental issues: i representing 3D scenes via 2D views leads to significant representation redun