E3DGS: Unified Geometric-Photometric Equivariance for 3D Gaussian Splatting via Color-as-Geometry Embedding

Researchers propose E3DGS, a unified geometric-photometric equivariant architecture for 3D Gaussian Splatting that treats color as a geometric entity via a matrix embedding, achieving SE(3) equivariance without Clebsch-Gordan tensor products. The method, validated on object vision and action-conditioned Gaussian world modeling, demonstrates strong robustness under camera-frame changes and improved data efficiency.

arXiv:2607.15536v1 Announce Type: new Abstract: 3D Gaussian Splatting 3DGS captures scenes by coupling explicit geometry position, covariance with view-dependent photometry Spherical Harmonics . However, building $\mathrm{SE} 3 $-equivariant architectures on these primitives presents a fundamental representation bottleneck. Color has been treated as a signal rather than a geometric entity, making it nontrivial to unify symmetry across geometry and appearance as the camera frame changes. While translations are handled by relative coordinates, rotations act heterogeneously across attributes: $\mu\mapsto R\mu$, $\Sigma\mapsto R\Sigma R^\top$, and $f \ell\mapsto D^\ell R f \ell$. This mismatch complicates strict equivariance, leading existing methods to either discard or flatten SH coefficients, thereby breaking symmetry. We propose a unified solution rooted in representation theory: for SH degrees $\ell\le2$, photometry is algebraically isomorphic to a rank-2 geometric tensor. We prove that the Wigner-$D$ action on these SH coefficients can be exactly reformulated as the conjugation action on $3\times3$ matrices. Leveraging this, we introduce the Unified Matrix Embedding, a lifting that maps all Gaussian attributes into a unified carrier space, $\mathfrak{gl} 3 $. Building on the "Color-as-Geometry" formulation, we present E3DGS, a rigid-body $\mathrm{SE} 3 $ equivariant architecture that processes 3D Gaussians without Clebsch-Gordan tensor products. Evaluations on object vision and action-conditioned Gaussian world modeling demonstrate that our unified approach yields strong robustness under camera-frame changes and improved data efficiency.