Sphere Retraction Normalizations
Researchers introduced p-SpheretNorm, a one-parameter family of angular retractions for spherical residual streams in deep neural networks, showing that the exponential map used in Geodesic Normalization (GeoNorm) is jus…