Xavier initialization

Researchers propose S-GAI, a spectral geometry-aware initialization framework for sigmoidal MLPs that encodes dataset geometry into network weights. Using SVD to extract class-wise spectral features, …

Neural network weights are initialized by dividing by the square root of the number of inputs to prevent the network from being dead before training begins. This technique, known as Xavier initializat…