The Silent Freeze: How Low-Precision Training Can Halt AI Progress
A new study reveals that low-precision training can cause AI model parameters to freeze when gradient updates fall below half the unit in the last place (ULP) of a weight, effectively halting learning. The phenomenon was…