cd/entity/Mixture of Softmaxes· home› entities› Mixture of Softmaxes
grep -l @mixture of softmaxes /news/*.json | wc -l → 1

Mixture of Softmaxes

mentions 1 type Person feed RSS

// recent coverage 1 mentions

17:32
2026-08-12
dev.to
machine-learning

The Softmax Bottleneck and What It Limits

A developer explains the softmax bottleneck in language models, showing that the rank of the log-probability matrix a model can produce is limited by the hidden dimension plus one, which caps the dist…

// co-occurs with top 1 entities
Yang et al. 1
// topics top 3 topics
machine learning 1 large language models 1 ai research 1