cd/entity/Switch Transformers· home› entities› Switch Transformers
grep -l @switch transformers /news/*.json | wc -l → 1

Switch Transformers

mentions 1 type Person feed RSS

// recent coverage 1 mentions

19:00
2026-08-14
gilesthomas.com
artificial-intelligence

Why do OpenAI's GPT-2 weights beat mine? Part two: IFT dropout

OpenAI's GPT-2 medium weights achieved the highest IFT score of 42.43 with only 2 IFT epochs, while a JAX model with no dropout and no MHA bias scored 21.45 with 5 epochs, and a JAX model with dropout…

// co-occurs with top 6 entities
OpenAI 1 GPT-2 1 JAX 1 Hugging Face 1 Raffel et al. 1 Srivastava et al. 1
// topics top 3 topics
artificial intelligence 1 machine learning 1 large language models 1