Why do OpenAI's GPT-2 weights beat mine? Part four: digging into dropout
In a technical blog post, an independent researcher investigating why OpenAI's GPT-2 small weights outperform their own models on an instruction-fine-tuning test discovered that their fine-tuning conf…