{"slug": "the-bitterest-lesson", "title": "The Bitterest Lesson", "summary": "OpenAI's InstructGPT/RLHF work showed that GPT-2-sized models (over 100x smaller than GPT-3) trained on the right task beat GPT-3, according to an essay by the author of The Bitterest Lesson. The essay argues that the priority order in machine learning is doing the right task > data > compute > algorithms, extending Rich Sutton's bitter lesson that compute beats algorithms. Scaling pre-training would need to reach roughly GPT-7 level to beat that baseline and GPT-9 to beat InstructGPT built on GPT-3, the essay states.", "body_md": "# The Bitterest Lesson\n\n**TL;DR: Compute drives progress in AI, but what good is progress if you are not doing the right task!**\n\nRich Sutton’s [bitter lesson](http://www.incompleteideas.net/IncIdeas/BitterLesson.html) states that compute beats algorithms.\n\nResearchers want to encode their clever ideas about intelligence into machines. Yet over and over, the approaches that win are the general ones which leverage more computation. Search beat hand-built chess knowledge. Neural networks beat hand-built vision features. The lesson is bitter because researchers love algorithms, yet cleverness matters less than scale.\n\nMy experience is that Sutton’s bitter lesson is the tip of an iceberg of bitterer lessons: beyond **compute** and **algorithms**, there’s **data** and even picking the **right task** to do ML on.\n\nThe bitterest lesson in ML is that **doing the right task > data > compute > algorithms.**\n\nSutton’s bitter lesson is easiest to see in games because there are two major differences to the real world: the **right** **task** is obvious (follow the rules to win or maximize score), and **data** can be endlessly generated through self-play (through compute). It makes sense that the next most important thing is **compute**.\n\nAt the end of the day, machine learning makes reward go up or loss go down. Someone still has to decide the objective to optimize though. Getting this right requires understanding the external system in which the model will operate. Without the right task, everything can work perfectly, with the most beautiful loss and scaling curves, but the model may still be useless!\n\nUnfortunately, ML research tends to attack these problems in the opposite order. Researchers love inventing algorithms. More recently, we have learned to love scaling curves. Meanwhile, data is messy. Choosing the right task often requires leaving the ML problem entirely to study users, products, organizations, or whatever part of the world is supposed to benefit.\n\nThis is not an argument against scale. Once the task and data are right, scale is incredible. It is an argument against treating scale as the be-all and end-all.\n\n## Bitter LLMs\n\n*When life gives you LLMs...*\n\nWe learned this lesson at OpenAI when making InstructGPT/RLHF: GPT-3 was an incredible model trained to predict the next token on internet text, but people wanted something that followed instructions more than they wanted a super-powered autocomplete. GPT-2-sized models (>100x smaller than GPT-3) trained *on the right task,*[1](https://www.completeskeptic.com/p/the-bitterest-lesson#footnote-1) even with the dumbest algorithm[2](https://www.completeskeptic.com/p/the-bitterest-lesson#footnote-2) and barely any compute, destroyed GPT-3.\n\n*Annotated from* *figure 31 from the InstructGPT paper*\n\nScaling pre-training would need to reach roughly GPT-7 level to beat even that baseline, and GPT-9 to beat InstructGPT built on GPT-3.[3](https://www.completeskeptic.com/p/the-bitterest-lesson#footnote-3)\n\n**You get what you optimize for and the bitterest lesson in ML is that the most important part of it isn’t ML at all.**\n\n∵ ⩆\n\n⩆ ∵", "url": "https://wpnews.pro/news/the-bitterest-lesson", "canonical_source": "https://typesafe.ai/blog/bitterest-lesson", "published_at": "2026-09-18 03:16:03+00:00", "updated_at": "2026-09-18 03:25:04.270084+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research"], "entities": ["Rich Sutton", "OpenAI", "InstructGPT", "RLHF", "GPT-3", "GPT-2", "GPT-7", "GPT-9"], "alternates": {"html": "https://wpnews.pro/news/the-bitterest-lesson", "markdown": "https://wpnews.pro/news/the-bitterest-lesson.md", "text": "https://wpnews.pro/news/the-bitterest-lesson.txt", "jsonld": "https://wpnews.pro/news/the-bitterest-lesson.jsonld"}}