Summary of Patrick O’Shaughnessy’s podcast with Sam Altman. Disclaimers:
- I haven’t started at OpenAI as this post is written.
- Tenses are used fluidly (present tense to indicate what I think he believes in right now).
- No AI used (for better or worse 🤷♂️).
Blue texts are my thoughts or interpretations.
“The past year was indeed very tough, and that’s partly my responsibility. But the year ahead could be our best twelve months”
** What went wrong:** Sam said that they spreaded themselves too thin and had to refocus on delivering the best and most cost-effective intelligence in order to empower people to build things. They originally made the bet that even if revenue growth took longer than expected to materialize, they could still monetize the GPUs;
** What coming ahead:** Better quality of models and better products built around them. They will continue to focus on (1) training great models that can be used in different ways that can generate economic value; (2) producing or partnering to manufacture the chips and systems, and finding a place to house them; and (3) building robots that can automate this entire process to continuously drive down costs across the entire supply chain.
Fundamentally, our business is about selling AI that enables people to build extraordinary products and services for each other using these components. … Building every vertical application ourselves, trying to compete with every venture and every company? We have zero interest in that. We genuinely just want to provide the platform. –– Sam Altman
Convictions and Worries
** Conviction on securing compute and building datacenters:** Sam expected the exponential model improvement especially after seeing GPT-4, where he believed that reasoning once solved will lead to agents that do huge economic work. He also expected huge demands and regardless of how efficient the models would become, compute would remain critical. That led to OpenAI trying to talk to cloud providers, chip manufacturers, and energy suppliers to secure unprecedentedly huge compute capacity (which many touted at that time as insane and reckless). They only got two yes––from Microsoft and Oracle––but that allowed them to move forward.
Interesting thing I’ve learned here is that data centers can now be built in deserts. This is possible because modern data centers use closed-loop water system and only as much water as an office building. They can also be powered by nuclear or solar energy instead of fossil fuels. This totally surprised me because all the water/energy arguments against building data centers can now be refuted.
OpenAI’s goal is to serve users with the best intelligence-price trade-offs, and Sam seemed unperturbed by the distillation attacks, saying that people are just gonna do what they wanna do and OpenAI will beat them at their own game. To Sam, the usage of their models is so high that even if model training (including R&D) is expensive, OpenAI serve enough customers that they can eat up the training cost and still make profits.
** His top worry is about AI safety:** He was shooked by the
Huggingface incident, where the OpenAI’s internal models hacked out of the sandbox and Huggingface platforms to find answers to cheat on ExploitGym. As a result, the team had d training and is trying to figure out ways of securing sandboxing; in the long-term, he thinks that we’d need to pace AI development to buy time for society to harden infrastructure around these unintended behaviors from capability progress. He is also aware of how these might come across as regulatory capature or collusion among frontier labs, so he urges for solutions for this.
Later on, he’s also worried about concentration of power––particularly terrified by a world where only a small group of people have access to AI and make decisions around it. He warned about not falling into the trap of AI safety and understandable fears, and growing up where internet has no rules, he wants to preserve that spirit with AI technology and let people self-determine the future.
Mission and Bottlenecks
** OpenAI's mission and "AGI being a genie that can grant wishes"**: Sam thinks that AI will become a genie that can grant wishes, and he hopes that OpenAI will continue to
- Get them into everyone’s hands and make people’s lives better than they otherwise they have been.
- Make sure people maintain control and agency.
He thinks that it will lead to more jobs and help more creative endeavors.
However, he doesn’t think that AGI is here yet because right now it cannot do tall orders such as “curing cancer” and more importantly, it suffers from limitations such as not being able to continuously learn. However, he sympathized with people who claim that AGI is already here as in this worldview, we already have a machinery that can produce better AI models where we are learning new science about them. In my opinion, I think this is not very well elaborated by Sam, as I think
When asked about what people in 2019 would think about GPT 5.6, he said that they would agree it is AGI, but they would be overestimating the economic impacts especially with respect to AI taking jobs. He gave several potential reasons: AI is jagged so still complementary to humans, and humans enjoy interacting with humans. Citing examples about AI-generated arts, he believes that there’s value to person behind the creation process; and in business, that means we would prefer somebody that can be held accountable for and not an AI CEO.
** Bottleneck at frontier labs: data, compute, ideas, or talent?** He thinks that it has been a cycle, and now it’s back to the era of research ideas again for the last six months. As RSI is approaching, he expects that the workflow of the researchers will change drastically like how software engineers don’t write code any more in the traditional sense, but it’s still important to have researchers to tell computers what they want.
Less-Organized Notes
Q: What’s it like becoming a dad and having growing kids in this era?
A: I think I have the best, most interesting job in the world, and it is still a very distant second to having kids.
I find the second half of the interview becomes really hard to summarized and grouped as above. It becomes more conversational. So here I try to use bullet points to capture interesting takes:
On being early: Best investment is made when it is not popular or following what other people are already doing.Honestly, I find this a bit hard to generalize because looking at AI progress, in my opinion you definitely should work on LLMs, which is a trendy topic. However, you should come in with your personal angle of attack where the reward-to-effort (or impact-to-effort) ratio is high. - Adaptability of humans. He thinks people adapt very quickly, citing how in pandemic, lockdowns happen quickly and people adapt accordingly.This probably suggests that when singularity happens, even if chaos ensue, it will not last long and people can still adapt and flourish just like during pandemic. - Evals that matters. Evals that matters is whether AI is useful to people. Right now, people approximate this via revenue, GDP, etc., and OpenAI has teams figuring out how to evaluate on superintelligent models. - Personal-AI. His vision of how future AI-human interactions may look like is that AI oversees everything the user does and continues to think when users are sleeping so once users wake up, their AI agent would present them new ideas or to-dos. This needs a lot of compute, and he believes that everyone like him would be willing to pay a lot for that always-on AI. - Taste. He believes it is still hard for future models to develop good taste. - Robotics. He thinks that ChatGPT moments for robotics would happen in like next two to three years, and that moment would look like “you type a command and a robot can do something crazy, and you can like watch it even if you are not physically there.” - ChatGPT release. Interestingly, OpenAI already had GPT-4 ready when they released ChatGPT, which they used a weaker version of the model (GPT-3.5) to avoid safety risks such as misuse.
Note that ChatGPT was released Nov 2022 and GPT-4 was released on March 2023.</br> The whole idea of developing ChatGPT was from seeing how users organically like to interact with AI in a chatbot fashion, and they were amazed when they built that interface and interacted with GPT-4 internally.
We decided to build a proper chatbot. We started working on it, completed GPT-4, and began using it internally. We realized, ‘This is huge. This will be a real inflection point in how the world perceives AI.’ But there were thorny issues: ‘Will this spread disinformation? Will it say deeply offensive things? Will we get into trouble?’ So we opted to start with a weaker version. Releasing both the chat interface and GPT-4 simultaneously felt like too big a leap. Instead, we launched the chat interface powered by GPT-3.5.
Marketing: Sam believes that great product markets itself simply based on utility. For user growth and value, smarter models, more compute, and better products will do. However, he thinks there’s a place for marketing: on where things are heading because people are feeling anxious about the technology. - Investment: I am personally surprised to learn that there's only one relentless all-in investor (who is always there) for OpenAI which is Josh Kushner.
Q: What have you learned about investors?
A: The number of investors who will actually show up and help you is unbelievably small.
Emotional state: When asked “what you don’t want me to know about you”, Sam thought for a few seconds and said that he was tired and repeat that it’s tiring. He didn’t reveal much on what keeps him going and commented that he would continue doing this for the rest of his career, but it’s really hard to explain to other people.Seeing how Lilian Weng quitting TML and joining OpenAI because she cannot continue at the pace a startup requires and wants a scoped role in a more predictable place, I have a lot more respect for Sam. While we can argue whether OpenAI should exist for existential AI safety reasons, it's undeniable that several scientific breakthroughs on AI come from early OpenAI era, and it's unimaginable how tough that is to navigate the field when giants like Yann did not believe in AGI. - What would happen the month after ASI is achieved: Sam expects little immediate change. He rejects the belief in a “machine god” that would transform everything almost instantly. He argues that the pace of human progress has been consistent, which is asmoothexponential curve: enormous when zoomed out and viewed across decades, but experienced as a series of incremental steps.
Q: In the story of this company, who is your favorite unsung hero?
A: The first person who comes to mind is Alec Radford. .. His work truly laid the foundation for the GPT series. Beyond many other critical contributions, he consistently inspired, guided, and pushed people toward directions that later proved immensely significant. … ‘He’s one of the kindest, most positive, and best people I’ve ever met.’
Codex and competitive advantage: He thinks ChatGPT didn’t play a huge role in the Codex takeoffs; it’s just Codex being the best product with the best model. He believes in product advantage in such AI competition: if somebody builds something better than Codex, he foresees users could just as easily migrate away from Codex. He believes that intelligence would become a commodity, but the scale of compute clusters and the ability to produce more computing power are durable advantage. - New hardware: He wants to create something that feels natural to be in constant presence in a social setting (such as in the conversation he is having). The reason is as mentioned before, he wants an AI that can stay always-on, proactive, and understand all the user’s context. - Potential oversupply of AI compute: In a future in which highly efficient genie can satisfy all demand with little compute, limited human attention could become the bottleneck, leaving more compute available than people need. - Scaling laws: Lots of people want/predict it to break down, but it still keeps going. - Mistake of messing up org structure: Sam shared that their original innovation of org structure as non-profit is a big mistake, even though back then it felt like a legitimate need.I don't think he shared much about the reason why it is a mistake, and I couldn't find much resources online. I also find it especially relevant in light of AI safety grantfunders talking about how not to default to non-profit.
Q: I always ask everyone the same traditional closing question: What’s the kindest thing someone has ever done for you?
A: I feel incredibly fortunate that so many people have gone to such great lengths to be kind to me throughout my life. When reflecting on this, what comes to mind are all those moments––scattered across different parts of my life––where people showed me extraordinary kindness. Just yesterday, my child shared his blueberries with me for the first time. It was very sweet—a beautiful moment.