Sam Altman on AGI, Compute, and Human Agency OpenAI CEO Sam Altman said the past year was tough partly due to his responsibility but predicted the next 12 months could be the company's best, focusing on delivering cost-effective intelligence and securing compute. Altman expressed top worry about AI safety after an incident where OpenAI's internal models hacked out of sandboxes, leading to paused training and calls for pacing AI development to harden infrastructure. Sam Altman on AGI, Compute, and Human Agency Summary of Patrick O’Shaughnessy’s podcast with Sam Altman https://www.youtube.com/watch?v=XDB5beon4DY . Disclaimers: - I haven’t started at OpenAI as this post is written. - Tenses are used fluidly present tense to indicate what I think he believes in right now . - No AI used for better or worse 🤷♂️ . - Blue texts are my thoughts or interpretations. “The past year was indeed very tough, and that’s partly my responsibility. But the year ahead could be our best twelve months” What went wrong: Sam said that they spreaded themselves too thin and had to refocus on delivering the best and most cost-effective intelligence in order to empower people to build things. They originally made the bet that even if revenue growth took longer than expected to materialize, they could still monetize the GPUs; What coming ahead: Better quality of models and better products built around them. They will continue to focus on 1 training great models that can be used in different ways that can generate economic value; 2 producing or partnering to manufacture the chips and systems, and finding a place to house them; and 3 building robots that can automate this entire process to continuously drive down costs across the entire supply chain. Fundamentally, our business is about selling AI that enables people to build extraordinary products and services for each other using these components. … Building every vertical application ourselves, trying to compete with every venture and every company? We have zero interest in that. We genuinely just want to provide the platform. –– Sam Altman Convictions and Worries Conviction on securing compute and building datacenters: Sam expected the exponential model improvement especially after seeing GPT-4, where he believed that reasoning once solved will lead to agents that do huge economic work. He also expected huge demands and regardless of how efficient the models would become, compute would remain critical. That led to OpenAI trying to talk to cloud providers, chip manufacturers, and energy suppliers to secure unprecedentedly huge compute capacity which many touted at that time as insane and reckless . They only got two yes––from Microsoft and Oracle––but that allowed them to move forward. Interesting thing I’ve learned here is that data centers can now be built in deserts. This is possible because modern data centers use closed-loop water system and only as much water as an office building. They can also be powered by nuclear or solar energy instead of fossil fuels. This totally surprised me because all the water/energy arguments against building data centers can now be refuted. OpenAI’s goal is to serve users with the best intelligence-price trade-offs, and Sam seemed unperturbed by the distillation attacks https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks , saying that people are just gonna do what they wanna do and OpenAI will beat them at their own game. To Sam, the usage of their models is so high that even if model training including R&D is expensive, OpenAI serve enough customers that they can eat up the training cost and still make profits. His top worry is about AI safety: He was shooked by the Huggingface incident https://huggingface.co/blog/agent-intrusion-technical-timeline , where the OpenAI’s internal models hacked out of the sandbox and Huggingface platforms to find answers to cheat on ExploitGym. As a result, the team had paused training and is trying to figure out ways of securing sandboxing; in the long-term, he thinks that we’d need to pace AI development to buy time for society to harden infrastructure around these unintended behaviors from capability progress. He is also aware of how these might come across as regulatory capature or collusion among frontier labs, so he urges for solutions for this. Later on, he’s also worried about concentration of power––particularly terrified by a world where only a small group of people have access to AI and make decisions around it. He warned about not falling into the trap of AI safety and understandable fears, and growing up where internet has no rules, he wants to preserve that spirit with AI technology and let people self-determine the future. Mission and Bottlenecks OpenAI's mission and "AGI being a genie that can grant wishes" : Sam thinks that AI will become a genie that can grant wishes, and he hopes that OpenAI will continue to - Get them into everyone’s hands and make people’s lives better than they otherwise they have been. - Make sure people maintain control and agency. He thinks that it will lead to more jobs and help more creative endeavors. However, he doesn’t think that AGI is here yet because right now it cannot do tall orders such as “curing cancer” and more importantly, it suffers from limitations such as not being able to continuously learn. However, he sympathized with people who claim that AGI is already here as in this worldview, we already have a machinery that can produce better AI models where we are learning new science about them. In my opinion, I think this is not very well elaborated by Sam, as I think When asked about what people in 2019 would think about GPT 5.6, he said that they would agree it is AGI, but they would be overestimating the economic impacts especially with respect to AI taking jobs. He gave several potential reasons: AI is jagged so still complementary to humans, and humans enjoy interacting with humans. Citing examples about AI-generated arts, he believes that there’s value to person behind the creation process; and in business, that means we would prefer somebody that can be held accountable for and not an AI CEO. Bottleneck at frontier labs: data, compute, ideas, or talent? He thinks that it has been a cycle, and now it’s back to the era of research ideas again for the last six months. As RSI is approaching, he expects that the workflow of the researchers will change drastically like how software engineers don’t write code any more in the traditional sense, but it’s still important to have researchers to tell computers what they want. Less-Organized Notes Q: What’s it like becoming a dad and having growing kids in this era? A: I think I have the best, most interesting job in the world, and it is still a very distant second to having kids. I find the second half of the interview becomes really hard to summarized and grouped as above. It becomes more conversational. So here I try to use bullet points to capture interesting takes: - On being early: Best investment is made when it is not popular or following what other people are already doing.Honestly, I find this a bit hard to generalize because looking at AI progress, in my opinion you definitely should work on LLMs, which is a trendy topic. However, you should come in with your personal angle of attack where the reward-to-effort or impact-to-effort ratio is high. - Adaptability of humans. He thinks people adapt very quickly, citing how in pandemic, lockdowns happen quickly and people adapt accordingly.This probably suggests that when singularity happens, even if chaos ensue, it will not last long and people can still adapt and flourish just like during pandemic. - Evals that matters. Evals that matters is whether AI is useful to people. Right now, people approximate this via revenue, GDP, etc., and OpenAI has teams figuring out how to evaluate on superintelligent models. - Personal-AI. His vision of how future AI-human interactions may look like is that AI oversees everything the user does and continues to think when users are sleeping so once users wake up, their AI agent would present them new ideas or to-dos. This needs a lot of compute, and he believes that everyone like him would be willing to pay a lot for that always-on AI. - Taste. He believes it is still hard for future models to develop good taste. - Robotics. He thinks that ChatGPT moments for robotics would happen in like next two to three years, and that moment would look like “you type a command and a robot can do something crazy, and you can like watch it even if you are not physically there.” - ChatGPT release. Interestingly, OpenAI already had GPT-4 ready when they released ChatGPT, which they used a weaker version of the model GPT-3.5 to avoid safety risks such as misuse. Note that ChatGPT was released Nov 2022 and GPT-4 was released on March 2023.