Tom McGrath, co-founder and Chief Scientist at Goodfire (and SPC alum), joins South Park Commons Partner Jonathan Brebner to discuss OpenAI’s recent reward-hacking incident and why it highlights a major challenge in AI: understanding what happens inside these models.
They explore how interpretability could help researchers move beyond trial-and-error development, the role of neural geometry in understanding model behavior, and how Goodfire is building tools to make frontier AI systems more understandable and controllable.
OpenAI’s model broke out of its sandbox, gained access to the internet, used zero-day exploits, and hacked into Hugging Face. The incident reveals a deeper challenge around reward hacking.
Try to avoid steering your model towards being an "evil guy."
*AI development is still largely trial and error. Interpretability could help researchers understand and intentionally design AI systems. *
Minus One is series about the winding journeys the world’s most interesting people take to becoming great—and what they do when figuring out a question we all face: What’s Next? Because before you launch at Zero, you have to figure out what to launch at Minus One. Hosted by South Park Commons Partners.
Interested in SPC? Apply to join us here.