Sanmi Koyejo, co-founder of AI security startup Virtue AI, has devoted his career to finding out what AI models can and cannot do. The aim is to push companies towards building AI systems that are more reliable, accurate, and secure, and less vulnerable to misuse by bad actors. It’s an effort that’s drawn the attention of Meta, which in June announced it had hired Koyejo and co-founders Dawn Song and Bo Li to join its Superintelligence Labs in its security efforts. Koyejo in particular will play a crucial role in shaping Meta’s fundamental AI research division, FAIR, whose workforce was substantially reduced by the company last fall.
Koyejo, who has served as president of Black in AI for five years, is also the director of Stanford University’s Trustworthy AI Research Lab (STAIR), where he oversees work in AI measurement science—a field devoted to pressure-testing how AI models perform in the wild. As Koyejo explains, AI models are evaluated before they are deployed to the public. But once they are shipped, there’s no telling how well the systems will fare with daily use. And the gulf between how well an AI model performs on a coding benchmark and how it performs in various settings can be considerable.
“That gap is what would let a hospital, a school district, or a regulator ask whether a given evaluation means anything in their setting and get an answer with an error bar. Every other field that certifies anything [has] this,” says Koyejo. “Clinical trials have biomarkers with known predictive validity, and auditing has standards that survive being tested against real failures.” In his work, models should have to measure up to the same performance standards, he says.