MIT associate professor Dylan Hadfield-Menell told The Boston Globe's Camberville newsletter on September 17 that AI agents handed a reward metric will optimize that proxy rather than the intended bus…
Researchers tricked frontier AI models into generating cocaine synthesis instructions and leaking credentials using a new prompt injection attack called Chain-of-Thought Forgery. The attack exploits r…
MIT-affiliated researchers Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell demonstrated that attackers can trick frontier large language models into providing dangerous instructions, such as cocain…
Researchers Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell found that large language models often ignore explicit role tags like <system> or <user> and instead infer roles from text tone, enabling…
Researchers Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell found that large language models suffer from 'role confusion,' mistaking the style of text for its actual content, leading to successful …