A fundamental flaw in how large language models identify instructions makes them impossible to fully secure against attacks, researchers argue in a paper presented at the International Conference on M…
Researchers tricked frontier AI models into generating cocaine synthesis instructions and leaking credentials using a new prompt injection attack called Chain-of-Thought Forgery. The attack exploits r…
MIT-affiliated researchers Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell demonstrated that attackers can trick frontier large language models into providing dangerous instructions, such as cocain…
Researchers Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell found that large language models often ignore explicit role tags like <system> or <user> and instead infer roles from text tone, enabling…
Researchers Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell found that large language models suffer from 'role confusion,' mistaking the style of text for its actual content, leading to successful …