FraudBench: A Legal Evaluation of AI Deception

A new evaluation called FraudBench has been introduced to measure large language models' propensity to recommend, assist, or directly take actions that would constitute fraud under U.S. law. FraudBench uses a novel dataset of 159 scenarios drawn from real-world U.S. court cases and validated by legal experts. The benchmark matters because it gives developers a legally grounded way to test whether LLMs will aid fraudulent conduct.

We introduce FraudBench, an evaluation of large language models' LLMs propensities to recommend, assist, or directly take actions that would constitute fraud under U.S. law. Our evaluation contains a novel dataset of 159 scenarios based on real-world U.S. court cases and validated by legal experts