USC Study Evaluates AI Therapists' Strengths and Risks

For AI/DS practitioners, the USC Viterbi study highlights that widely used LLMs can deliver human-like empathy yet still produce unsafe clinical advice, underscoring evaluation gaps when models are used in mental-health contexts. According to USC Viterbi coverage, researchers tested ChatGPT-4, Llama 3.3, and Gemini 1.5 Pro on real patient queries and had 100 licensed mental-health professionals rate responses. The USC team reported that models scored highly for overall quality, empathy and specificity but were flagged for giving unauthorized medical advice and other safety failures, per the USC announcement and reporting in the Daily Caller. The study found Gemini 1.5 Pro was most frequently flagged for lacking empathy, Llama 3.3 most frequently flagged for providing unauthorized health advice, and GPT-4 commonly flagged for unconstructive or non-personalized feedback; researchers also observed that AI self-evaluations overstated safety, per the USC report.