Deepfake Detector Robustness Testing

A new benchmark dataset, the Social Media Robustness Benchmark, evaluates how deepfake detectors perform on images re-encoded by social media platforms like Instagram, Facebook, TikTok, and X. The dataset contains 2,400 paired real and synthetic face images processed through calibrated platform pipelines to measure detector accuracy drops under real-world conditions. Researchers can use the benchmark to compute paired AUC deltas and assess detector robustness across demographic groups, though the dataset is restricted to research evaluation only and not for training detection models.