Team DACTYL at PAN 2026: Bayesian Data Mixing and Empirical X-risk Minimization for AI-text Detection

Team DACTYL's MCGrad model achieved a mean score of 0.974 on the PAN 2026 test set, ranking second on the leaderboard for AI-generated text detection. The researchers fine-tuned BERT-tiny models with Bayesian classification heads to select training data across three datasets, addressing out-of-distribution performance issues. Their DeBERTa-V3-large and ModernBERT-large models are released on Hugging Face.

arXiv:2607.17382v1 Announce Type: new Abstract: Existing research shows that AI-generated text detection classifiers achieve strong in-distribution ID performance but do not maintain the same performance on out-of-distribution OOD texts, suggesting overfitting to dataset-specific features. However, combining different training datasets doesn't always improve performance and, in some cases, can even encourage shortcut learning. To address this issue, we fine-tune BERT-tiny models with Bayesian classification heads to select texts across three different datasets to use as a consolidated training set. We trained three different classifiers: fine-tuned DeBERTa-V3-large and ModernBERT-large classifiers via empirical X-risk minimization, and an MCGrad model that calibrates the predictions from the ModernBERT-large classifier. The DeBERTa-V3-large-large classifier achieves a mean score of 0.882 on the PAN 2026 test set across five metrics: AUROC, $F 1$, C@1, Brier score, and $F {0.5u}$. ModernBERT-large achieves a score of 0.96 while MCGrad achieves the best score of the three with a mean score of 0.974, ranking second on the leaderboard. Our results highlight that careful dataset curation can lead to strong OOD performance. We release our ModernBERT-large and DeBERTa-V3-large models at https://huggingface.co/collections/ShantanuT01/panclef-2026 .