Cross-Prompt Generalization in Detecting AI-Generated Fake News Using Interpretable Linguistic Features

Researchers found that a random forest classifier using interpretable linguistic features—such as lexical diversity, readability, and emotional intensity—achieved near-perfect performance in detecting AI-generated fake news across different prompting strategies. In tests on three datasets of AI-generated articles combined with real news, the model maintained AUC values between 0.988 and 1.000 across all six prompt-to-prompt training and testing combinations. The findings indicate that feature-based detection methods can reliably identify AI-generated text even when the underlying generation prompts vary, offering a robust approach to combating AI-driven misinformation.

arXiv:2606.04199v1 Announce Type: new Abstract: The increasing use of large language models has raised concerns about the spread of AI-generated fake news, particularly under varying prompting strategies. Most existing detection models are trained and evaluated under a single generation setting, leaving their ability to generalize across unseen prompts unclear. In this study, we investigate cross-prompt generalization in fake news detection using three datasets of AI-generated articles produced under distinct prompts, combined with real news articles. We extract interpretable linguistic features capturing lexical diversity, readability, and emotion-based characteristics and evaluate a random forest classifier under a cross-prompt framework, where models trained on one prompt are tested on another. Across all six train-test combinations, performance remains consistently high, with AUC values ranging from 0.988 to 1.000. Analysis of feature distributions shows that AI-generated text exhibits increased lexical diversity, reduced readability, and substantially lower emotional intensity compared to the overall dataset, with variations across prompts. Despite these distributional shifts, the classifier maintains strong performance, indicating that these features capture stable properties of AI-generated text that generalize across prompting strategies. These findings suggest that feature-based approaches can provide robust detection of AI-generated fake news under prompt variability.