Characterizing Human-Likeness in AI Generated Poetry: A Zero-shot Classification Study

A new study from arXiv proposes a zero-shot detection pipeline to characterize human-likeness in AI-generated poetry, finding that GenAI poems are the most difficult to distinguish even without modification. The research aims to extract attributes that contribute to classification and misclassification of human and AI poems, providing evidence for the poetry distinguishability claim and strengthening modern detection pipelines.

arXiv:2607.26221v1 Announce Type: new Abstract: With the advancement of AI technologies, Generative AI GenAI and human written text have become nearly indistinguishable. Additionally, the global standardization of AI chatbots made academic malpractice more frequent. Furthermore, existing research indicates GenAI poems are the most difficult to distinguish even without any modification thus, GenAI poems are naturally deemed human-like by modern detectors. However, the objectivity of such dissertations needs to be verified against modern detection tools but the subjectivity of poetry and the black-box nature of the modern LLMs Large Language Models architectures made verification of such work quite complicated. Hence, the main objective of the research is to deduce the attributes of English poetry that contribute classification and misclassification of both human and AI poems and provide corroborating or contradicting evidence to the poetry distinguishability claim. For such characterizations, we propose a Zero-shot detection pipeline with a dataset consisting of both human and AI poems to verify the distinguishability of human and AI creation and extract the aforementioned crucial attributes for accurate classification. Extraction of such attributes provides benefits in two ways: firstly, it reduces the margin of training needed as only the poems based on misclassifying attributes need to be trained and fine tuned and finally provides a critical insight to the GenAI detection dilemma to strengthen the modern detection pipelines.