A recent study published in Nature has revealed that individuals struggle to differentiate between poetry written by famous poets and poetry generated by artificial intelligence that mimics their style. Surprisingly, participants in the study often preferred the poems created by AI over the works of real poets.
The research suggests that the intricacy of human-written poems can be mistaken for the confusion that AI-generated poetry can evoke. Furthermore, the AI-generated texts are perceived as more easily comprehensible and “humanized” by readers.
The study examined the verses of ten renowned poets spanning nearly 700 years of English literature including William Shakespeare, Walt Whitman, Emily Dickinson, and modern poetess Dorothy Laski. For each poet, researchers selected 5 poems and used ChatGPT 3.5 to generate an additional 5 poems mimicking the style of each author.
In the first experiment, 1634 participants were asked to distinguish between poems written by humans and those created by AI. The results revealed that individuals often misidentified the AI-generated poetry as human-written and vice versa. Additionally, participants at times mistook the works of famous poets for AI-generated content.
In a second experiment, nearly 700 individuals rated poetry based on various criteria including quality, beauty, emotionality, rhythm, and originality. Interestingly, participants who were unaware of the authors’ identities rated AI-generated poetry higher than that of real poets. However, when informed that the poetry was AI-generated, they rated it lower.
The findings of the study suggest that participants utilized flawed methods to differentiate between verses, often favoring simpler and more understandable AI-generated poems over complex and potentially confusing works by real poets. This phenomenon challenges previous studies and indicates a shift in the perception of generative technologies.