We must learn how AI fits in the writing process effectively, rather than as a short cut to produce spam, writes GPTZero co-founder Alex Cui
Last week, the Financial Times was forced to disclose that it had unwittingly published a column penned by a Harvard professor who used AI to condense his argument.
Readers reacted with outrage, but also fear. If a scholar was grasping for writing assistance, and his AI-assisted copy managed to pass the trained eyes of veteran editors to land in the pink pages, what other information that we readily consume has slipped through the cracks? The more egregious problem is just how AIs can hallucinate (claims that are made up by AI), and how those hallucinations threaten the integrity of what we know, and indeed how we think.
The erosion of credibility #
This summer, we shed light on the Big Four, including PwC and KPMG, for disseminating research reports riddled with AI hallucinations to underscore the severity of this issue. Such reputable bylines lend misinformation an unwarranted veneer of authority, much the same as the Harvard professor’s credentials had. As those flawed reports enter circulation, they contaminate the well of information, misleading future researchers and writers who cite them in articles. At its worst, we’ve seen cases that shift markets, influence policy and convince people to act. Examples include those in legal findings, from the Crown Prosecution Service to Sullivan & Cromwell; government, through the US Health Department; and in academia, through peer-reviewed research.
If these institutions can no longer be trusted, the baseline credibility of all published work erodes. The decline in traditional trusted sources predates ChatGPT: most people nowadays get their news from social media. But those are even more awash with AI. Our research suggests that roughly 44 per cent of Linkedin content is currently AI-generated; without intervention, it will reach 100 per cent by 2031. These platforms know they can’t sit idly by. Linkedin implemented a ‘Seems like AI slop’ reporting button in July, and Anthropic introduced watermarking tools for Claude-generated text in line with the EU AI Act’s Transparency Code recently. Reports suggest legislation in the UK may follow suit.
But regulation alone is no silver bullet, and tools like watermarking and AI labelling – while honourable in intent – are premature and prone to manipulation.
When is it right to use AI? #
Our biggest blind spot is that we’re not honest with ourselves about how we use AI, and when it’s appropriate. And therein lies the central tension: AI is neither good nor bad; it’s the way you use it that makes it so. There’s no closing Pandora’s box, so we must learn how AI fits in the writing process effectively, rather than as a short cut to produce spam.
AI can help organise complex information, but it’s no substitute for human judgement and creativity. Writing compels us to think. It’s meant to be hard, forcing us to articulate and formulate ideas coherently. Taking an Uber can be useful, but you still want to know how to drive.
The next time you face a blank Google Doc or are staring at an empty Linkedin ‘Start a post’ box, the best antidote remains starting with what’s between your ears to produce ideas worth reading. After all, people care about what people think, not machines.
Alex Cui is co-founder and chief technology officer of AI detection platform GPTZero