I recently saw two posts here saying the Pope's recent encyclical was largely written by AI. Two posts, on the front page at the same time.
Since I've sometimes crossposted blog posts here, I thought I'd write a post to at least make it clear that I have a different position.
We should not expect the Vatican to want or need to use AI to write documents meant for wide public consumption. Yes, the liturgical writing style they tend to use can be awkward, but reading and writing such documents is kinda what Catholic priests do. Perhaps it's hard to believe that unassisted humans — in this age where people's attention spans have been fried by phones and social media — can write something 2% as long as Zvi's AI newsletter series, but an organization with the resources of the Catholic Church actually can. Also, the encyclical was presumably written and reviewed by several people working collaboratively; it's not like it was just one guy who can secretly use ChatGPT on his own.
Now, the US government has been using AI for important documents recently, like tariffs and health stuff and Epstein censorship, but that's a very different situation: the US government has plenty of workers, but the leadership didn't want to involve too many people because they didn't trust existing government employees to agree with their agenda. (Also, Trump's cabinet members are probably dumber than the average Catholic bishop.)
Humans familiar with AI writing (and preferably with non-AI writing of the relevant type) are more reliable than AI-based AI detectors. This should be obvious to people with experience about this, but here's a citation I guess.
So, do we see people reading Magnifica Humanitas and saying "hey that's AI"? Not really. I read it, and I don't think it's AI. I don't think other forums are buying the "it's AI" argument either; here's r/ChatGPT being less credulous than LessWrong, and they seem to be familiar with what AI can do these days.
Yes, it has em-dashes, and some "not X but Y". But you can't analyze writing on such a simplistic level when it's a kind of thing that tends to use those. Is the quote below AI writing because it's using em-dashes and "not X but Y"?
For the completeness of so brief a prayer He added — in order that we should supplicate not touching the remitting merely, but touching the entire averting, of acts of guilt — Lead us not into temptation: that is, suffer us not to be led into it, by him (of course) who tempts; but far be the thought that the Lord should seem to tempt, as if He either were ignorant of the faith of any, or else were eager to overthrow it. Infirmity and malice are characteristics of the Devil...The final clause, therefore, is consonant, and interprets the sense of Lead us not into temptation; for this sense is, But convey us away from the Evil One. Yeah, they're in the training set. Not only is that an obvious thing for Pangram to do, it's the only plausible way they could get 0% consistently. If we wait long enough, maybe Pangram will update their model's training and this new encyclical will magically become 0% AI too!
Any study on AI detector reliability is using specific models, specific writing styles, and specific number thresholds. An encyclical is a different type of writing than such studies have used, so those results tend to not apply well. More generally, LLMs tend to copy some patterns used in formal writing, and most writing is not that. If you're checking some kind of writing that normally never uses em-dashes, you can just count em-dashes and get good accuracy results by such metrics.
If there are people around who paid for Pangram and just want to use it on something, and also have too much free time, I actually have a suggestion for something to do instead: a website that rates AI-ness of everything on the front page of Hacker News, and lets people filter links by AI-ness. A lot of the posts there are pretty obviously AI these days, and they're types of writing that automated AI detection should work better on. Blindly trusting software like Pangram is, uh, one of the things the encyclical was warning people about. So good job making its point, but also, maybe stop that.