When cloud computing was a new, cool buzzword thrown around that no one understood, a shortcut was suggested whether it is the right solution:
In any sentence, replace “in the cloud” with “on another person’s computer”.
“Computing on the cloud” becomes “computing on another person’s computer”.
“I backup my data to the cloud” becomes “backup to another person’s computer”.
This makes it easier to weigh benefits – they provide more space, and provide maintenance, against drawbacks – they have your data and fully control the computer it sits on.
Today, “AI” is thrown around as an all-encompassing buzzword. “Enhance your start-up with AI”, “AI helps prepare lawsuit”, “AI-written lawsuit contains mistakes”, “Fighting wildfires with the help of AI”, “Get data insights with AI”.
Is there a similar shortcut that clarifies thinking?
To expose overuse of the term “using AI” without meaning, I propose to replace it with “using computers”: “Enhance your start-up with computers”, “Using computers helps prepare lawsuit”, “Computer-written lawsuit contains mistakes”, “Fighting wildfires with the help of computers”, “Get data insights with computers”.
This makes a few things clear: First, there are benefits to using computers because of their automatic data processing. This is not new with AI. Secondly, the replacement is effective at stripping away the mythical meaning of “AI” as an independent actor, removing the possibility to delegate ownership to it. Saying “my computer did it” sounds today much sillier than “my AI did it”.
So, replace “using AI” with “using computers”. It reveals to you how little the statement by itself tells you.
To seriously talk about AI, we have to unpack the term. In general terms, we are talking about methods that expand their capabilities with increasing data. Therefore to judge whether AI is good or not for an application, we need to find out:
-
Where do the data come from, who made them?
-
Question 1 leads you to problematic biases in the training data that the AI inherits. It can also reveal copyright issues and whether the data producers are fairly compensated.
-
By what performance metric do the claimants want to be judged?
-
Question 2 reveals what people value, and whether this aligns with your values.
-
Is the performance better in that metric than the current method?
-
Question 3 reveals whether there was an improvement made so far, and may help distinguish vaporware from genuine improvement. The baseline may be another AI method, for example comparing LLMs to Markov Chains trivially reveals how much better LLMs are. Only very few analyses truly lack a baseline.
Note that I did not include “what is the model?” – this is the least important technical detail.
Questions 1) and 3) are often not made by the claimants, but reused. This is often not or extremely briefly described, reflecting how much time the AI developers spent on it. These three questions are essential for putting the AI claim into context.
Some cop-outs:
- If only similar performance is achieved, the AI developers often point to improvements in processing time, which should be added to the performance metric answer of question 2. It’s a valid goal to achieve somewhat comparable performance at much faster speed.
- My pet peeve cop-out is “it has potential for improvement in the future.” This may be technically true, because models might learn with more training data. However, at least in a scientific and policy context a opportunity for critically and objectively examining the outcome, in context of the historical baseline, is being skipped.
Demand from AI news articles to know the data origin, performance metric, and historic baseline.
If they don’t give that information, replace “using AI” with “using computers.”