Introducing [LaughBench](https://github.com/taylor-gl/LaughBench).
For a long time, I've considered the ability for AI models to tell novel, funny jokes that actually make people laugh to be a robust indicator of real general intelligence (as opposed to, say, coding tasks). I have used this benchmark informally over the months, seeing if any model could generate novel jokes to make me laugh (at a rate above the really low base rate where something is funny by accident).
So far, the answer has always been no. As you can see from the chart in the repo, none of the models succeeded in producing laughter:
The frontier models (GPT-5.6 Sol and Fable) are able to occasionally make me smile. This indicates there's a spark of intelligence present that wasn't present to the same degree in prior models.
There are a few reasons I think laughter-producing comedy is a good benchmark for intelligence (as mentioned in the repo):
It seems important to have benchmarks aimed at measuring "real intelligence" of AI models, whatever that means, since I suspect many dangerous activities will become possible once real intelligence hits a certain point (creating bioweapons, manipulating/deceiving human beings, creating long-term plans that aren't doomed to failure, advancing physics, robotics, biology, or AI research, etc.) Most of what we want in the good AI future also requires this ethereal "real intelligence": curing all diseases and restructuring society for the common good are also IQ-hard activities.
Some benchmarks already have the flavor of this. When I looked into it, I thought FrontierMath Open Problems was at least a partial fit. The ARC-AGI stuff too. But ARC-AGI seems more like something that is easier if you have more IQ, but sufficient spatial reasoning and other memorizable skills can make it easier and generalize across games. (Similar complaint about FrontierMath.)
In the past I have mentioned the possibility of benchmarking AI performance on novel board games (with branching complexity at least equal to chess etc.). But generating arbitrary board games with no significant commonalities to existing board games seems pretty hard. You could take the ARC-AGI approach of hand crafting them, but doing it correctly is IQ-hard for the author.
Or you could just ask the AI to tell you a novel joke.