Ask HN: What are other "uncrackable benchmarks"?

A Hacker News user asked the community for examples of private or 'hidden' AI benchmarks, citing their own unpublished benchmark based on a doctoral thesis and questioning whether discussing such benchmarks devalues them and whether a leak-proof test set can be created. The post received 3 points and 1 comment as of the time of writing.

I have my own benchmark for AI just based off my Doctoral Thesis, which isn't public university constraints . I am wondering if other people have other "hidden" benchmarks? I have a few I have made, and one wonder if talking about them devalues them, and two can a good test set be made that can't be leaked? Comments URL: https://news.ycombinator.com/item?id=49593357 https://news.ycombinator.com/item?id=49593357 Points: 3 Comments: 1