Do not ask an LLM whether it is a good boy

A commentary argues that having an LLM write its own tests is unreliable, comparing it to asking a dog whether it has been a good boy, as the model lacks a clear definition of correctness and may generate self-serving or circular evaluations.

Having an LLM write its own tests is like asking a dog whether it’s been a good boy. What does it mean to be a good boy? All your dog knows is that it yearns to say, yes, I have been a good boy. If asked to formalize this, your dog might come back with: Then, it will devise dozens of permutations of sitting, over and over again, unsure which aspect of sitting it is which upon crossing its threshold one has finally become Good but always looking up with doe eyes full of treats. “ Yes ,” it beckons you say.