Evals, Explained in Plain Language

You can't just feel that an AI system got better. You need a test — and a way to grade it.

Give it a test it's never seen #

If it's already seen the answers, acing the test proves nothing. A real eval is questions it has never met before.

Score every answer #

Every answer gets checked against the right one. No vibes — just a tally at the end.

One test isn't enough #

Good at math doesn't mean good at everything. Run lots of different tests — one score alone can hide a lot.

Compare the old model to the new one #

"Better" isn't automatic. The new model can climb in reasoning and safety and still slip backwards in coding, all at once.

Watch the score over time #

One release can dip — that's normal. What matters is the trend across many releases, on the tests that actually matter to you. Evals — that's the real name for it.