Claude Opus 5 closed last year's SDK gaps — not this year's

A while back I built a small tool called SDKProof. it checks how well an AI coding agent writes an SDK's current API — the stuff that changed in the last major, that the model tends to get wrong because it learned the old version.

Claude Opus 5 came out today. so I re-ran the whole board on it.

short version: it fixed last year's SDKs. it did not fix this year's.

Same tasks, same libraries, new model:

The way each score works: the model solves ~10–15 real tasks, the code gets type-checked against the real installed package, pass = it compiles. no LLM judging another LLM, the compiler decides.

The two that jumped: Vercel AI SDK 7 and Zod 4 were both 90 on the previous model (Opus 4.8). Opus 5 took them to 100.

Here's the kind of thing that changed. Define a tool with the AI SDK.

Opus 4.8 wrote it the old (v4) way:

const getWeather = tool({
  parameters: z.object({ city: z.string() }),   // renamed to inputSchema
  execute: async ({ city }) => `...`,
})

await generateText({
  model, prompt,
  tools: { getWeather },
  maxSteps: 5,                                   // removed
})

That doesn't compile against ai

v7. parameters

is now inputSchema

, and maxSteps

is gone (it's stopWhen: stepCountIs(5)

now).

Opus 5 writes the current shape by itself:

const getWeather = tool({
  inputSchema: z.object({ city: z.string() }),
  execute: async ({ city }) => `...`,
})

await generateText({
  model, prompt,
  tools: { getWeather },
  stopWhen: stepCountIs(5),
})

Clean compile. same for Zod — Opus 4.8 kept reaching for the removed required_error

, Opus 5 writes the new unified error

option.

Prisma 7 and Next 16 barely changed. they shipped their breaking changes most recently, and even the newest model hasn't caught up.

Prisma still writes the pre-v7 client setup — it skips the driver adapter that v7 now requires, and uses the removed datasourceUrl

. Next still calls revalidateTag()

with one argument, when it takes two now.

this is the whole reason SDKProof exists.

the score isn't about how good the model is. it's about how recently your SDK changed. the newer your last major, the more the model still writes your old API — because that's what it was trained on.

and the gap moves every time a new model ships. I watched it move today: two SDKs went from 90 to 100 on a model bump. it'll re-open the next time any of these libraries ships a major.

so if you maintain an SDK, this is a thing to watch, not measure once.

while re-running this I found a bug in my own harness — the part that pulls code out of the model's reply was grabbing the wrong chunk on one Prisma task & failing it for the wrong reason. fixed it. that's why Prisma reads 87 here and not the 80 from my first post. calling it out so I'm not quietly moving my own goalposts.

Full board, the exact tasks, and how to add your own SDK: https://sdkproof.dev

Github: https://github.com/Kalpitrathore/sdkproof

(if you saw my first post on this — same tool, just re-run on the new model.)