# The AI Was Right. The Answer Was Still Wrong.

> Source: <https://dev.to/akanksha_sharma/the-ai-was-right-the-answer-was-still-wrong-2pl4>
> Published: 2026-09-25 14:37:12+00:00

Have you ever asked AI to do something very specific and it did almost everything right?

Except the one thing you specifically told it **not** to do. 

That made me curious.

Instead of asking **“Which AI model is smartest?”** I wanted to test something much more specific:

**Can AI actually follow small instructions while solving a task?**

I created a benchmark focused on **instruction following in coding tasks**.

For example:

Fix this JavaScript function.

Don't use `map()`.

Return only the corrected code.

The model isn't only expected to fix the bug.

It also needs to follow every instruction.

So I check:

Because sometimes **the answer is correct but the task isn't.**

The testing flow is simple:

**Same prompt → Multiple models → Same evaluation → Compare**

For every task, I:

This keeps the test focused on the model's behavior rather than changing the task for each model.

I didn't want a simple **right/wrong** score.

A response can solve the coding problem while still breaking one of the instructions.

So I measure two things separately:

**1. Task Correctness**

Did the model actually solve the underlying coding problem?

**2. Instruction Compliance**

Did it follow the individual constraints in the prompt?

For example if a task contains four requirements and the model follows three:

**3 / 4 = 75% instruction compliance**

Across the benchmark these scores can show whether a model is not only capable of solving the problem but also capable of **doing exactly what was asked**.

AI is becoming a bigger part of everyday coding.

We often focus on whether a model can generate working code but real developer tasks usually come with constraints:

“Don't change this.”

“Use this approach.”

“Return only the code.”

“Don't use this method.”

Those tiny instructions can completely change whether the response is actually useful.

That's the behavior I wanted to investigate.

The interesting part isn't simply finding a model with the highest number.

I'm looking for **patterns**.

Do models struggle more when there are several constraints?

Does solving the code correctly make instruction following more reliable?

Which types of instructions are most commonly ignored?

And can a model be **technically right while still being practically wrong?**

Those are the questions this benchmark is designed to explore.

If I expand the benchmark I'd add:

That could reveal whether instruction following remains reliable as tasks become more realistic.

Now I'm curious about your experience.

**What's the smallest instruction you've given an AI that it completely ignored?** 😭

Drop it in the comments.

Maybe your most frustrating AI moment becomes the next benchmark task.