# My MRI interpretation vision model pointed at the right kidney and wrote "left"

> Source: <https://dev.to/lg_design/my-vision-model-pointed-at-the-right-kidney-and-wrote-left-i0>
> Published: 2026-09-17 07:35:22+00:00

I build a tool that reads MRI and CT scans with general vision models and explains the findings in plain language. The most repeated error in that pipeline isn't a missed tumour or an invented measurement. It's **left and right**.

We ran eight real studies through four readers (Claude, Gemini, Grok and Google's MedGemma), with the radiologist's signed report as ground truth. Here's what came back:

In medicine a left/right swap is a "never event". You don't ship a report that does this, and a better prompt doesn't fix it. So here is where the error comes from, and the small, model-free check that now catches it.

The detail that cracked it came from MedGemma. We ask each reader for two things per finding: a sentence, and a location on a specific slice. On one study MedGemma put its crosshair **17–26 mm to the right of the midline**, which matched the radiologist. Then it wrote that the finding was *"slightly to the left"*.

The crosshair was correct. The sentence described the side of the **picture**, not the side of the **patient**.

That's the radiological convention catching the model out. An axial scan is displayed as if you're standing at the patient's feet looking up, so the patient's right is on the left of the screen. A model trained mostly on ordinary photos describes what it sees: "on the left of the image". In a report, that's the wrong side.

This failure turns out to be useful. **When the coordinates are right and only the sentence is wrong, you can check one against the other without involving a model at all.**

Our volumes are converted from DICOM to NIfTI in the browser (dcm2niix compiled to WebAssembly). A NIfTI file carries an affine: a 4×4 matrix mapping voxel indices `(i, j, k)` to millimetres in world space. That space is **RAS**: +x is the patient's **R** ight, +y Anterior, +z Superior.

So once a finding has a voxel location, the sign of its world-x gives a second, independent answer about which side it's on:

```
export function voxToMm(affine: Affine, [i, j, k]: [number, number, number]) {
  const m = (r: number) =>
    affine[r][0] * i + affine[r][1] * j + affine[r][2] * k + affine[r][3]
  return [m(0), m(1), m(2)] as [number, number, number]
}

const OFF_MIDLINE_MM = 8

export function sideOfPoint(mm: [number, number, number]): Side | null {
  const x = mm[0]
  if (!Number.isFinite(x) || Math.abs(x) < OFF_MIDLINE_MM) return null
  return x > 0 ? 'right' : 'left' // RAS: +x is the patient's RIGHT
}
```

The 8 mm dead zone matters. A central disc extrusion sits at x ≈ 0 and belongs to neither side, and floating-point noise shouldn't get read as a side.

Reports come back in the user's language, so an English-only regex would silently check nothing for most users. The side words are prefix patterns, because almost every one of these languages inflects them (*destro/destra*, *rechts/rechten*, *prawy/prawa*):

``` js
const RIGHT_WORDS =
  /\bright\w*|\bdestr[aeio]\w*|\bderech\w*|\bdroit\w*|\brecht[aensr]\w*|\bdireit\w*|\bpraw(?!d)\w*|\bsağ\w*|يمين|أيمن|اليمنى/iu
const LEFT_WORDS =
  /\bleft\w*|\bsinistr\w*|\bizquierd\w*|\bgauche\w*|\blink[aensr]\w*|\besquerd\w*|\blew\w*|\bsol\b|يسار|أيسر|اليسرى/iu
const BOTH_SIDES =
  /\bbilateral\w*|\bbilaterale?\w*|\bbeidseit\w*|\bobustronn\w*|\bambos\b|\bambas\b|\bdes deux côtés\b|ثنائي/iu
```

Two traps are handled on purpose:

`\b` needs a non-word character before the `r`, and `b` is a word character. MRI reports say "bright" constantly.`(?!d)`.
Then there's the rule that took longest to get right: **a sentence that names both sides claims neither.**

```
export function sideClaimed(text: string): Side | null {
  if (!text) return null
  if (BOTH_SIDES.test(text)) return null
  const right = RIGHT_WORDS.test(text)
  const left = LEFT_WORDS.test(text)
  if (right === left) return null // none, or one of each
  return right ? 'right' : 'left'
}
```

"The left L5 root is displaced; the right is not" mentions both words. If you picked either one, you'd be inventing a claim the sentence never made.

This is the part I got wrong first. A **left knee** MRI sits entirely at negative x. Every finding in it is "on the left", while the text is talking about the medial and lateral compartments of that one knee. Run the check there and every knee report gets flagged.

So the sign of x only means a body side when the field of view reaches well past the midline in both directions. You can't read that off the origin, because the affine can rotate the grid. You have to check all eight corners:

``` js
const MIDLINE_MARGIN_MM = 40

function worldXRange(affine: Affine, [nx, ny, nz]: [number, number, number]) {
  let lo = Infinity, hi = -Infinity
  for (const i of [0, nx - 1])
    for (const j of [0, ny - 1])
      for (const k of [0, nz - 1]) {
        const x = voxToMm(affine, [i, j, k])[0]
        lo = Math.min(lo, x); hi = Math.max(hi, x)
      }
  return [lo, hi]
}

export function straddlesMidline(affine?: Affine, dims?: [number, number, number]) {
  if (!affine || !dims) return false
  const [lo, hi] = worldXRange(affine, dims)
  return lo <= -MIDLINE_MARGIN_MM && hi >= MIDLINE_MARGIN_MM
}
```

The 40 mm margin is generous on purpose. A spine or abdomen study clears it easily, and a limb never does.

```
export function lateralityConflict(opts: {
  text: string
  locations?: { mm?: [number, number, number] }[]
  affine?: Affine
  dims?: [number, number, number]
}): Side | null {
  const claimed = sideClaimed(opts.text)
  if (!claimed) return null
  if (!straddlesMidline(opts.affine, opts.dims)) return null

  // Several points must agree with each other before they outvote the sentence.
  const sides = (opts.locations ?? [])
    .map((l) => (l.mm ? sideOfPoint(l.mm) : null))
    .filter((s): s is Side => s !== null)
  if (sides.length === 0 || !sides.every((s) => s === sides[0])) return null

  return sides[0] === claimed ? null : sides[0]
}
```

Count the `return null` s. There are four ways for this function to say nothing, and one way for it to speak. That's deliberate. It isn't a classifier trying to arbitrate uncertain cases. It exists to catch **the contradictions that are certain**: the text names one side, the geometry clearly and consistently shows the other, and the scan covers both sides of the body.

We don't silently rewrite the sentence. The crosshair is *usually* right (that's the failure we measured), but the sentence is the part a person reads and acts on. So the conflict is stored on the finding (`sideConflict: 'right'`) and shown, instead of being quietly "fixed" by code that could itself be wrong.

When a multimodal model gives you **structured output next to prose**, the two halves often fail independently. Coordinates, slice indices, bounding boxes and counts come from a different part of the model's behaviour than the sentence does. Whenever the domain gives you a deterministic way to relate the two (here: an affine matrix and a coordinate convention), you get a check that costs microseconds, needs no second model, and can't hallucinate.

It's worth finding those checks before reaching for another prompt.

*I'm Lorenzo, and I build [ReadYourScan](https://readyourscan.com), which opens DICOM/NIfTI studies in the browser and explains them in plain language. It's not a diagnosis. The browser viewer is open source (MIT): [lgdesignee/readyourscan-viewer](https://github.com/lgdesignee/readyourscan-viewer).*