{"slug": "my-mri-interpretation-vision-model-pointed-at-the-right-kidney-and-wrote-left", "title": "My MRI interpretation vision model pointed at the right kidney and wrote \"left\"", "summary": "A developer building an MRI/CT interpretation tool found that left/right laterality errors were the most common failure across four vision models (Claude, Gemini, Grok, and Google's MedGemma) tested on eight real studies. The root cause is that models describe the side of the displayed image rather than the patient's anatomical side, since axial scans show the patient's right on the viewer's left. The developer added a model-free check that compares a finding's RAS world-x coordinate against the side word in the generated text, using an 8 mm midline dead zone and multilingual prefix regexes to catch mismatches.", "body_md": "I build a tool that reads MRI and CT scans with general vision models and explains the findings in plain language. The most repeated error in that pipeline isn't a missed tumour or an invented measurement. It's **left and right**.\n\nWe ran eight real studies through four readers (Claude, Gemini, Grok and Google's MedGemma), with the radiologist's signed report as ground truth. Here's what came back:\n\nIn medicine a left/right swap is a \"never event\". You don't ship a report that does this, and a better prompt doesn't fix it. So here is where the error comes from, and the small, model-free check that now catches it.\n\nThe detail that cracked it came from MedGemma. We ask each reader for two things per finding: a sentence, and a location on a specific slice. On one study MedGemma put its crosshair **17–26 mm to the right of the midline**, which matched the radiologist. Then it wrote that the finding was *\"slightly to the left\"*.\n\nThe crosshair was correct. The sentence described the side of the **picture**, not the side of the **patient**.\n\nThat's the radiological convention catching the model out. An axial scan is displayed as if you're standing at the patient's feet looking up, so the patient's right is on the left of the screen. A model trained mostly on ordinary photos describes what it sees: \"on the left of the image\". In a report, that's the wrong side.\n\nThis failure turns out to be useful. **When the coordinates are right and only the sentence is wrong, you can check one against the other without involving a model at all.**\n\nOur volumes are converted from DICOM to NIfTI in the browser (dcm2niix compiled to WebAssembly). A NIfTI file carries an affine: a 4×4 matrix mapping voxel indices `(i, j, k)` to millimetres in world space. That space is **RAS**: +x is the patient's **R** ight, +y Anterior, +z Superior.\n\nSo once a finding has a voxel location, the sign of its world-x gives a second, independent answer about which side it's on:\n\n```\nexport function voxToMm(affine: Affine, [i, j, k]: [number, number, number]) {\n  const m = (r: number) =>\n    affine[r][0] * i + affine[r][1] * j + affine[r][2] * k + affine[r][3]\n  return [m(0), m(1), m(2)] as [number, number, number]\n}\n\nconst OFF_MIDLINE_MM = 8\n\nexport function sideOfPoint(mm: [number, number, number]): Side | null {\n  const x = mm[0]\n  if (!Number.isFinite(x) || Math.abs(x) < OFF_MIDLINE_MM) return null\n  return x > 0 ? 'right' : 'left' // RAS: +x is the patient's RIGHT\n}\n```\n\nThe 8 mm dead zone matters. A central disc extrusion sits at x ≈ 0 and belongs to neither side, and floating-point noise shouldn't get read as a side.\n\nReports come back in the user's language, so an English-only regex would silently check nothing for most users. The side words are prefix patterns, because almost every one of these languages inflects them (*destro/destra*, *rechts/rechten*, *prawy/prawa*):\n\n``` js\nconst RIGHT_WORDS =\n  /\\bright\\w*|\\bdestr[aeio]\\w*|\\bderech\\w*|\\bdroit\\w*|\\brecht[aensr]\\w*|\\bdireit\\w*|\\bpraw(?!d)\\w*|\\bsağ\\w*|يمين|أيمن|اليمنى/iu\nconst LEFT_WORDS =\n  /\\bleft\\w*|\\bsinistr\\w*|\\bizquierd\\w*|\\bgauche\\w*|\\blink[aensr]\\w*|\\besquerd\\w*|\\blew\\w*|\\bsol\\b|يسار|أيسر|اليسرى/iu\nconst BOTH_SIDES =\n  /\\bbilateral\\w*|\\bbilaterale?\\w*|\\bbeidseit\\w*|\\bobustronn\\w*|\\bambos\\b|\\bambas\\b|\\bdes deux côtés\\b|ثنائي/iu\n```\n\nTwo traps are handled on purpose:\n\n`\\b` needs a non-word character before the `r`, and `b` is a word character. MRI reports say \"bright\" constantly.`(?!d)`.\nThen there's the rule that took longest to get right: **a sentence that names both sides claims neither.**\n\n```\nexport function sideClaimed(text: string): Side | null {\n  if (!text) return null\n  if (BOTH_SIDES.test(text)) return null\n  const right = RIGHT_WORDS.test(text)\n  const left = LEFT_WORDS.test(text)\n  if (right === left) return null // none, or one of each\n  return right ? 'right' : 'left'\n}\n```\n\n\"The left L5 root is displaced; the right is not\" mentions both words. If you picked either one, you'd be inventing a claim the sentence never made.\n\nThis is the part I got wrong first. A **left knee** MRI sits entirely at negative x. Every finding in it is \"on the left\", while the text is talking about the medial and lateral compartments of that one knee. Run the check there and every knee report gets flagged.\n\nSo the sign of x only means a body side when the field of view reaches well past the midline in both directions. You can't read that off the origin, because the affine can rotate the grid. You have to check all eight corners:\n\n``` js\nconst MIDLINE_MARGIN_MM = 40\n\nfunction worldXRange(affine: Affine, [nx, ny, nz]: [number, number, number]) {\n  let lo = Infinity, hi = -Infinity\n  for (const i of [0, nx - 1])\n    for (const j of [0, ny - 1])\n      for (const k of [0, nz - 1]) {\n        const x = voxToMm(affine, [i, j, k])[0]\n        lo = Math.min(lo, x); hi = Math.max(hi, x)\n      }\n  return [lo, hi]\n}\n\nexport function straddlesMidline(affine?: Affine, dims?: [number, number, number]) {\n  if (!affine || !dims) return false\n  const [lo, hi] = worldXRange(affine, dims)\n  return lo <= -MIDLINE_MARGIN_MM && hi >= MIDLINE_MARGIN_MM\n}\n```\n\nThe 40 mm margin is generous on purpose. A spine or abdomen study clears it easily, and a limb never does.\n\n```\nexport function lateralityConflict(opts: {\n  text: string\n  locations?: { mm?: [number, number, number] }[]\n  affine?: Affine\n  dims?: [number, number, number]\n}): Side | null {\n  const claimed = sideClaimed(opts.text)\n  if (!claimed) return null\n  if (!straddlesMidline(opts.affine, opts.dims)) return null\n\n  // Several points must agree with each other before they outvote the sentence.\n  const sides = (opts.locations ?? [])\n    .map((l) => (l.mm ? sideOfPoint(l.mm) : null))\n    .filter((s): s is Side => s !== null)\n  if (sides.length === 0 || !sides.every((s) => s === sides[0])) return null\n\n  return sides[0] === claimed ? null : sides[0]\n}\n```\n\nCount the `return null` s. There are four ways for this function to say nothing, and one way for it to speak. That's deliberate. It isn't a classifier trying to arbitrate uncertain cases. It exists to catch **the contradictions that are certain**: the text names one side, the geometry clearly and consistently shows the other, and the scan covers both sides of the body.\n\nWe don't silently rewrite the sentence. The crosshair is *usually* right (that's the failure we measured), but the sentence is the part a person reads and acts on. So the conflict is stored on the finding (`sideConflict: 'right'`) and shown, instead of being quietly \"fixed\" by code that could itself be wrong.\n\nWhen a multimodal model gives you **structured output next to prose**, the two halves often fail independently. Coordinates, slice indices, bounding boxes and counts come from a different part of the model's behaviour than the sentence does. Whenever the domain gives you a deterministic way to relate the two (here: an affine matrix and a coordinate convention), you get a check that costs microseconds, needs no second model, and can't hallucinate.\n\nIt's worth finding those checks before reaching for another prompt.\n\n*I'm Lorenzo, and I build [ReadYourScan](https://readyourscan.com), which opens DICOM/NIfTI studies in the browser and explains them in plain language. It's not a diagnosis. The browser viewer is open source (MIT): [lgdesignee/readyourscan-viewer](https://github.com/lgdesignee/readyourscan-viewer).*", "url": "https://wpnews.pro/news/my-mri-interpretation-vision-model-pointed-at-the-right-kidney-and-wrote-left", "canonical_source": "https://dev.to/lg_design/my-vision-model-pointed-at-the-right-kidney-and-wrote-left-i0", "published_at": "2026-09-17 07:35:22+00:00", "updated_at": "2026-09-17 07:53:42.790920+00:00", "lang": "en", "topics": ["artificial-intelligence", "computer-vision", "ai-safety", "ai-products", "large-language-models"], "entities": ["Claude", "Gemini", "Grok", "MedGemma", "Google", "dcm2niix", "NIfTI", "DICOM"], "alternates": {"html": "https://wpnews.pro/news/my-mri-interpretation-vision-model-pointed-at-the-right-kidney-and-wrote-left", "markdown": "https://wpnews.pro/news/my-mri-interpretation-vision-model-pointed-at-the-right-kidney-and-wrote-left.md", "text": "https://wpnews.pro/news/my-mri-interpretation-vision-model-pointed-at-the-right-kidney-and-wrote-left.txt", "jsonld": "https://wpnews.pro/news/my-mri-interpretation-vision-model-pointed-at-the-right-kidney-and-wrote-left.jsonld"}}