{"slug": "endoseg-what-we-built-what-we-didn-t-and-why-the-overlay-lies-a-little", "title": "EndoSeg: What We Built, What We Didn't, and Why the Overlay Lies a Little", "summary": "A developer built EndoSeg, an ovarian-lesion ultrasound segmenter, for the Nebius Serverless AI Builders Challenge. The model was fine-tuned on the MMOTU dataset and runs inference in the browser via ONNX Runtime Web, achieving a Dice score of 0.755 on the test set. However, the developer warns that the UI's clinical-sounding labels (e.g., 'Endometrioma / Chocolate Cyst') are based on shape heuristics, not the model's output, and the segmenter cannot diagnose endometriosis or distinguish lesion types.", "body_md": "EndoSeg was built for the Nebius Serverless AI Builders Challenge\n\nwe had fine-tune an ovarian-lesion ultrasound segmenter on Nebius Serverless Jobs, export it to ONNX, and run inference in the browser so the scan never leaves the user's device.\n\nThis post is the honest version of that story — including the gap between a green mask that looks clinical and a model that cannot tell you whether a patient has endometriosis.\n\n`Research / education only. EndoSeg is not a medical device. It does not diagnose endometriosis, score disease risk, or replace imaging review by a clinician`\n\n.\n\n```\n## The Short Verdict\n\n`| What the UI Can Show | What the Model Actually Outputs |\n| :--- | :--- |\n| Teal overlay, Dice, agreement %, \"max probability\" | A per-pixel lesion-vs-background mask |\n| \"Endometrioma / Chocolate Cyst\" style labels | *Nothing from the network* — shape heuristics in JS |\n| Cloud vs local \"agreement\" | Mask overlap between two segmenters, not diagnostic truth |\n\n> **Note:** There is no code`` path that computes P(endometriosis), healthy vs diseased, or endometrioma vs other cyst from learned class labels.`\n```\n\nThat wasn't accidental drift. The PRD scoped MVP as binary ovarian-lesion segmentation for Nebius's medical image segmentation track. Endometriosis is the motivation and the disclaimer — not the training objective. We planned a P1 endometrioma type-flag head; we never trained it. MMOTU's eight class IDs are discarded at preprocess time (mask > 0).\n\nSo if a doctor loads a healthy ovary and still sees a finding-like overlay: that's expected behavior of this segmenter, not a mystery bug.\n\n*Cloud → browser pipeline on Nebius*\n\n```\nMMOTU ultrasound\n    → Job N1 preprocess (CPU)\n    → Job N2 U-Net fine-tune (GPU, Dice+BCE)\n    → Job N3 ONNX export + parity check\n    → static browser app (ONNX Runtime Web, WebGPU → WASM)\n    → optional Endpoint + token-hiding proxy (\"Compare to cloud\")\n```\n\nPrivacy-by-design is real for the default path: model weights are fetched once and cached; local inference runs in a Web Worker. Images stay on-device unless the user explicitly clicks cloud compare.\n\nNumbers we will stand behind\n\nTrained on MMOTU (1,469 frames), 50 epochs:\n\nSplit Dice IoU\n\nVal (best @ epoch 33)\n\n0.764\n\n0.670\n\nTest\n\n0.755\n\n0.655\n\nLimitation: on 8 random test images, per-sample Dice ranged from 0.02 to 0.97. Medium/large lesions usually look right; small/thin ones sometimes get a confident \"typical blob.\" Aggregate Dice hides that.\n\nAlso: this MMOTU mirror lacks recoverable patient IDs, so the split is stratified by file — not strictly patient-disjoint.\n\nWhere the product language overshoots the model\n\nPixel confidence ≠ disease confidence. \"Mean confidence\" / \"Max probability\" are statistics over the sigmoid mask, not P(disease).\n\nClass IDs were thrown away. Preprocess binarizes masks. Class 5 (\"Normal ovary\") can still light up as foreground.\n\nType labels are heuristics. age-clinical.js maps mask geometry (Round / Unilocular → \"Endometrioma / Chocolate Cyst\"). That is not a classifier.\n\nAgreement / Dice vs cloud measures mask overlap — useful ONNX sanity check — not clinical correctness.\n\nWhat a doctor would need vs what they get\n\nClinician wants EndoSeg today\n\nEndometrioma vs other cyst vs normal\n\nNot from the model\n\nClear healthy / abnormal call\n\nSegmentation of \"something\"\n\nConfidence in a diagnosis\n\nPixel confidence + mask agreement\n\nLesion localization\n\nYes — this is the real MVP\n\nEven the unfinished type head would answer \"which MMOTU tumor type?\" — still not \"does this patient have endometriosis?\"\n\nWhy we're publishing this anyway\n\nFor the challenge: real Nebius Jobs + Endpoint usage, reproducible train → ONNX → browser path, privacy-default demo, honest failure modes.\n\nFor readers: do not let the UI narrate a classifier you never trained. A footer disclaimer does not cancel a chocolate-cyst label driven by circularity.\n\n```\ngit clone https://github.com/Gnana-Shishir-Kumar/nebius-challenge.git\ncd nebius-challenge\nnpx serve browser\n```\n\nFull pipeline: bash repro.sh. Configs under nebius/.\n\nIf we course-correct next\n\nStop collapsing MMOTU classes in preprocess.\n\nTrain a classification / multi-task head (endometrioma / other / normal).\n\nShow class probabilities — demote or remove shape→disease heuristics.\n\nKeep browser-private segmentation as localization, not silent diagnosis.\n\nTakeaway\n\nEndoSeg is aligned with the Nebius PRD: browser-private lesion segmentation, fine-tuned and optionally compared in the cloud.\n\nIt is not a disease-identification tool. The overlay answers where lesion-like pixels are. It does not answer whether this patient has endometriosis.\n\nRepo: [https://github.com/Gnana-Shishir-Kumar/nebius-challenge](https://github.com/Gnana-Shishir-Kumar/nebius-challenge)", "url": "https://wpnews.pro/news/endoseg-what-we-built-what-we-didn-t-and-why-the-overlay-lies-a-little", "canonical_source": "https://dev.to/gnanashishirkumar/endoseg-what-we-built-what-we-didnt-and-why-the-overlay-lies-a-little-35h9", "published_at": "2026-07-15 22:24:13+00:00", "updated_at": "2026-07-15 22:36:11.540493+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "computer-vision", "ai-products", "developer-tools"], "entities": ["Nebius", "MMOTU", "ONNX", "EndoSeg", "Gnana-Shishir-Kumar"], "alternates": {"html": "https://wpnews.pro/news/endoseg-what-we-built-what-we-didn-t-and-why-the-overlay-lies-a-little", "markdown": "https://wpnews.pro/news/endoseg-what-we-built-what-we-didn-t-and-why-the-overlay-lies-a-little.md", "text": "https://wpnews.pro/news/endoseg-what-we-built-what-we-didn-t-and-why-the-overlay-lies-a-little.txt", "jsonld": "https://wpnews.pro/news/endoseg-what-we-built-what-we-didn-t-and-why-the-overlay-lies-a-little.jsonld"}}