{"slug": "segment-what-it-can-draw-querying-a-text-to-image-generator-with-its-own-images", "title": "Segment What It Can Draw: Querying a Text-to-Image Generator with Its Own Images for Skin Lesion Segmentation", "summary": "A training-free method called GENIQ achieves Dice scores of 0.85-0.88 on three dermoscopy test sets for skin lesion segmentation, according to an arXiv paper (2611.07061v1). GENIQ builds initial skin and lesion prototypes from images generated by a general text-to-image diffusion transformer, then refines them using the most skin-like and lesion-like patches from unlabelled dermoscopy images, scoring test patches along the refined direction with a gated threshold to suppress spill-over around small lesions. The authors report GENIQ outperforms zero-shot and word-based attention methods on all three sets and a hand-designed dermoscopy pipeline in Dice and IoU on two, without inference-time annotations or additional models.", "body_md": "arXiv:2611.07061v1 Announce Type: new \nAbstract: Skin lesion segmentation without annotated masks requires the target to be specified through other cues. Existing approaches rely on dermoscopy-specific rules, annotation-derived model priors, or user-selected regions. A general text-to-image diffusion transformer offers a text interface to the target, but its concept-word attention is unreliable on dermoscopy images: in the model's attention-output space, the word-derived skin-to-lesion direction is poorly aligned with the direction separating real skin from real lesions. We show that the direction derived from generated healthy-skin and lesion images is substantially better aligned. Based on this finding, we propose GENIQ, a training-free method that constructs initial skin and lesion prototypes from generated images and refines them using the most skin-like and lesion-like patches from unlabelled dermoscopy images. It scores test patches along the refined direction and applies a gated threshold to suppress contextual spill-over around small lesions. Without inference-time annotations or additional models, GENIQ achieves Dice scores of 0.85-0.88 on three dermoscopy test sets. It outperforms zero-shot and word-based attention methods on all three sets, and a hand-designed dermoscopy pipeline in Dice and IoU on two.", "url": "https://wpnews.pro/news/segment-what-it-can-draw-querying-a-text-to-image-generator-with-its-own-images", "canonical_source": "https://arxiv.org/abs/2611.07061", "published_at": "2026-11-06 05:00:00+00:00", "updated_at": "2026-11-06 05:23:39.592897+00:00", "lang": "en", "topics": ["computer-vision", "generative-ai", "ai-research", "machine-learning", "artificial-intelligence"], "entities": ["GENIQ", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/segment-what-it-can-draw-querying-a-text-to-image-generator-with-its-own-images", "markdown": "https://wpnews.pro/news/segment-what-it-can-draw-querying-a-text-to-image-generator-with-its-own-images.md", "text": "https://wpnews.pro/news/segment-what-it-can-draw-querying-a-text-to-image-generator-with-its-own-images.txt", "jsonld": "https://wpnews.pro/news/segment-what-it-can-draw-querying-a-text-to-image-generator-with-its-own-images.jsonld"}}