{"slug": "multimodal-skin-lesion-classification-with-swin-transformer-and-clinical-fusion", "title": "Multimodal Skin Lesion Classification with Swin Transformer and Clinical Metadata Fusion", "summary": "A new multimodal framework combining Swin Transformer image features with clinical metadata achieved 92.55% test accuracy and a 91.33% macro F1-score on a public dataset for skin lesion classification, outperforming unimodal approaches. The study, released on arXiv (2608.07574v1), applied temperature scaling to reduce expected calibration error and incorporated uncertainty estimation and explainability analysis, demonstrating a trustworthy approach for early skin cancer diagnosis.", "body_md": "arXiv:2608.07574v1 Announce Type: new\nAbstract: Skin lesion classification plays an important role in supporting the early diagnosis of skin cancer. However, automated analysis remains challenging due to class imbalance, inter-class similarity, and intra-class variability in dermoscopic images. This paper proposes a multimodal classification framework that combines Swin Transformer-based image features with structured clinical metadata to improve diagnostic performance through integrated visual-context learning. Experiments on a publicly available dataset show that the proposed model achieves a test accuracy of 92.55% and a macro F1-score of 91.33%, with strong performance across minority classes. Temperature scaling is applied as a post-hoc calibration method, resulting in a reduction in expected calibration error and improving prediction reliability, while uncertainty estimation is incorporated to further assess the confidence of model predictions. Qualitative explainability analysis further shows that the model focuses on lesion regions during inference. Therefore, the results demonstrate that multimodal fusion, combined with calibration and interpretability analysis, provides an effective and trustworthy approach for automated skin lesion classification.", "url": "https://wpnews.pro/news/multimodal-skin-lesion-classification-with-swin-transformer-and-clinical-fusion", "canonical_source": "https://arxiv.org/abs/2608.07574", "published_at": "2026-08-11 04:00:00+00:00", "updated_at": "2026-08-11 04:24:42.537156+00:00", "lang": "en", "topics": ["machine-learning", "computer-vision", "artificial-intelligence"], "entities": ["Swin Transformer", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/multimodal-skin-lesion-classification-with-swin-transformer-and-clinical-fusion", "markdown": "https://wpnews.pro/news/multimodal-skin-lesion-classification-with-swin-transformer-and-clinical-fusion.md", "text": "https://wpnews.pro/news/multimodal-skin-lesion-classification-with-swin-transformer-and-clinical-fusion.txt", "jsonld": "https://wpnews.pro/news/multimodal-skin-lesion-classification-with-swin-transformer-and-clinical-fusion.jsonld"}}