{"slug": "multi-teacher-contrastive-distillation-for-edge-efficient-pathology-foundation", "title": "Multi-Teacher Contrastive Distillation for Edge-Efficient Pathology Foundation Models", "summary": "Researchers propose MuCoDi, a pretraining framework that distills knowledge from multiple large pathology foundation models into compact edge-oriented encoders, achieving near-teacher performance with orders-of-magnitude smaller model size. On a Raspberry Pi 5, sub-million-parameter students achieve up to 605-fold speedup over Virchow2 while retaining 66.5% to 66.9% external AUROC, enabling practical edge deployment of pathology AI.", "body_md": "arXiv:2607.05533v1 Announce Type: new\nAbstract: Computational pathology foundation models (PFMs) have advanced whole-slide image analysis. However, their size and inference cost hinder local deployment in pathology departments. We propose MuCoDi, a pretraining framework that distills frozen tile embeddings from multiple PFMs into compact edge-oriented encoders. Instead of regressing individual teacher features, MuCoDi trains lightweight MobileOne and RepViT students with a contrastive distillation objective adapted from MoCo v3, where cached Virchow2, UNI2, and H-Optimus-1 embeddings replace momentum-encoder keys. We pretrain students on 14.3M TCGA tiles from only 11.8K WSIs and evaluate frozen encoders on 23 clinically curated downstream classification tasks. RepViT-based MuCoEdge students retain near-teacher performance while reducing model size by orders of magnitude: MuCoEdge-R2.3 and MuCoEdge-R1.5 reach 71.0% external AUROC, within 0.8 percentage points of the best teacher (Virchow2, 71.8%), while MuCoEdge-R2.3 obtains the best external F1 and the second-best AUPRC (51.8% and 53.3%). MuCoEdge-R1.0 reaches 70.9% AUROC with only 6.4M parameters and 1.12 GFLOPs. On a Raspberry Pi 5, sub-million-parameter MobileOne students achieve up to 605-fold single-tile speedup over Virchow2 while retaining 66.5% to 66.9% external AUROC, demonstrating that PFM-quality pathology representations can be moved toward practical edge deployment. Code is available at https://anonymous.4open.science/r/mucodi-6243.", "url": "https://wpnews.pro/news/multi-teacher-contrastive-distillation-for-edge-efficient-pathology-foundation", "canonical_source": "https://arxiv.org/abs/2607.05533", "published_at": "2026-07-08 04:00:00+00:00", "updated_at": "2026-07-08 04:08:13.880055+00:00", "lang": "en", "topics": ["machine-learning", "computer-vision", "ai-infrastructure", "ai-research"], "entities": ["MuCoDi", "MobileOne", "RepViT", "Virchow2", "UNI2", "H-Optimus-1", "TCGA", "Raspberry Pi 5"], "alternates": {"html": "https://wpnews.pro/news/multi-teacher-contrastive-distillation-for-edge-efficient-pathology-foundation", "markdown": "https://wpnews.pro/news/multi-teacher-contrastive-distillation-for-edge-efficient-pathology-foundation.md", "text": "https://wpnews.pro/news/multi-teacher-contrastive-distillation-for-edge-efficient-pathology-foundation.txt", "jsonld": "https://wpnews.pro/news/multi-teacher-contrastive-distillation-for-edge-efficient-pathology-foundation.jsonld"}}