Inpainting Insights: Elevating Visual XAI with Photorealistic Perturbations

Researchers have adapted LIME, a popular perturbation-based XAI method, to use generative inpainting for creating photorealistic perturbed image samples, improving explanation quality by avoiding unrealistic out-of-distribution artifacts that can mislead models. The work, detailed in arXiv:2607.15482v1, addresses a key limitation of traditional perturbation techniques in image explainability.

arXiv:2607.15482v1 Announce Type: new Abstract: The increasing complexity of state-of-the-art machine learning models has made their behavior progressively harder to interpret, spurring rapid advancements in the field of eXplainable Artificial Intelligence XAI . Among many methods proposed, perturbation-based approaches play a major role. By systematically altering perturbing input features, these approaches measure the impact on the model's predictions. For image data, traditional perturbation techniques, often involve replacing pixel values e.g., with a pre-defined color. However, such approaches, but also more refined deterministic techniques, generate unrealistic out-of-distribution samples and often leave visible artifacts, which can mislead the model and compromise explanation quality. In this work, we adjust LIME, a widely used perturbation-based method, to demonstrate how generative inpainting can improve perturbation-based explanations for images. We achieve photorealistic perturbed samples that align better with the original data distribution and enhance explanation quality.