Sanity checks and improvements for patch visualisation in prototype-based image classification

Romain Xu-Darme (1, 2) , Georges Quénot (2) , Zakaria Chihani (1) , Marie-Christine Rousset (3)

1 LSL - Laboratoire Sûreté des Logiciels
2 MRIM - Modélisation et Recherche d’Information Multimédia [Grenoble]
3 SLIDE - ScaLable Information Discovery and Exploitation [Grenoble]

In this work, we perform an in-depth analysis of the visualisation methods implemented in two popular self-explaining models for visual classification based on prototypes - ProtoPNet and ProtoTree. Using two fine-grained datasets (CUB-200-2011 and Stanford Cars), we first show that such methods do not correctly identify the regions of interest inside of the images, and therefore do not reflect the model behaviour. Secondly, using a deletion metric, we demonstrate quantitatively that saliency methods such as Smoothgrads or PRP provide more faithful image patches. We also propose a new relevance metric based on the segmentation of the object provided in some datasets (e.g. CUB-200-2011) and show that the imprecise patch visualisations generated by ProtoPNet and ProtoTree can create a false sense of bias that can be mitigated by the use of more faithful methods. Finally, we discuss the implications of our findings for other prototype-based models sharing the same visualisation method.

Mots clés: Case-based Reasoning; Saliency Map; Evaluation metrics; Interpretability of AI

Domaines: Intelligence artificielle [cs.AI]

Romain Xu-Darme, Georges Quénot, Zakaria Chihani, Marie-Christine Rousset. Sanity checks and improvements for patch visualisation in prototype-based image classification. 2023. ⟨cea-04253851v1⟩