ExplainabilityAudit: An Automated Evaluation of Local Explainability in Rooftop Image Classification

Authors

Duleep Rathgamage Don, Kennesaw State UniversityFollow
Jonathan Boardman, Kennesaw State UniversityFollow
Sudhashree Sayenju, Kennesaw State UniversityFollow
Ramazan Aygun, Kennesaw State UniversityFollow
Yifan Zhang, Kennesaw State UniversityFollow
Bill Franks, Kennesaw State UniversityFollow
Sereres Johnston, The Travelers Companies, Inc.Follow
George Lee, The Travelers Companies, Inc.Follow
Dan Sullivan, The Travelers Companies, Inc.Follow
Girish Modgil, The Travelers Companies, Inc.Follow

Department

Statistics and Analytical Sciences

Document Type

Conference Proceeding

Submission Date

2022

Abstract

Explainable Artificial Intelligence (XAI) is a key concept in building trustworthy machine learning models. Local explainability methods seek to provide explanations for individual predictions. Usually, humans must check these explanations manually. When large numbers of predictions are being made, this approach does not scale. We address this deficiency for a rooftop classification problem specifically with ExplainabilityAudit, a method that automatically evaluates explanations generated by a local explainability toolkit and identifies rooftop images that require further auditing by a human expert. The proposed method utilizes explanations generated by the Local Interpretable Model-Agnostic Explanations (LIME) framework as the most important superpixels of each validation rooftop image during the prediction. Then a bag of image patches is extracted from the superpixels to determine their texture and evaluate the local explanations. Our results show that 95.7% of the cases to be audited are detected by the proposed system.