Statistical Aspects of SHAP: Functional ANOVA for Model Interpretation
Published in arXiv, 2022
Recommended citation: Herren, A., & Hahn, P. R. (2022). Statistical Aspects of SHAP: Functional ANOVA for Model Interpretation. arXiv preprint arXiv:2208.09970. https://arxiv.org/abs/2208.09970
SHAP (Lundberg and Lee 2017) is a popular tool for assessing feature importance in machine learning models. This paper looks at some of the statistical challenges that present themselves in estimating SHAP scores:
- How many synthetic samples to generate and pass through the model’s prediction function (and by what sampling scheme)
- How to choose a reference distribution for the averaging taking place in each of the synthetic samples
In investigating these questions, the paper discusses several connections with the sensitivity analysis and design of experiments literature, in particular:
- Functional ANOVA and the notion of effective dimensionality (Kucherenko et al 2009)
- Fractional factorial designs and the hypothesis of factor sparsity (Box and Meyer 1986)
The paper is available on Arxiv and code supporting the paper’s numeric examples is on Github.
References
George EP Box and R Daniel Meyer. An Analysis for Unreplicated Fractional Factorials. Technometrics, 28(1):11–18, 1986.
S Kucherenko, Maria Rodriguez-Fernandez, C Pantelides, and Nilay Shah. Monte Carlo Evaluation of Derivative-based Global Sensitivity Measures. Reliability Engineering & System Safety, 94(7):1135– 1148, 2009.
Scott M Lundberg and Su-In Lee. A Unified Approach to Interpreting Model Predictions. In Advances in neural information processing systems, pages 4765–4774, 2017.