Karl Ho
Data Generation datageneration.io
Knowledge Mining
Karl Ho
School of Economic, Political and Policy Sciences
University of Texas at Dallas
Interpretable Machine Learning (IML)
Introduction
Interpretable Machine Learning (IML) is a subfield of machine learning that focuses on developing, understanding, and analyzing models that generate comprehensible and meaningful explanations for their predictions.
- The primary goal of IML is to provide transparency and trust in the decision-making process of machine learning models, particularly in complex or high-stakes domains.
IML
-
Model Transparency
-
Model Interpretation
-
Feature Importance
-
Human Factors
-
Ethical Considerations
-
Evaluation
IML
1. Model Transparency: Creating models that are intrinsically interpretable, allowing humans to understand how input features contribute to the model's predictions. Examples include linear regression, decision trees, and rule-based models.
IML
2. Model Interpretation: Developing techniques to extract insights from complex, black box models, making their predictions more understandable. This includes model-agnostic methods like LIME, SHAP, and partial dependence plots, as well as model-specific techniques for deep learning, random forests, and other models.
IML
3. Feature Importance: Identifying and ranking the most important features in a model, providing insights into which variables have the most significant impact on predictions.
IML
4. Human Factors: Considering the human aspect of interpretability, such as cognitive biases, trust, and expertise, to ensure that explanations are useful and actionable for end-users.
IML
5. Ethical Considerations: Addressing fairness, accountability, and transparency concerns in machine learning models to avoid potential biases, discrimination, or unintended consequences.
IML
6. Evaluation: Developing methods to assess the quality and effectiveness of interpretable models and explanations, using both quantitative and qualitative approaches.
Molnar
-
Importance of interpretable machine learning
-
The Human Side of Interpretability
-
Model-Agnostic Methods
-
Model-Specific Methods
-
Rule-Based Models
-
Interpretability in Unsupervised Learning
-
Interpretable Models for Structured Data
-
Interpretability in Deep Learning
-
Evaluation of Interpretability
Partial Dependence Plot
Partial Dependence Plot (PDP) visualizes the relationship between a feature (or a set of features) and the predicted outcome of a machine learning model while averaging out the effects of all other features.
Partial Dependence Plot
Partial dependence plots help in understanding the marginal effect of a single feature or the interaction between multiple features on the model's predictions.Observe the x-axis, which represents the Petal.Length values.
The y-axis represents the average predicted probabilities for each class (setosa, versicolor, and virginica) based on the random forest model.
For each class, the line in the PDP shows how the predicted probability changes as the Petal.Length value increases or decreases.
Partial Dependence Plot
The rug plot at the bottom of the chart displays the distribution of Petal.Length values in the dataset.
Interpretation:
As the Petal.Length increases, the predicted probability for setosa (red line) decreases, while the probabilities for versicolor (green line) and virginica (blue line) increase.
For short petal lengths (around 1 to 2), the model predicts a high probability for the setosa class, whereas versicolor and virginica have low probabilities.
For intermediate petal lengths (around 3 to 5), the versicolor class has a higher probability compared to the other two classes.
Partial Dependence Plot
For longer petal lengths (greater than 5), the virginica class has the highest probability.
This PDP demonstrates that the Petal.Length feature has a significant impact on the random forest model's predictions for iris species. Shorter petal lengths are associated with setosa, intermediate lengths with versicolor, and longer lengths with virginica.
Partial Dependence Plot
Citation:
Friedman, Jerome H. 2001. "Greedy function approximation: A gradient boosting machine." Annals of Statistics, 29(5), 1189-1232.
Shapley values
Shapley values calculate the importance of a feature by comparing what a model predicts with and without the feature. However, since the order in which a model sees features can affect its predictions, this is done in every possible order, so that the features are fairly compared.
Shapley values
Shapley values
Source: Christoph Molnar
Why IML?
-
Interpretable machine learning plays a crucial role in real-world applications, particularly in fields like healthcare, finance, criminal justice, and other areas where decisions have significant implications for individuals and society.
-
By providing understandable explanations, IML helps build trust in machine learning models, supports human decision-making, and promotes responsible AI development.
Important works
- Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2018). A survey of methods for explaining black box models. ACM Computing Surveys (CSUR), 51(5), 1-42.
- Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why should I trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1135-1144).
- Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems (pp. 4765-4774).
- Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015). Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1721-1730).
Reference
Lundberg, Scott M. and Su-In Lee. 2017. A Unified Approach to Interpreting ModelPredictions (https://arxiv.org/pdf/1705.07874.pdf)
Molnar, Christoph. 2019. Interpretable machine learning. Lulu. com,
Knowledge Mining: Interpretable Machine Learning
By Karl Ho
