Local

Global

Machine learning explanations

Black box

model

Data

Black box

model

Data

Data

Single

prediction

Entire

model

Local

Global

Black box

model

Data

Black box

model

Data

Data

Single

prediction

Entire

model

Machine learning explanations

• Which features played a role in the decision?

 

• Why did the model make this classification?

 

• What can I do to affect the prediction?

 

• What features does the
  model use in general?

 

• Does the model treat
  everyone fairly?

 

• Does it discriminate
  against groups?

ExplainExplore

Interactive visual explanation

Local

Global

Contribution-Value   Plots

Local

Global

ExplainExplore

Interactive visual explanation

StrategyAtlas

Local

Global

ExplainExplore

CV Plots

Interactive visual explanation

Local

Global

ExplainExplore

CV Plots

StrategyAtlas

Interactive visual explanation

Fraud detection explanations
for sick-leave insurances

Data

 

 

 

 

 

 

 

 

• Missing/incorrect values

Model

• 100 Random Forest
• 500 trees each
• ~25 decisions per tree
• 1.312.471 decisions total!

Real world scenario

My solution

[1] Palczewska, Anna et. al. Interpreting random forest classification models using a feature contribution method. In Integration of reusable systems, pp. 193–218. Springer, 2014.

0              1               2              3             x

y

2

1

 

7 : 7

Feature contribution

[2] Friedman, Jerome H. Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5): pp. 1189–1232, 2001. 

0%

100%

200

100

0

Duration illness

Company	ABC Inc
Employees	5
Duration illness	days
...	...

Partial dependence

[3] Ribeiro, Marco Tulio et. al. Why should i trust you?: Explaining the predictions of any classifier. In

Proceedings of the 22nd ACM SIGKDD, pp. 1135–1144. ACM, 2016.

[4] Deng, Houtao. Interpreting tree ensembles with inTrees. arXiv preprint arXiv:1408.5456 , pp. 1–18, 2014.

0              1               2              3             x

y

2

1

 

Local rule extraction

Fraud team happy! 🎉

Paper presented at:
Workshop on Human Interpretability in Machine Learning

Stockholm, Sweden

Questions?

Visual exploration of
machine learning explanations

Local

Global

Surrogate learning

0              1               2             3             x

y

2

1

 

Problem

Help data scientists to create and tune explanatory surrogate models.

• ← Any tabular data set

• ← Any Python classifier, or PMML

• ← Different surrogate models

Configuration view

• ← Surrogate fidelity: R2
• ← Prediction
• ← Feature contribution

• Shows values  or contribution →
• Line color = predicted class →
• Compare selected instance with data →
• Clusters indicate ‘strategies’ →

Feature view

Context view

Context view

Context view

Questions?

More info at

explaining.ml

Machine Learning Interpretability through Contribution-Value Plots

Local

Global

Pre-existing techniques as elementary building blocks.

Prediction (ŷ)

x

ŷ

The idea

Pre-existing techniques as elementary building blocks.

[1]    Friedman, J. H. "Greedy function approximation: a gradient boosting machine.", 2001.

Prediction (ŷ)

Local PDP [1]

Sensitivity

analysis

x

ŷ

The idea

Pre-existing techniques as elementary building blocks.

Prediction (ŷ)

Sensitivity

analysis

x

ŷ

[2]   Goldstein, A., et al. "Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation.", 2015.

Local PDP [1]

ICE plot [2]

Repeat

The idea

Pre-existing techniques as elementary building blocks.

[4]   Lundberg, S. M., et. al.. "A unified approach to interpreting model predictions.". 2017.

Feature 1

Feature 2

Feature 3

Feature 4

Feature 5

[3]   Ribeiro, M. T., et. al. ""Why should i trust you?" Explaining the predictions of any classifier.", 2016.

The idea

Pre-existing techniques as elementary building blocks.

Prediction (ŷ)

Sensitivity

analysis

x

ŷ

Local PDP [1]

ICE plot [2]

Repeat

The idea

Pre-existing techniques as elementary building blocks.

Prediction (ŷ)

Local PDP

ICE plot

Contribution (β)

Sensitivity

analysis

Repeat

x

ŷ

x

β

The idea

Pre-existing techniques as elementary building blocks.

Prediction (ŷ)

Local PDP

ICE plot

Contribution (β)

LCV plot

Sensitivity

analysis

Sensitivity

analysis

Repeat

x

ŷ

x

β

The idea

Pre-existing techniques as elementary building blocks.

Prediction (ŷ)

Local PDP

ICE plot

Contribution (β)

LCV plot

GCV plot

Sensitivity

analysis

Sensitivity

analysis

Repeat

Repeat

x

ŷ

x

β

The idea

Wine acidity (pH)

x

β

An exemplary data model exploration

Data

• 998 red wines
• 11 features

Model

• Random Forest (100 trees)
• Predict quality: 👍 / 👎

An exemplary data model exploration

x

β

x

β

An exemplary data model exploration

An exemplary data model exploration

Line fading .      .

x

β

Questions?

More info at

explaining.ml

Strategy analysis for
machine learning interpretability

Local

Global

How?

The basic principle

How?

The basic principle

What?

The full interface

What?

Method 1: Heat map cluster analysis

What?

Method 2: Density plots

Data

All data

Selection

What?

Method 3: Decision trees

Questions?

More info at

explaining.ml

Benefits of ML Explanations:
Improved Learning in AI-assisted tasks

Approach

Approach

Approach

TRIAL

TRIAL

TRIAL

TRIAL

TRIAL

TRIAL

TRIAL

TRIAL

AI advice removed!

AI advice

No AI advice

Results

Key takeaways

Users learn faster 

when supported by 

XAI explanations;

but there is much

less benefit for 

more difficult tasks. 

Results

Results