Interpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking

Background

Machine learning is usually considered as a black box

First we let $\mathcal{X} \subseteq \mathbb{R}^n$ be the n-dimentional feature vector space.
So feature vector is represented as $\mathbf{x} = (x_1, x_2, ..., x_n)^T \in \mathcal{X}$ .
WLOG, the problem simplified as a binary classification problem, i.e. $\mathbf{y} = \{-1,+1\}$ .
Also, there's a function $f: \mathcal{X} \mapsto \mathbf{y}$ , i.e. classifier, and $\hat{f} \approx f$ as the classifier learned from data.
Here we assume $f$ to be an ensemble of K tree-based classifiers, i.e. $\hat{f} = \phi(\hat{h}_1,...,\hat{h}_k)$ .
WLOG, we assume $\phi$ to be the majority vote.

$p^+_{k,j}$ as the j-th path in tree $T_k$ .
For all $T_k \in T^-$ , we calculate the perturbed feature vector $x^+_{j(\epsilon)}$ , by
$x^+_{j(\epsilon)}[i] = \theta_i - \epsilon \text{ if the i-th condition is } (x_i \leq \theta_i)$ or,
$x^+_{j(\epsilon)}[i] = \theta_i + \epsilon \text{ if the i-th condition is } (x_i \gt \theta_i)$

= $140/90 + \epsilon$

For all trees, get positive paths $p^+_k$ .
Get perturbed instance $x^+_{j(\epsilon)}$ by $buildPositiveInst(x, p^+_{k,j}, \epsilon)$
Check if $\hat{f}(x^+_{j(\epsilon)}) = +1$ , and put the instance to a set $S$ if satisfied.
For each instances in $S$ , select the one with smallest $\delta(x, x^+_{j(\epsilon)})$ as $x'$