Random Forest
1. Bagging
2. Random subspace
3. Random combination
Session
Bagging
Bagging
D = (x_1, y_1), (x_2, y_2), (x_3, y_3), ..., (x_n, y_n)
Bagging
D = (x_1, y_1), (x_2, y_2), (x_3, y_3), ..., (x_n, y_n)
\tilde{\mathcal{D}}_1 = (x_1, y_1), ..., (x_n, y_n)
\tilde{\mathcal{D}}_2 = (x_3, y_3), ..., (x_n, y_n)
\tilde{\mathcal{D}}_3 = (x_1, y_1), (x_2, y_2), ...
\tilde{\mathcal{D}}_r = (x_2, y_2), ..., (x_n, y_n)
Random Sample
Bagging
\tilde{\mathcal{D}}_1 = (x_1, y_1), ..., (x_n, y_n)
\tilde{\mathcal{D}}_2 = (x_3, y_3), ..., (x_n, y_n)
\tilde{\mathcal{D}}_3 = (x_1, y_1), (x_2, y_2), ...
\tilde{\mathcal{D}}_r = (x_2, y_2), ..., (x_n, y_n)
Bagging
\tilde{\mathcal{D}}_2 = (x_3, y_3), ..., (x_n, y_n)
DTree(
) =
g_2
\tilde{\mathcal{D}}_1 = (x_1, y_1), ..., (x_n, y_n)
DTree(
) =
g_1
\tilde{\mathcal{D}}_3 = (x_1, y_1), (x_2, y_2), ...
DTree(
) =
g_3
\tilde{\mathcal{D}}_r = (x_2, y_2), ..., (x_n, y_n)
DTree(
) =
g_r
Bagging
g_2
g_1
g_3
g_r
x'
Classification
1
(0
0
0)
Mode
Regression
0.80
(0.20
0.19
0.21)
Avg
Bagging
Classification
Regression
0.80,
(0.20,
0.19,
0.21)
Avg
0.35
1,
(0,
0,
0)
Mode
0
Random subspace
Random subspace
D = (x_1, y_1), (x_2, y_2), (x_3, y_3), ..., (x_n, y_n)
(x_{i1}, x_{i2}, x_{i3}, ..., x_{id})
Random Sample
(x_{i1}, x_{i3})
Re Sample in each branch
Random combination
Random combination
\begin{matrix}
-0.0208 & -0.0721 & 0.7353 & 0.5336 & -0.5744\\
0.6013 & -0.5269 & -1.1291 & 1.6979 & -0.0146\\
-0.7770 & -2.4699 & 1.5543 & -0.1125 & 1.8515\\
\vdots & \vdots & \vdots & \vdots & \vdots\\
0.6690 & -1.2484 & -1.2613 & -0.1969 & 0.1639
\end{matrix} \in R_{d, n}
Sample from normal distribution
Random combination
\begin{matrix}
-0.0208 & -0.0721 & 0.7353 & 0.5336 & -0.5744\\
0.6013 & -0.5269 & -1.1291 & 1.6979 & -0.0146\\
-0.7770 & -2.4699 & 1.5543 & -0.1125 & 1.8515\\
\vdots & \vdots & \vdots & \vdots & \vdots\\
0.6690 & -1.2484 & -1.2613 & -0.1969 & 0.1639
\end{matrix} \in R_{d, n}
ramdom select p keep
p \ll d
\begin{matrix}
0.0 & -0.0721 & 0.0 & 0.0 & -0.5744\\
0.6013 & -0.5269 & 0.0 & 0.0 & 0.0\\
0.0 & 0.0 & 1.5543 & 0.0 & 1.8515\\
\vdots & \vdots & \vdots & \vdots & \vdots\\
0.6690 & 0.0 & -1.2613 & -0.1969 & 0.0
\end{matrix} \in R_{d, n}
Random combination
x_{i1}, x_{i2}, x_{i3}, ..., x_{id}
\begin{matrix}
0.0 & -0.0721 & 0.0 & 0.0 & -0.5744\\
0.6013 & -0.5269 & 0.0 & 0.0 & 0.0\\
0.0 & 0.0 & 1.5543 & 0.0 & 1.8515\\
\vdots & \vdots & \vdots & \vdots & \vdots\\
0.6690 & 0.0 & -1.2613 & -0.1969 & 0.0
\end{matrix} \in R_{d, n}
x'_{i1}
x'_{i2}
x'_{i3}
x'_{i4}
x'_{in}