To transform or not to transform?
Rebecca Barter
Some context...
House price
$
$
House size
$
$
$
\textrm{House price} = \alpha + \beta \textrm{House size} + \epsilon
Some context...
House price
$
$
House size
$
$
$
\textrm{House price} = \alpha + \beta \textrm{House size} + \epsilon
Some context...
House price
$
$
log(House size)
$
$
$
log( )
log
( )
\textrm{House price} = \alpha + \beta \log(\textrm{House size}) + \epsilon
When do people usually transform their variables?
Poor model accuracy
Variables are not normal
Weird looking residual plot
Outliers are present
Relationship doesn't look linear
indicates your model assumptions are wrong
(if you care about inference)
If you can predict better with transformed variables, great!
(if you care about pred. accuracy)
Common misconception:
Variables in a regression don't actually need to be normal
...but the "errors" do!*
*Actually... we don't even need the errors to be normal.
They just need to be independent, have zero mean and common variance
Common transformations:
the logarithm
Interpreting log transforms:
increasing
Drawbacks:
can't be used for values
What it does:
compresses large vales and spreads small values
\beta
Y \sim X:
by
increases
by
X
1
Y
\log(Y) \sim \log(X):
multiplying
e^\beta
by
multiplies
by
X
Y
e
Y \sim \log(X):
multiplying
\beta
by
increases
by
X
Y
e
\log(Y) \sim X:
increasing
e^\beta
by
multiplies
by
X
Y
\textrm{1}
\leq 0
Common transformations:
the square-root
Interpreting log transforms:
increasing
Drawbacks:
can't be used for values
What it does:
compresses large vales and spreads small values
Y \sim X:
by
increases
by
X
1
Y
\sqrt{Y} \sim \sqrt{X}:
increasing
by
increases
by
X
Y
1
Y \sim \sqrt{X}:
increasing
by
increases
by
X
Y
1
\sqrt{Y} \sim X:
increasing
by
increases
by
X
Y
1
< 0
\beta
?
?
?
Common course of action
Run regression on the raw data
Compute R2
Transform variables because you think you can do better
Re-run regression with transformed variables
Re-compute R2
Repeat
Consequences of transformation
Incorrectly interpreting model coefficients
Multiple testing problems
To transform or not to transform?
Goal: predict life expectancy based on GDP per capita
\widehat{\textrm{Life expectancy}} = 54.0 + 0.000765 \times(\textrm{GDP per cap} )
R^2 = 0.34
\widehat{\textrm{Life expectancy}} = -9.1 + 8.4 \log(\textrm{GDP per cap} )
R^2 = 0.65
Goal: estimate effect of GDP on life expectancy
The bottom line is...
To transform or not to transform?
By Rebecca Barter
