Generalized Linear Models
A Generalization over linear models.
y = Bx + e
Need not be gaussian.
A link function to link E[y|x] -> Bx
- u(x) = E[y|x]
- g(u(x)) = Bx
Creating a GLM is simple!
- Specify a Exponential family distribution.
- Specify a link function.
- You just created you GLM !
Modeling #people affected in an epidemic wrt time
- Want to model y(t) where y(t) is the number of people affected on day t.
- Since epidemic spreads quickly, E[y|t] = a*exp(d*t) seems a plausible choice of expected value of the count.
Reference : https://www.youtube.com/watch?v=X-ix97pw0xY
Modeling #people affected in an epidemic wrt time
- E[y|t] = a*exp(d*t)
- Want a function which converts E[y|t] to a linear combination in input t.
- Use a log link function.
- log(E[y|t]) = log(a) + d*t
- Since we are dealing with counts, poisson seems an off the shelf choice for distribution of y.
Modeling #people affected in an epidemic wrt time
- E[y|t] = a*exp(d*t)
- Log link: log(E[y|t]) = log(a) + d*t
- y~Poisson( with lambda = E[y|t])
We just created a GLM for our problem !
Modeling whether a child's spinal surgery would go right
- Covariates/Inputs are
- age
- position in spine
- number of vertebrae involved.
Modeling whether a child's spinal surgery would go right
- y is boolean.
- Distribution can only be Bernoulli !
Modeling whether a child's spinal surgery would go right
- y~Bernauli(lambda).
- For link function, we need a function [0,1]=> R
- One choice could be inverse functions of CDFs.
How the theory works out?
Exponential family distributions
- P(y|η) = b(y) *exp(η*T(y) - a(η))
- Exponential * (fn of η) * (fn of y)
- T(y) is what we want to predict. Hence, it is generally y.
- Bernoulli, Poisson, Normal distributions etc are in exponential family.
Normal distribution as an exponential family distribution
- P(y|η) = b(y) *exp(η*T(y) - a(η))
- P(y;μ) = 1/√2π * exp( -(y-μ)^2 /2)
- P(y;μ) = 1/√2π * exp( -y^2) * exp(y*μ -μ^2/2)
- b(y) = 1/√2π * exp( -y^2)
- T(y) = y
- η = μ
- a(η) = η^2/2
GLM assumptions
- y|x;ϴ ~ Exponential family
- P(y|η) = b(y) *exp(η*T(y) - a(η))
- Given x, we predict expected value of T(y)
- h(x) = E[y|x]
- η = ϴ`x
Generalized Linear Models