Generalized Linear Models

A Generalization over linear models.

y = Bx + e

Need not be gaussian.

A link function to link E[y|x] -> Bx

• u(x) = E[y|x]
• g(u(x)) = Bx

Creating a GLM is simple!

• Specify a Exponential family distribution.
• Specify a link function.
• You just created you GLM !

Modeling #people affected in an epidemic wrt time

• Want to model y(t) where y(t) is the number of people affected on day t.
• Since epidemic spreads quickly, E[y|t] = a*exp(d*t) seems a plausible choice of expected value of the count.

Reference : https://www.youtube.com/watch?v=X-ix97pw0xY

Modeling #people affected in an epidemic wrt time

• E[y|t] = a*exp(d*t)
• Want a function which converts E[y|t] to a linear combination in input t.
• Use a log link function.
  • log(E[y|t]) = log(a) + d*t
• Since we are dealing with counts, poisson seems an off the shelf choice for distribution of y.

Modeling #people affected in an epidemic wrt time

• E[y|t] = a*exp(d*t)
• Log link: log(E[y|t]) = log(a) + d*t
• y~Poisson( with lambda = E[y|t])

We just created a GLM for our problem !

Modeling whether a child's spinal surgery would go right

• Covariates/Inputs are
  • age
  • position in spine
  • number of vertebrae involved.

Modeling whether a child's spinal surgery would go right

• y is boolean.
• Distribution can only be Bernoulli !

Modeling whether a child's spinal surgery would go right

• y~Bernauli(lambda).
• For link function, we need a function [0,1]=> R
• One choice could be inverse functions of CDFs.

How the theory works out?

Exponential family distributions

• P(y|η) = b(y) *exp(η*T(y) - a(η))
  • ​Exponential * (fn of η) * (fn of y)
• T(y) is what we want to predict. Hence, it is generally y.
• Bernoulli, Poisson, Normal distributions etc are in exponential family.

Normal distribution as an exponential family distribution

• P(y|η) = b(y) *exp(η*T(y) - a(η))
• P(y;μ) = 1/√2π * exp( -(y-μ)^2 /2)
• P(y;μ) = 1/√2π * exp( -y^2) * exp(y*μ -μ^2/2)
  • b(y) = 1/√2π * exp( -y^2)
  • T(y) = y
  • η = μ
  • a(η)  = η^2/2

GLM assumptions

1. y|x;ϴ ~ Exponential family
   • P(y|η) = b(y) *exp(η*T(y) - a(η))
2. Given x, we predict expected value of T(y)
   • h(x) = E[y|x]
3. η = ϴ`x