Generalized Linear Models

A Generalization over linear models.

y = Bx + e

Need not be gaussian.

A link function to link E[y|x] -> Bx

  • u(x) = E[y|x]
  • g(u(x)) = Bx

Creating a GLM is simple!

  • Specify a Exponential family distribution.
  • Specify a link function.
  • You just created you GLM !

Modeling #people affected in an epidemic wrt time

  • Want to model y(t) where y(t) is the number of people affected on day t.
  • Since epidemic spreads quickly, E[y|t] = a*exp(d*t) seems a plausible choice of expected value of the count.

Reference : https://www.youtube.com/watch?v=X-ix97pw0xY

Modeling #people affected in an epidemic wrt time

  • E[y|t] = a*exp(d*t)
  • Want a function which converts E[y|t] to a linear combination in input t.
  • Use a log link function.
    • log(E[y|t]) = log(a) + d*t
  • Since we are dealing with counts, poisson seems an off the shelf choice for distribution of y.

Modeling #people affected in an epidemic wrt time

  • E[y|t] = a*exp(d*t)
  • Log link: log(E[y|t]) = log(a) + d*t
  • y~Poisson( with lambda = E[y|t])

We just created a GLM for our problem !

Modeling whether a child's spinal surgery would go right

  • Covariates/Inputs are
    • age
    • position in spine
    • number of vertebrae involved.

Modeling whether a child's spinal surgery would go right

  • y is boolean.
  • Distribution can only be Bernoulli !

Modeling whether a child's spinal surgery would go right

  • y~Bernauli(lambda).
  • For link function, we need a function [0,1]=> R
  • One choice could be inverse functions of CDFs.

How the theory works out?

Exponential family distributions

  • P(y|η) = b(y) *exp(η*T(y) - a(η))
    • ​Exponential * (fn of η) * (fn of y)
  • T(y) is what we want to predict. Hence, it is generally y.
  • Bernoulli, Poisson, Normal distributions etc are in exponential family.

Normal distribution as an exponential family distribution

  • P(y|η) = b(y) *exp(η*T(y) - a(η))
  • P(y;μ) = 1/√2π * exp( -(y-μ)^2 /2)
  • P(y;μ) = 1/√2π * exp( -y^2) * exp(y*μ -μ^2/2)
    • b(y) = 1/√2π * exp( -y^2)
    • T(y) = y
    • η = μ
    • a(η)  = η^2/2

GLM assumptions

  1. y|x;ϴ ~ Exponential family
    • P(y|η) = b(y) *exp(η*T(y) - a(η))
  2. Given x, we predict expected value of T(y)
    • h(x) = E[y|x]
  3. η = ϴ`x

Generalized Linear Models

By qplum

Generalized Linear Models

Linear Models are a powerful class of fully connected models. In trading, however, there are rules of thumb to follow when using GLM ( generalized linear models )

  • 1,573