• We usually assume the samples drawn from targeted population are independent and identically distributed (i.i.d.).

• This assumption does not hold when we have data with multilevel structure:
  - clustered and nested data (i.e. individuals within areas)
  - longitudinal data (i.e. repeated measurements within individuals)
  - non-nested structures (i.e. individuals within areas and belonging to some subgroups such as occupations)
   
• Samples within each group are dependent, while samples between groups stay independent
   
• Two sources of variations:
  - variations within groups
  - variations between groups

• A longitudinal study:
  - n = 3
  - t = 3
   
• Complete pooling
  - poor performance
   
• No pooling
  - infeasible for large n
   
• Partial pooling

• An alternative solution: include categorical individual indicators in the traditional linear regression model.
   
• Why do we still need mixed-effects models?

1. Account for both individual- and group-level variations when estimating group-level coefficients.
    
2. Easily model variations among individual-level coefficients, especially when making predictions for new groups.
    
3. Allow us to estimate coefficients for specific groups, even for groups with small n

Fixed and Random Effects

• Random Effects: varying coefficients
• Fixed Effects: varying coefficients that are not themselves modeled

Fixed and Random Effects

What's the difference between no-pooling models and mixed-effects models only with varying intercepts?

• In no-pooling models, the intercept is obtained by least squares estimates, which equals to the fitted intercepts in models that are run separately by group.
• In mixed-effects models, we assign a probability distribution to the random intercept: 

Pull the codes and dataset: https://github.com/benhhu/R-Mixed-Effects-Model

Load the Packages and Data

1,000 participants

5 repeated measurements

bmi

time

id

age

race: 1=white, 2=black, 3=others

gender: 1=male, 2=female

edu: 1=<HS, 2=HS, 3=>HS

sbp

am: 1=measured in morning

ex: #days exercised in the past year

Varying-intercept Model with No Predictors

Varying-intercept Model with an individual-level predictor

Varying-intercept Model with both individual-level and group-level predictors

Varying Slopes Models

With only an individual-level predictor

Varying Slopes Models

Add a group-level predictor

Non-nested Models

Mixed-Effects Logistic Model

Empty model

Mixed-Effects Logistic Model

Add bmi and race

Mixed-Effects Poisson Model

Parameter Estimation Algorithms

• ML: maximum likelihood
   
• REML: restricted maximum likelihood
  - default in lmer()
• PQL: pseudo- and penalized quasilikelihood
   
• Laplace approximations
  - default in glmer()
• GHQ: Gauss-Hermite quadrature
   
• McMC: Markov chain Monte Carlo

Bolker BM, Brooks ME, Clark CJ, Geange SW, Poulsen JR, Stevens MHH, et al. 2009. Generalized linear mixed models: A practical guide for ecology and evolution. Trends in ecology & evolution 24:127-135.

Mixed-Effects Model vs. GEE