Addressing Measurement Invariance in Scales for PTSD Using Nonlinear Latent Variable Models

Veronica T. Cole

Center for Developmental Science, UNC-Chapel Hill

Department of Psychology, Wake Forest University

Symposium Title
Confirmatory Factor Analysis Models of PTSD:
A Critique and Some Best Practice Suggestions

Agenda

• A brief introduction to measurement invariance
• Why should PTSD researchers care about measurement invariance?
  • General consequences
  • Measurement invariance in PTSD
• Modeling measurement using nonlinear factor models
• The Galveston Bay Recovery Study

• A brief introduction to measurement invariance
• Why should PTSD researchers care about measurement invariance?
  • General consequences
  • Measurement invariance in PTSD
• Modeling measurement using nonlinear factor models
• The Galveston Bay Recovery Study

$$y_{i1}$$

$$y_{i2}$$

$$y_{i3}$$

$$\eta_{i1}$$

$$y_{i4}$$

$$y_{i5}$$

$$y_{i6}$$

$$\eta_{i2}$$

Confirmatory factor analysis

A key assumption

When we measure a latent variable with a set of items, these items measure the latent variable equally well for all subjects.

HACK : WRITING ::

TRUANT : SCHOOL

THIEF : PROPERTY

MERCENARY : WARFARE

CRIMINAL : FELONY

DEFENDANT : ACCUSATION

Curley & Schmitt, 1993

Example: SAT Analogies

$$y_{i1}$$

$$y_{i2}$$

$$y_{i3}$$

$$\eta_{i1}$$

$$y_{i4}$$

$$y_{i5}$$

$$y_{i6}$$

$$\eta_{i2}$$

Configural measurement invariance

Meredith, 1993

$$y_{i1}$$

$$y_{i2}$$

$$y_{i3}$$

$$\eta_{i1}$$

$$y_{i4}$$

$$y_{i5}$$

$$y_{i6}$$

$$\eta_{i2}$$

Weak metric invariance

Meredith, 1993

$$y_{i1}$$

$$y_{i2}$$

$$y_{i3}$$

$$\eta_{i1}$$

$$y_{i4}$$

$$y_{i5}$$

$$y_{i6}$$

$$\eta_{i2}$$

Strong metric invariance

Meredith, 1993

• A brief introduction to measurement invariance
• Why should PTSD researchers care about measurement invariance?
  • General consequences
  • Measurement invariance in PTSD
• Modeling measurement using nonlinear factor models
• The Galveston Bay Recovery Study

(Why) do PTSD researchers care?

In order for the assumption of measurement invariance to be a concern for PTSD researchers, there needs to be evidence that...

• violations of this assumption undermine the validity of inferences in latent variable models
• this assumption is violated in current indices of PTSD

(Why) do PTSD researchers care?

In order for the assumption of measurement invariance to be a concern for PTSD researchers, there needs to be evidence that...

• violations of this assumption undermine the validity of inferences in latent variable models
• this assumption is violated in current indices of PTSD

Consequences of Violating MI Assumption: In Theory

In the absence of...

• configural invariance: n​o between-groups comparisons of any kind are possible.
• weak metric invariance: no between-groups comparisons of factor variances or covariances are possible.
• strong metric invariance: no between-groups comparisons of factor means are possible.

Meredith, 1993; Widaman & Reise, 1997; Vandenberg & Lance, 2000

Consequences of Violating MI Assumption: In Practice

• Configural invariance is absolutely necessary.
• Partial weak and strong metric non-invariance may be tolerable.
  • ​Equivocal findings about unmodeled effects
    • Factor score estimates are unbiased in some cases
    • More severe bias in regression coefficients​
  • We will discuss methods of directly accommodating measurement non-invariance

Byrne, Shavelson, & Muthén, 1989; Chalmers, 2016; Curran et al., 2016, 2018

(Why) do PTSD researchers care?

In order for the assumption of measurement invariance to be a concern for PTSD researchers, there needs to be evidence that...

• violations of this assumption undermine the validity of inferences in latent variable models
• this assumption is violated in current indices of PTSD

(Why) do PTSD researchers care?

In order for the assumption of measurement invariance to be a concern for PTSD researchers, there needs to be evidence that...

• violations of this assumption undermine the validity of inferences in latent variable models
• this assumption is violated in current indices of PTSD

Measurement Invariance in PTSD

• Males vs. females​
  • Minor non-invariance between male and female veterans
  • More substantial non-invariance across male and female survivors of a broader range of traumas
• Original vs. translated measures​
  • Some evidence that Spanish and English versions of PCL-C  show metric invariance
  • Minor differences in items, not consequential at scale level
• Older vs. younger
  • Particularly between age groups of children and adolescents

King et al., 1995; Chung & Breslau, 2008; Marshall, 2004; Orlando & Marshall, 2002; Miles et al., 2008; Contractor et al.,2013

Measurement Invariance in PTSD

• Military deployment
  • ​DTC is invariant across veterans from pre-Vietnam, Vietnam, and Iraq/Afghanistan
  • Weak metric invariance for PCL in deployed vs. non-deployed
• Country of origin​
  • ​Configural, but not metric, invariance of the HTC across asylum seekers from different global regions
  • General factor structure assessed in a few refugee samples
    • US-resettled refugees of Khmer Rouge
    • Refugees from civil war in Sierra Leone, 2001-2006

McDonald et al., 2008; Mansfield et al., 2010; Rasmussen et al., 2016; Palmieri et al., 2007; Vinson & Chang, 2012

• A brief introduction to measurement invariance
• Why should PTSD researchers care about measurement invariance?
  • General consequences
  • Measurement invariance in PTSD
• Modeling measurement using nonlinear factor models
• The Galveston Bay Recovery Study

$$y_{i1}$$

$$y_{i2}$$

$$y_{iJ}$$

...

$$\eta_{i}$$

$$x_{ip}$$

Directly modeling measurement non-invariance

Moderated nonlinear factor analysis (MNLFA) allows measurement parameters to be linear functions of the parameters, thus accommodating both impact and measurement non-invariance.

$$E\left(y_{ij}|\eta_i, x_{ip}\right) = g^{-1}\left(\lambda_{ij}\eta_i + \nu_{ij}\right)$$

$$\lambda_{ij} = \lambda_0 + \lambda_p x_{ip}$$

$$\nu_{ij} = \nu_0 + \nu_p x_{ip}$$

$$\eta_i \sim \left(\alpha_i,\psi_i\right)$$

$$\alpha_i = \alpha_0 + \alpha_p x_{ip}$$

Automated MNLFA

$$y_{i1}$$

$$y_{i2}$$

$$y_{iJ}$$

...

$$\eta_{i}$$

$$x_{i1}$$

• Used a data-driven search procedure to test each covariate for measurement non-invariance and impact.
• Procedure iterates through multiple models containing different combinations of measurement non-invariance
• Implemented via R package aMNLFA now on CRAN.

$$x_{i2}$$

.

.

.

$$x_{iP}$$

Automated MNLFA

$$y_{i1}$$

$$y_{i2}$$

$$y_{iJ}$$

...

$$\eta_{i}$$

$$x_{i1}$$

• Used a data-driven search procedure to test each covariate for measurement non-invariance and impact.
• Procedure iterates through multiple models containing different combinations of measurement non-invariance
• Implemented via R package aMNLFA now on CRAN.

$$x_{i2}$$

.

.

.

$$x_{iP}$$

Automated MNLFA

$$y_{i1}$$

$$y_{i2}$$

$$y_{iJ}$$

...

$$\eta_{i}$$

$$x_{i1}$$

• Used a data-driven search procedure to test each covariate for measurement non-invariance and impact.
• Procedure iterates through multiple models containing different combinations of measurement non-invariance
• Implemented via R package aMNLFA now on CRAN.

$$x_{i2}$$

.

.

.

$$x_{iP}$$

Automated MNLFA

$$y_{i1}$$

$$y_{i2}$$

$$y_{iJ}$$

...

$$\eta_{i}$$

$$x_{i1}$$

• Used a data-driven search procedure to test each covariate for measurement non-invariance and impact.
• Procedure iterates through multiple models containing different combinations of measurement non-invariance
• Implemented via R package aMNLFA now on CRAN.

$$x_{i2}$$

.

.

.

$$x_{iP}$$

Automated MNLFA

$$y_{i1}$$

$$y_{i2}$$

$$y_{iJ}$$

...

$$\eta_{i}$$

$$x_{i1}$$

• Used a data-driven search procedure to test each covariate for measurement non-invariance and impact.
• Procedure iterates through multiple models containing different combinations of measurement non-invariance
• Implemented via R package aMNLFA now on CRAN.

$$x_{i2}$$

.

.

.

$$x_{iP}$$

• A brief introduction to measurement invariance
• Why should PTSD researchers care about measurement invariance?
  • General consequences
  • Measurement invariance in PTSD
• Modeling measurement using nonlinear factor models
• The Galveston Bay Recovery Study

Galveston Bay Recovery Study

• Study of experiences following Hurricane Ike, which hit the Galveston Bay on 9/13/2008
• N = 656 adult residents of Galveston and Chambers Counties
• First measured between 2 and 5 months after the hurricane
• Demographics
  • 15.5% Black; 11.1% Hispanic/Latinx
  • 59.9% Female
  • 38.3% over 55

PCL-C Re-Experiencing

• The PTSD Checklist Civilian Version (PCL-C) was administered at all three assessments
  • Only considering the first time point here
  • PCL-C pertains to "most traumatic" experience of subject's life
• Here we consider the re-experiencing factor due to its relatively stable structure
• Items assessed how much a given symptom bothered subject
• Measured on a five-point scale
  • Not at all
  • A little bit
  • Moderately
  • Quite a bit
  • Extremely
• Final sample: N = 512

PCL-C Re-Experiencing

How much were you bothered by...

• Repeated, disturbing memories, thoughts, or images of this stressful experience?
• Repeated, disturbing dreams of this stressful experience
• Suddenly acting or feeling as if this stressful experience were happening again, as if you were reliving it?
• Feeling very upset when something reminded you of this stressful experience?
• Having physical reactions, for example, heart pounding, trouble breathing, sweating, when something reminded you of this stressful experience?

Analytic strategy

• Maximum likelihood for categorical outcomes
  • Standard fit statistics not available
  • Use WLSMV to test model without covariates
    • Unidimensional model with one residual covariance showed adequate fit, \(\chi^2(4) = 19.328, RMSEA = .086, CFI = .99, TLI = .99)\)
• Accounted for complex sampling
  • Subjects sampled from 77 segments in 5 strata, rated from most to least hurricane damage
  • Sampling weights used in all analyses
• Three covariates: self-reported gender (male vs. female); age group; race/ethnicity

Re-Experiencing

Final Model

Re-Experiencing

Age Non-Invariance: Memories

Gender Non-Invariance: Dreams

Conclusions

These results underscore an important rule about measurement non-invariance:

\[ \neq \]

statistically significant

practically meaningful

As in many of the prior studies, measurement non-invariance here likely would not cause scale scores to be particularly biased.

Conclusions

These results underscore an important rule about measurement non-invariance:

\[ \neq \]

statistically significant

practically meaningful

However...

• The age effect may be worthy of further investigation
• This group was largely homogeneous with regard to trauma type, so results may be stronger for others
• Potential confounding of memory and PTSD symptoms?
  • This could be an interesting problem for explanatory IRT.

Acknowledgements

Susan Ennett

Nisha Gottfredson

Andrea Hussong

Dan Bauer

Patrick Curran

Thank you!

colev@wfu.edu

vcole.web.unc.edu         github.com/vtcole