missing data

4CE longitudinal lab values

workflow proposal

Step 0: Drop variable with high missing rate (> 80%: troponin_high, procalcitonin, fibrinogen, troponin_nomal

Step 1: Naively impute missing data points of each variable using functional PCA {fdapace}

Step 2: Drop rows with the most (originally) missing values, record the proportion of rows dropped for each patient (pdrop)

Step 3: Put NAs back in the CRP variable where it was missing.

Step 4: Train CRP on Leukocytes, Albumin and pdrop (mixed effect model, XGBoost, Amelia II) with available data

Step 5: Use the fitted model to predict the missing CRP values. 

Step 6: Repeat Steps 3–5 separately for each variable that has missing data (Leukocytes and Albumin).

CRP, Albumin, Leukocytes

for each cycle:

nRMSD(a) = \sqrt{\frac{\sum{_{p,i}}I_{p,a,i}\left(\frac{X_{p,a,i} - Y_{p,a,i}}{max(Y_{p,a}) - min(Y_{p,a})}\right)^2}{\sum_{p,i}I_{p,a,i}}}


patient \(p\)

lab \(a\)

time index \(i\)

mask one extra value per lab per patient

Amelia II



  • a proportion of patients would excluded
  • missing at random assumption

pattern of missingness?

other details

x_n = \frac{x-min(x)}{max(x) - min(x)}


Gaussian processes

\(f(t_i)\) have a joint Gaussian distribution


locality constraint

closer time points have more similar measurement values

cov(f(t_1), f(t_2)) = \alpha e^{-(t_1-t_2)^2/l}

Step 1: extract separate univariate time series for each patient and variable

Step 2: GPfit: MLE over \(\alpha\) and \(l\)

Step 3: infer values at unobserved time points

4CE missing data analysis

By Trang Le

4CE missing data analysis


  • 73
Loading comments...

More from Trang Le