ishanu chattopadhyay
Asst Professsor
ishanu@uchicago.edu
VeRITAS AI
Detect adversarial responses ("lying") in structured interviews:
A Generative AI To Read Your Mind
VeRITAS
High Complexity
Low Surprise
Responses that are reflective of symptoms
structured interview
properties of true responses
1
2
3
Existing tools are unreliable, can be defeated with coaching, and have poor performance
Hidden structure of cross-talk between responses to interview items
PTSD diagnostic interview
Q-Net
Three parameters
Kolmogorov complexity
Surprise
Naive diagnostic Risk
304 VA participants with physician validated PTSD (malingering possibility not considered?)
310 online participants with no mental health diagnosis asked to intentionally malinger
~5% successfully beat the test
~89% of PTSD positive patients pass the test
complexity is high for "truthful" response patterns
"Positive" and "negative" Q-nets inferred from Case and Control cohort can be used as a diagnostic
surprise is low for
"truthful" response patterns
27 forensic psychiatrists recruited to take the challenge
~3.8% successfully beat the test
Identical Correlation Structure in three independent populations
complexity and surprise
score and diagnosis (estimate or ground truth)
Minimum AUC = \(0.95 \pm 0.005\)
Cannot be coached, or memorized
Number of possible responses
Minimum Performance (n=624)
Average Time: 3.5 min
No. of Items: 20
AUC > 0.95
PPV > 0.86
NPV > 0.92
At least 83.3% sensitivity at 94% specificity
Beat the test!
Substance Abuse Disorder
malingering
SUD
No SUD
Cook County Data
Estimated malingering rate 0.34
For \(i = 1, \ldots, n\), let \(P_i := P(X_i\,|\,X_j=x_j \text{ for } j \neq i)\) denote the conditional distribution of \(X_i\) given the values of the other components of \(X\).
Finally, for each \(i = 1, \ldots, n\), let \(\Phi^P_i\) denote an estimate of the distribution \(P_i\).
Then the set \(\Phi^P := \{\Phi^P_i\}_{i=1}^n\) is called a Quasinet (Qnet).
Q-nets
persistence function
q-distance function
Jensen-Shannon Divergence
q-distance
"physics" informed, adaptive distance between response vectors
smaller distances imply a quatitatively high probability of spontaneous jump
$$J \textrm{ is the Jensen-Shannon divergence }$$
Sanov's Theorem & Pinsker's Inequality
Theorem
Malingering Condition
Connection to Kolmogorov Complexity
The algorithmic complexity of a response \(x\) conditional on the number of survey items \(n\) is at most \(\kappa(x) + O(1) \).
Lemma 1
Lemma 2
DoD Applications