Ishanu Chattopadhyay
University of Chicago
CCTS 40500 / CCTS 20500 / BIOS 29208
Winter 2023
Lecture 2
Contact
ishanu@uchicago.edu
900 E57 ST
KCBD 10152
Room: BSLC 313
Monday 9.30 - 12.20 AM
Resources
https://github.com/zeroknowledgediscovery/course_notes
RCC Midway
PLEASE CHECK if your RCC midway usernames work
I grade on progress and effort
Innovative approaches get more credit
There will be homeworks, not weekly but periodically
Midterm and finals are not in-class exams
Again, effort and innovation get more credit
Class Time
MON 9.30 AM (~3 hrs)
FRI 9.00 AM (0.5-1 hr)
Performance Metrics
Diagnostic Tests
Bayesian Statistics
Does the patient have the disorder?
Not Always Obvious
autism
dementia
Does the patient have risk of the disorder ?
Not Always Obvious
autism
dementia
How do we quantify risk?
How do we map risk to severity?
prevalence is intrinsic property of the disease
Manic Episode with no Bipolar history
prevalence: ~10%
Idiopathic Pulmonary Fibrosis
prevalence: ~0.5%
The decision threshold is upto us to decide
Impacts sensitivity & specificity
Each choice of a threshold produces a different test
Let sensitivity be \(s\), specificity be \(c\), and prevalence P/(N+P) be \(\wp\).
Then:
Hence, s=c is NO BETTER than a coin toss!
HW.
For 2 random samples, AUC is the probability that the positive sample is ranked higher than the negative one
Prove this using Bayes' Theorem
$$t_p/f_p$$
$$\frac{\rho}{1-\rho}$$
Cost | Positive | Negative |
---|---|---|
Test Positive | $0 | $x |
Test Negative | $y | $0 |
Cost Optimization to choose operating point
Criminal Justice: $$C(f_n) = 0 $$
Healthcare (Covid test?)
$$C(f_p) = 0 $$
naive dichotomy
Covid tests are similar
What happens if we test again?
0.045
0.045
1-0.045
0.69
But confirmatory tests might not be always feasible
(H)
HW: Show that the second expression is true
Note on beta distribution:
The Universal Metric
Also not computable
HW will be posted on canvas.