Bayesian Methods
A brief introduction to
T. Paternina
Machine Learning Journal Club
05/04/2017
Overview
- Quick reminder on probability
- Bayes' rule and HIV diagnosis
- Inference & comparing models
- Why Bayes'?
Quick reminder
P(A,B) = P(A)*P(B)
P(X=6) = {1 \over6}
P(X=6,Y=6) = {1 \over6}* {1 \over6}
***
Quick reminder
P(A \cup B) = P(A)+P(B)
P(X=6 \cup X=5) = {1 \over6} + {1 \over6}
***
or
Quick reminder
P(A)+P(B)=1
P(X\ne6) = 1- {1 \over6}
***
Bayes' rule
Bayes' rule
| Total | |||
|---|---|---|---|
| 43 | 9 | 52 | |
| 44 | 4 | 48 | |
| Total | 87 | 13 | 100 |
P(A|B) = {P(A,B) \over P(B)}
The probability of an event A conditioned by an event B is equal to the joint probability of A & B over the probability of B.
P(\space\space\space|\space\space\space) = {P(\space\space\space,\space\space\space) \over P(\space\space\space)}
={44/100\over 48/100} = {44\over48}
\approx 91.6\%
example from wikipedia (https://en.wikipedia.org/wiki/Contingency_table)
HIV diagnosis
HIV diagnosis
(U.S. military style)
(x2)
(x2)
(x2)
HIV+
HIV diagnosis
(U.S. military style)
P(HIV+|ELISA+)?
ELISA sensitivity (TP) = 93%
ELISA specificity (TN) = 99%
HIV+ prevalence = 1.48/1000
P(ELISA+|HIV+)
P(ELISA-|HIV-)
P(HIV+)
Posterior probability
Prior probability
P(HIV+)
P(HIV-)
Prior probability
P(+|HIV+)
P(-|HIV-)
P(-|HIV+)
P(+|HIV-)
P(A|B) = {P(A,B) \over P(B)}
P(HIV+|+) = {P(HIV+,+) \over P(+)}= {P(+|HIV+)P(HIV+)\over P(+,HIV+) +P(+,HIV-)}
0.00148
0.99852
0.93
0.07
0.99
0.01
= {0.00148 * 0.93 \over (0.00148*0.93)+ (0.99852*0.01)} \approx 0.12
Posterior probability
Updating probabilities
0.00148
0.12
1st ELISA
Prior probability
Updating probabilities
0.00148
0.12
1st ELISA
Prior probability
0.93
2n ELISA
Prior probability
Updating probabilities
0.00148
0.12
1st ELISA
0.93
2n ELISA
0.999
3rd ELISA
Prior probability
Prior probability
Prior probability
Bayesian Inference & comparing models
Effectiveness of contraception
40
20 standard
20 pill
9 months
16
4
p:
probability of a pregnancy coming from the treatment group.
| p | 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 |
|---|---|---|---|---|---|---|---|---|---|
| prior | 0.06 | 0.06 | 0.06 | 0.06 | 0.52 | 0.06 | 0.06 | 0.06 | 0.06 |
Testing several models
k:
number of pregnancies in the treatment group.
What's the value of p?
Computing likelihoods
Likelihood:
probability of the observed data given a particular model.
P(data|model)
P(k=4|n=20,p)
| p | 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 |
|---|---|---|---|---|---|---|---|---|---|
| prior | 0.06 | 0.06 | 0.06 | 0.06 | 0.52 | 0.06 | 0.06 | 0.06 | 0.06 |
| L | 0.0898 | 0.2182 | 0.1304 | 0.035 | 0.0046 | 0.0003 | 0 | 0 | 0 |
P(p|k=4,n=20)={L*P(p) \over P(k=4,n=20)}
={{L * P(p) \over \sum_{i} L_iP(p_i)}}
| p | 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 |
|---|---|---|---|---|---|---|---|---|---|
| prior | 0.06 | 0.06 | 0.06 | 0.06 | 0.52 | 0.06 | 0.06 | 0.06 | 0.06 |
| L | 0.0898 | 0.2182 | 0.1304 | 0.035 | 0.0046 | 0.0003 | 0 | 0 | 0 |
| post. | 0.1748 | 0.4248 | 0.2539 | 0.0681 | 0.0780 | 0.0005 | 0 | 0 | 0 |
Once you go Bayesian...
- Updating degree of belief as we gain information.
- Perform model parameter estimation from data.
- Quantify the level of uncertainty in a model.
References and resources
- Coursera course on Bayesian Statistics
- 2016 lecture by an expert on the field
- Website linking many resources
Bayesian Methods
By tpaternina
