Bayesian networks

History

   2011 ACM Turing Award

  • named after Thomas Bayes

    uses Bayes' conditioning

  • informal variant in 1913

      John Henry Wigmore

Motivation

  • model of the problem domain probability distribution
  • probability theory provides a consistent calculus
  • intuitively interpretable
  • handles missing values
  • flexible applicability

Conditional Independence

Two random variables X and Y are conditionally independent given a third random variable Z if and only if they are independent in their conditional probability distribution given Z.

P(X|Y,Z) = P(X|Z)
P(XY,Z)=P(XZ)

Bayesian network

Each node is conditionally independent of its nondescendants given its parents

Chain rule

P(X_1,X_2, ... ,X_n) = P(X_1)P(X_2|X_1)P(X_3|X_1,X_2) ... P(X_n|...)
P(X1,X2,...,Xn)=P(X1)P(X2X1)P(X3X1,X2)...P(Xn...)
P(X_1,X_2,...,X_n) = \prod(X_i| parents(X_i))
P(X1,X2,...,Xn)=(Xiparents(Xi))

Naive bayes

Inference

  • deriving logical conclusions from premises known or assumed to be true
  • BNs have all necessary information
  • can compute any subset of variables from any other
  • generally NP-Hard
  • exact inference
  • Monte Carlo methods

Inference

P(R=T|G=T) = ?
P(R=TG=T)=?

Inference

Demo

Learning BN

  • estimate parameters
  • learn structure

Estimating parameters

  • Maximum likelihood for complete data
  • EM for incomplete data

Expectation maximization

  • initialize parameters ignoring missing data
  • repeat until model converges

E  - calculate missing values using learned model

 

M - relearn model with new (computed) data

Structure learning

  • state space search
  • score based
  • initially no connections or expert made
  • penalty for each connection
  • must avoid cycles
  • correlation between attributes, MAP
  • using ML resluts in maximum network

BNs and timeseries

  • dynamic BNs
  • each point in time, timeslice, is BN
  • conditional dependencies between and within timeslices

BNs and timeseries

Applications of BNs

  • victims identification
  • oil exloration
  • wireless 3G and 4G codecs
  • spam filtering
  • cancer risk modeling
  • biomonitoring
  • decision support systems

Using Bonaparte, all victims of the plane crashes in Tripoli (2010) and the Ukraine (MH17, 2014) were identified. In 2012 Bonaparte was used to solve a notorious 13 years old cold case (the Vaatstra Case). Recently Bonaparte was used to identify a serial rapist in Utrecht (2014).

Bayesian Network for Disaster Victim
Identification

Estimates the type of soil and the probability that it contains oil, gas or other valuable minerals, based on drilling measurements. The system is based on a Bayesian network where the probability computation is done using a Monte Carlo sampling method.

 

Used by Shell.

References

Bayesian networks

By Martin Barus

Bayesian networks

  • 838