# Bayesian networks

## History

- created by Judea Pearl in 1985

2011 ACM Turing Award

- named after Thomas Bayes

uses Bayes' conditioning

- informal variant in 1913

## Motivation

- model of the problem domain probability distribution
- probability theory provides a consistent calculus
- intuitively interpretable
- handles missing values
- flexible applicability

## Conditional Independence

Two random variables *X* and *Y* are **conditionally independent** given a third random variable *Z* if and only if they are independent in their conditional probability distribution given *Z*.

## Bayesian network

Each node is conditionally independent of its nondescendants given its parents

## Chain rule

# Naive bayes

## Inference

- deriving logical conclusions from premises known or assumed to be true
- BNs have all necessary information
- can compute any subset of variables from any other
- generally NP-Hard
- exact inference
- Monte Carlo methods

## Inference

## Inference

# Demo

## Learning BN

- estimate parameters
- learn structure

## Estimating parameters

- Maximum likelihood for complete data
- EM for incomplete data

## Expectation maximization

- initialize parameters ignoring missing data
- repeat until model converges

E - calculate missing values using learned model

M - relearn model with new (computed) data

## Structure learning

- state space search
- score based
- initially no connections or expert made
- penalty for each connection
- must avoid cycles
- correlation between attributes, MAP
- using ML resluts in maximum network

## BNs and timeseries

- dynamic BNs
- each point in time, timeslice, is BN
- conditional dependencies between and within timeslices

## BNs and timeseries

## Applications of BNs

- victims identification
- oil exloration
- wireless 3G and 4G codecs
- spam filtering
- cancer risk modeling
- biomonitoring
- decision support systems

Using Bonaparte, all victims of the plane crashes in Tripoli (2010) and the Ukraine (MH17, 2014) were identified. In 2012 Bonaparte was used to solve a notorious 13 years old cold case (the Vaatstra Case). Recently Bonaparte was used to identify a serial rapist in Utrecht (2014).

Bayesian Network for Disaster Victim

Identification

Estimates the type of soil and the probability that it contains oil, gas or other valuable minerals, based on drilling measurements. The system is based on a Bayesian network where the probability computation is done using a Monte Carlo sampling method.

Used by Shell.

## References

- "Machine Learning by Pedro Domingos."
*Coursera*. University of Washington - F. V. Jensen, T. D. Nielsen, "Bayesian Networks and Decision Graphs", 8 Feb. 2007
- "Bayesian Network."
*Wikipedia*. Wikimedia Foundation, n.d. Web. 17 Apr. 2015 - Mihajlovic V, M Petkovic.
*Dynamic Bayesian Networks: A State Of The Art*. 1st ed. University of Twente, 2015

#### Bayesian networks

By Martin Barus