# CS6015: Linear Algebra and Random Processes

## Lecture 31:  Describing distributions compactly, Bernoulli distribution, Binomial distribution

### Can PMF be specified compactly?

p_X(x) = \begin{cases} \frac{1}{36} & if~x = 2 \\ \frac{2}{36} & if~x = 3 \\ \frac{3}{36} & if~x = 4 \\ \frac{4}{36} & if~x = 5 \\ \frac{5}{36} & if~x = 6 \\ \frac{6}{36} & if~x = 7 \\ \frac{5}{36} & if~x = 8 \\ \frac{4}{36} & if~x = 9 \\ \frac{3}{36} & if~x = 10 \\ \frac{2}{36} & if~x = 11 \\ \frac{1}{36} & if~x = 12 \\ \end{cases}

### can be tedious to enumerate when the support of X is large

\mathbb{R}_X = \{1, 2, 3, 4, 5, 6, \dots, \infty\}

### X: random variable indicating the number of tosses after which you observe the first heads

p_X(x) = \begin{cases} .. & if~x = 1 \\ .. & if~x = 2 \\ .. & if~x = 3 \\ .. & if~x = 4 \\ .. & if~x = 5 \\ .. & if~x = 6 \\ .. & .. \\ .. & .. \\ .. & if~x = \infty \\ \end{cases}
p_X(x) = (1-p)^{(x-1)}\cdot p

### Can PMF be specified compactly?

\mathbb{R}_X = \{1, 2, 3, 4, 5, 6, \dots, \infty\}

### X: random variable indicating the number of tosses after which you observe the first heads

p_X(x) = (1-p)^{(x-1)}\cdot p

### Can PMF be specified compactly?

\mathbb{R}_X = \{1, 2, 3, 4, 5, 6, \dots, \infty\}

### X: random variable indicating the number of tosses after which you observe the first heads

p_X(x) = (1-p)^{(x-1)}\cdot p

### the entire distribution can be specified by some parameters

P(label = cat | image) ?

p_X(x) = f(x)

\{0, 1\}
X: \Omega

\Omega:

\{0, 1\}
X: \Omega

\Omega:
A:

### event that the outcome is success

A:
Let~P(A) = P(success) = p
p_X(1) = p
p_X(0) = 1 - p
p_X(x) = p^x(1 - p)^{(1- x)}

\{0, 1\}
X: \Omega

### {failure, success}

\Omega:
p_X(x) \geq 0
\sum_{x \in\{0, 1\}}p_X(x) = p_X(0) + p_X(1)
= (1-p) + p = 1
\sum_{x \in\{0, 1\}}p_X(x) = 1 ?

(k \in [0, n])

p_X(x) = ?
X:

### random variable indicating the the number of successes in n trials

x \in \{0, 1, 2, 3, \dots, n\}

p_X(x) = ?
X:

### random variable indicating the the number of successes in n trials

x \in \{0, 1, 2, 3, \dots, n\}

p_X(x)

S, F

2^n~outcomes

### ... n times

TTT\\ TTH\\ THT\\ THH\\ HTT\\ HTH\\ HHT\\ HHH

### Example: n = 3, k = 1

\Omega
0\\ 1\\ 2\\ 3\\
X
A = \{HTT, THT, TTH\}
p_X(1) = P(A)
P(A) = P(\{HTT\}) \\+ P(\{THT\}) \\+P(\{TTH\})
P(\{HTT\}) = p(1-p)(1-p)
P(\{THT\}) = (1-p)p(1-p)
P(\{TTH\}) = (1-p)(1-p)p

### Example: n = 3, k = 1

A = \{HTT, THT, TTH\}
= 3 (1-p)^{(3-1)}p^1
p_X(1) = P(A) = 3 (1-p)^2p
= {3 \choose 1} (1-p)^{(3-1)}p^1

### Example: n = 3, k = 2

B = \{HTH, HHT, THH\}
= 3 (1-p)^{(3-2)}p^2
p_X(2) = P(B) = 3 (1-p)p^2
= {3 \choose 2} (1-p)^{(3-2)}p^2

n \choose k

n \choose k

p^k

(1-p)^{(n-k)}
n \choose k
p^k
(1-p)^{(n-k)}

n \choose k

p^k

(1-p)^{(n-k)}
n \choose k
p^k
(1-p)^{(n-k)}
p_X(k) =

p, n

### n = 50, p = 0.1

P(getting~infected) = P(at~least~one~success)
= 1 - P(0~successes)
= 1 - p_X(0)
= 1 - {50 \choose 0}p^0(1-p)^{50}
= 1 - 1*1*0.9^{50} = 0.9948

### n = 10, p = 0.1

P(getting~infected) = P(at~least~one~success)
= 1 - P(0~successes)
= 1 - p_X(0)
= 1 - {10 \choose 0}p^0(1-p)^{10}
= 1 - 1*1*0.9^{10} = 0.6513

### n = 10, p = 0.02

P(getting~infected) = P(at~least~one~success)
= 1 - P(0~successes)
= 1 - p_X(0)
= 1 - {10 \choose 0}p^0(1-p)^{10}
= 1 - 1*1*0.98^{10} = 0.1829

### n = 25, p =0.9, p = 0.5

import seaborn as sb
import numpy as np
from scipy.stats import binom

x = np.arange(0, 25)
n=25
p = 0.1

dist = binom(n, p)
ax = sb.barplot(x=x, y=dist.pmf(x))
n \choose k
p^k
(1-p)^{(n-k)}
p_X(k) =
2042975 * 0.1^9 *0.9^{16}
= 0.000378

### n = 25, p =0.9, p = 0.5

import seaborn as sb
import numpy as np
from scipy.stats import binom

x = np.arange(0, 25)
n=25
p = 0.1

dist = binom(n, p)
ax = sb.barplot(x=x, y=dist.pmf(x))
n \choose k
p^k
(1-p)^{(n-k)}
p_X(k) =
2042975 * 0.1^{16} *0.9^{9}
= 7.1*10^{-11}

### n = 25, p =0.9, p = 0.5

import seaborn as sb
import numpy as np
from scipy.stats import binom

x = np.arange(0, 25)
n=25
p = 0.1

dist = binom(n, p)
ax = sb.barplot(x=x, y=dist.pmf(x))
n \choose k
p^k
(1-p)^{(n-k)}
p_X(k) =

### Binomial Distribution

p_X(x) \geq 0
\sum_{i=0}^n p_X(i) = 1 ?

n \choose k
p^k
(1-p)^{(n-k)}
p_X(k) =

### Binomial Distribution

\sum_{i=0}^n p_X(i) = 1 ?

### ... n times

\sum_{i=0}^n p_X(i)
= p_X(0) + p_X(1) + p_X(2) + \cdots + p_X(n)
= {n \choose 0} p^0(1-p)^{n} + {n \choose 1} p^1(1-p)^{(n - 1)} + {n \choose 2} p^2(1-p)^{(n - 2)} + \dots {n \choose n} p^n(1-p)^{0}
(a+b)^n = {n \choose 0} a^0b^{n} + {n \choose 1} a^1b^{(n - 1)} + {n \choose 2} a^2b^{(n - 2)} + \dots {n \choose n} a^n(b)^{0}
a = p, b = 1- p

n \choose k
p^k
(1-p)^{(n-k)}
p_X(k) =

### Bernoulli

n = 1, k \in \{0, 1\}
p_X(0) = {1 \choose 0} p^0 (1-p)^1 = 1 - p
p_X(1) = {1 \choose 1} p^1 (1-p)^0 = p