# CS6015: Linear Algebra and Random Processes

## Lecture 28:  Probability Space, Axioms of Probability, Designing Probability Functions

### Independence

\mathbb{P}: \mathcal{F} \rightarrow [0,1]
X: \mathcal{\Omega} \rightarrow \mathbb{R}

### Estimating Parameters

X_1, X_2, \dots, X_t,\dots

(ML)

P(A) = ?

### The axioms of probability

P(A) \geq 0~\forall A

P(\Omega) = 1

### If the events                              are mutually disjoint  then

A_1, A_2, \dots, A_n
= \sum_{i=1}^n P(A_i)
P(A_1\cup A_2 \cup \dots \cup A_n)

### The axioms of probability

= \sum_{i=1}^n P(A_i)
P(A_1\cup A_2 \cup \dots \cup A_n)

A_1
A_2
A_3
A_4
A_5
A_6

### Given

P(A_1),P(A_2),P(A_3),P(A_4),P(A_5),P(A_6)
B

C

### : the event that the outcome is

P(B) = P(A_1)+P(A_3)+P(A_5)
\geq 5
P(C) = P(A_5)+P(A_6)
D

### : the event that the outcome is a mult. of 3

P(D) = P(A_3)+P(A_6)

### Property 1:

P(A) = 1 - P(A^\mathsf{c})
A \cup A^\mathsf{c} = \Omega
1 = P(\Omega) = P(A \cup A^\mathsf{c}) = P(A) + P(A^\mathsf{c})
\therefore P(A) = 1 - P(A^\mathsf{c})

### Property 2:

P(A) \leq 1
P(A) = 1 - P(A^\mathsf{c})

### Property 3:

P(A \cup B) = P(A) + P(B) - P(A \cap B)
A^\mathsf{c}
P(A \cup B) = P(A \cup (B \cap A^\mathsf{c}))
= P(A) + P(B \cap A^\mathsf{c})
P(B) = P((B \cap A^\mathsf{c}) \cup (B \cap A))
= P(B\cap A^\mathsf{c}) + P(B\cap A)
\therefore P(B\cap A^\mathsf{c}) = P(B) - P(B\cap A)
= P(A) + P(B) - P(B\cap A)

### Property 4:

\Omega = A_1 \cup A_2 \cup \dots \cup A_n

### the sum of the probabilities of all outcomes is equal to 1

P(\Omega) = P(A_1 \cup A_2 \cup \dots \cup A_n) = \sum_{i=1}^n P(A_i)
\therefore \sum_{i=1}^n P(A_i) = 1
\underbrace{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
\Omega

### Property 5:

P(\phi) = 0
\therefore 1 = P(\Omega) = P(\Omega \cup \phi) = P(\Omega) + P(\phi)
\therefore 1 = 1 + P(\phi) \implies P(\phi) = 0

### Outcomes: {0,   1,   2,   3,   4,   5,   6}

A_0, A_1, A_2, A_3, A_4, A_5, A_6
P(A_0)=0.3, P(A_1)=0.45, P(A_2)=0.12, P(A_3)=0.02, \\ P(A_4)=0.07, P(A_5)=0.01, P(A_6)=0.03

### What is the prob. of scoring even no. of runs?

X = A_0 \cup A_2 \cup A_4 \cup A_6
P(X) = P(A_0 \cup A_2 \cup A_4 \cup A_6)
= P(A_0) + P(A_2) + P(A_4) + P(A_6)

### Outcomes: {0,   1,   2,   3,   4,   5,   6}

A_0, A_1, A_2, A_3, A_4, A_5, A_6
P(A_0)=0.3, P(A_1)=0.45, P(A_2)=0.12, P(A_3)=0.02, \\ P(A_4)=0.07, P(A_5)=0.01, P(A_6)=0.03

### What is the prob. of scoring less than 5 runs?

X = A_0 \cup A_1 \cup A_2 \cup A_3 \cup A_4
P(X) = P(A_0 \cup A_1 \cup A_2 \cup A_3 \cup A_4)
= P(A_0) + P(A_1) + P(A_2) + P(A_3) + P(A_4)

### Outcomes: {0,   1,   2,   3,   4,   5,   6}

A_0, A_1, A_2, A_3, A_4, A_5, A_6
P(A_0)=0.3, P(A_1)=0.45, P(A_2)=0.12, P(A_3)=0.02, \\ P(A_4)=0.07, P(A_5)=0.01, P(A_6)=0.03

### The prob. that the runs will be div. by 2 or 3?

X_1 = \{0, 2, 4, 6\} = A_0 \cup A_2 \cup A_4 \cup A_6
X_2 = \{0, 3, 6\} = A_0 \cup A_3 \cup A_6
X_1 \cap X_2 = \{0, 6\} = A_0 \cup A_6
P(X_1) = 0.3 + 0.12 + 0.07 + 0.03 = 0.52
P(X_2) = 0.3 + 0.02 + 0.03 = 0.35
P(X_1 \cap X_2) = 0.3 + 0.03 = 0.33
P(X_1 \cup X_2) = P(X_1) + P(X_2) - P(X_1 \cap X_2)
\therefore P (X_1 \cup X_2) = 0.52 + 0.35 - 0.33 = 0.54

### The ball bearings manufactured by a factory have two types of defects. Suppose the probability of having Type 1 defect is 0.01, having Type 2 defect is 0.02 and having both is 0.0005

P(A_1)=0.01, P(A_2)=0.03, P(A_1 \cap A_2) = 0.005
P(A_1 \cup A_2)= P(A_1) + P(A_2) - P(A_1 \cap A_2)

### What is the prob. that the bearing will have type 1 or type 2 defect ?

= 0.01 + 0.02 - 0.005 = 0.025
A_2
A_1

### The ball bearings manufactured by a factory have two types of defects. Suppose the probability of having Type 1 defect is 0.01, having Type 2 defect is 0.02 and having both is 0.0005

P(A_1)=0.01, P(A_2)=0.03, P(A_1 \cap A_2) = 0.005
P(A_1 \cup A_2)^\mathsf{c}= 1 - P(A_1 \cup A_2)

### What is the prob. that the bearing will have neither type 1 nor type 2 defect ?

= 1 - 0.025 = 0.975
A_2
A_1

### Karl Pearson tossed a coin 24000 times he observed that the number of heads was 12012

P(H) = \frac{12012}{24000} = 0.5005

### We can think of the probability of an event as the fraction of times the event occurs when an experiment is repeated a large number of times

P(A_i) = \frac{no.~of~times~the~outcome~is~in~A_i}{total~no.~of~times~the~experiment~was~repeated}

P(A_i) \geq 0~?

### Does P() satisfy the axioms?

P(\Omega)=1~?
P(\Omega) = \frac{no.~of~times~the~outcome~is~in~S}{total~no.~of~times~the~experiment~was~repeated} = 1
\Omega
P(A_1 \cup A_2) = P(A_1)+ P(A_2)~?
P(A_1 \cup A_2) = \frac{k_1 + k_2}{k} = \frac{k_1}{k} + \frac{k_2}{k}
= P(A_1)+ P(A_2)
A_1
A_2

### Does P() satisfy the axioms?

\Omega
P(A_1 \cup A_2) = P(A_1)+ P(A_2)~?
|\Omega| = n \implies 2^n~subsets \implies 2^n~events
A_1, A_2, A_3, \dots, A_n

### If the frequencies of                                       are known then the probability of any event can be computed

A_1, A_2, A_3, \dots, A_n

### Frequency of the event "forest": 15000

P(forest) = \frac{15000}{100000} = 0.15

### Frequency of the event "infected": 1 million

P(infected) = \frac{1000000}{20000000} = 0.05

Flu
\Omega

### Equally likely outcomes

P(H) = P(T) = k
\Omega = H \cup T
P(\Omega) = P(H \cup T)
= P(H) + P(T)
\Omega = \{H, T\}
H
T
= 2k
= 1
\therefore P(H) = P(T) = k = \frac{1}{2}

### We can now compute the probability of all 4 subsets of

\Omega
\phi, \{H\}, \{T\}, \{H, T\}

A_i

### : event that the out come is i

A_1, A_2, A_3, A_4, A_5, A_6

### partition

\Omega
P(A_1) = P(A_2) = P(A_3) = P(A_4) = P(A_5) = P(A_6) = k
P(S) = \sum_{i=1}^6 P(A_i) = 6k = 1
\therefore P(A_i) = \frac{1}{6}

### We can now compute the probability of all subsets of

\Omega
\Omega = \{1, 2, 3, 4, 5, 6\}
A_1
A_2
A_3
A_4
A_5
A_6

\Omega
E

### : outcome is even

\Omega = \{1, 2, 3, 4, 5, 6\}
A_1
A_2
A_3
A_4
A_5
A_6
O

D

E

### (the outcomes are of course disjoint)

P(E) = \sum_{i=1}^{k} \frac{1}{n} = \frac{k}{n}
\frac{1}{n}
P(X) = \frac{number~of~outcomes~in~X}{number~of~outcomes~in~\Omega}

### Are the axioms of probability satisfied?

\frac{1}{n}
P(X) = \frac{number~of~outcomes~in~X}{number~of~outcomes~in~\Omega}
P(A_i) \geq 0~?

### : ratio of two positive numbers

P(\Omega)=1~?
P(A_1 \cup A_2) = P(A_1)+ P(A_2)~?

### : contains all outcomes

A_1
A_2
P(A_1 \cup A_2) = \frac{k_1 + k_2}{n} = \frac{k_1}{n} + \frac{k_2}{n}
= P(A_1) + P(A_2)

### What is the probability of getting a black card?

\frac{1}{n}
P(B) = \frac{26}{52}
P(X) = \frac{number~of~outcomes~in~X}{number~of~outcomes~in~\Omega}

### What is the probability of getting 3 aces?

n = {52 \choose 3} = 22100
P(A) = \frac{4}{22100}

### What is the probability of hitting the red circle at the centre?

P(C) = \frac{\pi r^2}{\pi R^2} = (\frac{r}{R})^2
R

r

### Keep tossing the coin till you get the first head

\Omega = \{1,2,3,\dots\}

### Suppose

P(n) = \frac{1}{2^n}

### Is this a valid probability distribution?

P(n) \geq 0
P(\Omega) = \sum_{i=1}^{\infty}\frac{1}{2^n} = \frac{1}{2}\sum_{i=0}^{\infty} \frac{1}{2^i} = \frac{1}{2}\cdot\frac{1}{1 - \frac{1}{2}} = 1

### What about the third axiom?

P(outcome~is~even) = P(\{2\} \cup \{4\} \cup \{6\} \cup \dots )

### disjoint events

P(outcome~is~even) = P(2) + P(4) + P(6) + \dots
=\frac{1}{2^2} + \frac{1}{2^4} + \frac{1}{2^6} + \dots = \frac{1}{2^2}(1 + \frac{1}{4} + \frac{1}{4^2} + \dots) = \frac{1}{3}

### We state its correct version now which accounts for infinite (countable) events A1,A2,…AnA_1, A_2, \dots A_n

= \sum_{i=1}^\infty P(A_i)
P(A_1\cup A_2 \cup \dots )