One Fourth Labs
We deliver courseware in AI and related areas
Recap: List of Topics
Descriptive Statistics
Probability Theory
Inferential Statistics
Different types of data
Different types of plots
Measures of centrality and spread
Counting, Sample spaces, events
Conditional Prob. (3 laws)
RVs, Expectation and Distributions
Sampling strategies
Distributions of Sampling Statistics
Hypothesis testing (z-test, t-test)
ANOVA, Chi-square test
Interval and Point Estimators
Learning Objectives
What are random variables?
What is a probability mass function ?
What are Expectation and Variance (and some of their properties) ?
What is a probability density function?
What is a normal distribution?
What are some standard probability mass functions?
Random Variable
Recap
Experiments, sample spaces, events
Axioms and Laws of Probability
This chapter: Focus on numerical quantities associated with the outcomes of experiments
Mapping outcomes to
(1,1)
(1,2)~(2,1)
(1,3)~(2,2)~(3,1)
(1,4)~(2,3)~(3,2)~(4,1)
(1,5)~(2,4)~(3,3)~(4,2)~(5,1)
(1,6)~(2,5)~(3,4)~(4,3)~(5,2)~(6,1)
(2,6)~(3,5)~(4,4)~(5,3)~(6,2)
(3,6)~(4,5)~(5,4)~(6,3)
(4,6)~(5,5)~(6,4)
(5,6)~(6,5)
(6,6)
\Omega
\mathbb{R}
In board games we care about the sum and not the outcomes which lead to the sum
2
3
4
5
6
7
8
9
10
11
12
\mathbb{R}
Q of Interest: What is the probability that the sum will be 10?
Mapping outcomes to
\Omega
\mathbb{R}
In board games we care about the sum and not the outcomes which lead to the sum
\mathbb{R}
Q of Interest: What is the probability that the sum will be 10?
\Omega
4
Q of Interest: What is the probability that a student's CGPA is 4.5?
Mapping outcomes to
\mathbb{R}
4.25
4.5
5
\mathbb{R}
CGPA
Height
Weight
Vit. D3
Age
\Omega
Qs of Interest:
What is the probability that an employee has 2 children?
What is the probability that an employee's monthly salary is greater than 50K?
Mapping outcomes to
\mathbb{R}
\mathbb{R}
Experiment: Randomly select an employee
: All employees of the organisation
: Number of years of experience, number of projects, salary, income tax, num. children
\Omega
Qs of Interest:
What is the probability that the size of the farm is less than 2 acres?
What is the probability that the total yield is greater than 1 ton?
Mapping outcomes to
\mathbb{R}
\mathbb{R}
Experiment: Randomly select a farm
: All farms in the state
: size of the farm, total yield, soil moisture, water content
Mapping outcomes to
\mathbb{R}
X: \Omega
\mathbb{R}
A random variable is a function from a set of possible outcomes to the set of real numbers
(could be a subset of R)
function
(random variable)
domain
range
Mapping outcomes to
\mathbb{R}
\Omega
\mathbb{R}
Multiple functions (random variables) are possible for the given domain (sample space)
students
(domain)
X_1: height
\mathbb{R}
\mathbb{R}
X_1: weight
X_1: CGPA
Notation
X (n_1, n_2)
: sum of the numbers on two dice (n1 + n2)
Y (student)
: height of the student
X
Y
Unlike functions we don't write brackets and arguments!
Questions
What are the values that the random variable can take?
What are the probabilities of the values that the random variable can take?
X
(discrete or continuous)
(we will return back to this later)
Types of random variables
Discrete
(finite or countably infinite)
the sum of the numbers on two dice
the outcome of a single die
the number of tosses after which a heads appears (countably infinite)
the number of children that an employee has
the number of cars in an image
Types of random variables
Continuous
the amount of rainfall in Chennai
the temperature of a surface
the density of a liquid
the height of a student
the haemoglobin level of a patient
Probability Mass Function
What are the probabilities of the values that the random variable can take?
Assigning probabilities
Q of Interest:Â What is the probability that the value of the random variable will be x?
P(X = x)~?
\forall x \in \{2,3,4,5,6,7,8,9,10,11,12\}
X: \Omega
\mathbb{R}
P(X=x)
[0,1]
\{2,3,4,5,6,7,8,9,10,11,12\}
An assignment of probabilities to all possible values that a discrete RV can take is called the distribution of the discrete random variable
Assigning probabilities
P(X = x)
\{1, 2,3,4,5,6\}
For discrete RVs we can think of the distribution as a table
X: \Omega
\mathbb{R}
P(X=x)
[0,1]
x
1
2
3
4
5
6
\frac{1}{6}
\frac{1}{6}
\frac{1}{6}
\frac{1}{6}
\frac{1}{6}
\frac{1}{6}
Assigning probabilities
\{2,3,4,5,6,7,8,9,10,11,12\}
X: \Omega
\mathbb{R}
P(X=x)
[0,1]
(1,1)
(1,2)~(2,1)
(1,3)~(2,2)~(3,1)
(1,4)~(2,3)~(3,2)~(4,1)
(1,5)~(2,4)~(3,3)~(4,2)~(5,1)
(1,6)~(2,5)~(3,4)~(4,3)~(5,2)~(6,1)
(2,6)~(3,5)~(4,4)~(5,3)~(6,2)
(3,6)~(4,5)~(5,4)~(6,3)
(4,6)~(5,5)~(6,4)
(5,6)~(6,5)
(6,6)
\Omega
2
3
4
5
6
7
8
9
10
11
12
x
Event: X = x
P(X = x)
\frac{1}{36}
\frac{2}{36}
\frac{3}{36}
\frac{4}{36}
\frac{5}{36}
\frac{6}{36}
\frac{5}{36}
\frac{4}{36}
\frac{3}{36}
\frac{2}{36}
\frac{1}{36}
Assigning probabilities
\{2,3,4,5,6,7,8,9,10,11,12\}
X: \Omega
\mathbb{R}
P(X=x)
[0,1]
2
3
4
5
6
7
8
9
10
11
12
x
P(X = x)
\frac{1}{36}
\frac{2}{36}
\frac{3}{36}
\frac{4}{36}
\frac{5}{36}
\frac{6}{36}
\frac{5}{36}
\frac{4}{36}
\frac{3}{36}
\frac{2}{36}
\frac{1}{36}
Assigning probabilities
\{2,3,4,5,6,7,8,9,10,11,12\}
X: \Omega
\mathbb{R}
p_X(x) = P(X=x) = P({\omega \in \Omega: X(\omega) = x})
[0,1]
Key idea:
Think of the event corresponding toÂ
X = x
Once we know this event (subset of sample space) we know how to compute P(X=x)Â
(Probability distribution of the random variable X)
P(X=x)
Probability Mass Function
p_{X}(x)
Probability Mass Function (PMF)
Probability Distribution
Distribution
Properties of a PMF
p_X(x) \geq 0
p_X(x) = P(X = x) = P(\{\omega \in \Omega: X(\omega) = x\} ) \geq 0
\sum_{x \in \mathbb{R}_X} p_X(x) = 1
\mathbb{R}_X \subset \mathbb{R}
(the set of values that the RV can take)
(the support of the RV)
p_X(x) \geq 0
\sum_{x \in \mathbb{R}_X} p_X(x) = 1
Properties of a PMF
\sum_{x \in \mathbb{R}_X} p_X(x) = 1
\sum_{x\in \mathbb{R}_X} p_X(x) = \sum_{x\in \mathbb{R}_X} P(X =x)
RHS is the sum of the probabilities of disjoint events which partitionÂ
\Omega
RHS sums to 1
\therefore
Proof:
Discrete distributions
Probability Mass Functions for discrete random variables
Recap
Random variables
Distribution of a random variable
An assignment of probabilities to all possible values that a discrete RV can take
(can be tedious even in simple cases)
Can PMF be specified compactly?
p_X(x) = \begin{cases} \frac{1}{36} & if~x = 2 \\ \frac{2}{36} & if~x = 3 \\ \frac{3}{36} & if~x = 4 \\ \frac{4}{36} & if~x = 5 \\ \frac{5}{36} & if~x = 6 \\ \frac{6}{36} & if~x = 7 \\ \frac{5}{36} & if~x = 8 \\ \frac{4}{36} & if~x = 9 \\ \frac{3}{36} & if~x = 10 \\ \frac{2}{36} & if~x = 11 \\ \frac{1}{36} & if~x = 12 \\ \end{cases}
can be tedious to enumerate when the support of X is large
\mathbb{R}_X = \{1, 2, 3, 4, 5, 6, \dots, \infty\}
X: random variable indicating the number of tosses after which you observe the first heads
p_X(x) = \begin{cases} .. & if~x = 1 \\ .. & if~x = 2 \\ .. & if~x = 3 \\ .. & if~x = 4 \\ .. & if~x = 5 \\ .. & if~x = 6 \\ .. & .. \\ .. & .. \\ .. & if~x = \infty \\ \end{cases}
p_X(x) = (1-p)^{(x-1)}\cdot p
compact
easy to compute
no enumeration needed
but ... ....
Can PMF be specified compactly?
p: probability~of~heads
\mathbb{R}_X = \{1, 2, 3, 4, 5, 6, \dots, \infty\}
X: random variable indicating the number of tosses after which you observe the first heads
p_X(x) = (1-p)^{(x-1)}\cdot p
How did we arrive at the above formula?
Is it a valid PMF (satisfying propertied of a PMF)
What is the intuition behind it?
(we will return back to these Qs later)
Can PMF be specified compactly?
\mathbb{R}_X = \{1, 2, 3, 4, 5, 6, \dots, \infty\}
X: random variable indicating the number of tosses after which you observe the first heads
p_X(x) = (1-p)^{(x-1)}\cdot p
For now, the key point is
it is desirable to have the entire distribution be specified by one or few parameters
Can PMF be specified compactly?
Why is this important?
the entire distribution can be specified by some parameters
P(label = cat | image) ?
cat? dog? owl? lion?
p_X(x) = f(x)
A very complex function whose parameters are learnt from data!
Bernoulli Distribution
Experiments with only two outcomes
Outcome: {positive, negative}
Outcome: {pass, fail}
Outcome: {hit, flop}
Outcome: {spam, not spam}
Outcome: {approved, denied}
\{0, 1\}
X: \Omega
Bernoulli Random Variable
 {failure, success}
\Omega:
Bernoulli trials
Bernoulli Distribution
\{0, 1\}
X: \Omega
Bernoulli Random Variable
 {failure, success}
\Omega:
A:
event that the outcome is success
A:
Let~P(A) = P(success) = p
p_X(1) = p
p_X(0) = 1 - p
p_X(x) = p^x(1 - p)^{(1- x)}
Bernoulli Distribution
\{0, 1\}
X: \Omega
Bernoulli Random Variable
 {failure, success}
\Omega:
p_X(x) \geq 0
\sum_{x \in\{0, 1\}}p_X(x) = p_X(0) + p_X(1)
= (1-p) + p = 1
\sum_{x \in\{0, 1\}}p_X(x) = 1 ?
Is Bernoulli distribution a valid distribution?
Binomial Distribution
Repeat a Bernoulli trial n times
independent
identical
(success/failure in one trial does not affect the outcome of other trials)
(probability of success 'p' in each trial is the same)
... n times
What is the probability of k successes in n trials?
(k \in [0, n])
Binomial Distribution (Examples)
Each ball bearing produced in a factory independently non-defective with probability p
... n times
If you select n ball bearings what is the probability that k of them will be defective?
Binomial Distribution (Examples)
The probability that a customer purchases something from your website is p
... n times
What is the probability that k out of the n customers will purchase something?
Assumption 1 : customers are identical (economic strata, interests, needs, etc)
Assumption 2 : customers are independent (one's decision does not influence another)
Binomial Distribution (Examples)
Marketing agency: The probability that a customer opens your email is p
... n times
If you send n emails what is the probability that the customer will open at least one of them?
Binomial Distribution
... n times
p_X(x) = ?
X:
random variable indicating the the number of successes in n trials
x \in \{0, 1, 2, 3, \dots, n\}
Challenge:Â
n and k can be very largeÂ
difficlult to enumerate all probabilities
Binomial Distribution
... n times
p_X(x) = ?
X:
random variable indicating the the number of successes in n trials
x \in \{0, 1, 2, 3, \dots, n\}
Desired:
Fully specify       in terms of n and p
p_X(x)
(we will see how to do this)
Binomial Distribution
... n times
S, F
How many different outcomes can we have if we repeat a Bernoulli trial n times?
(sequence of length n from a given set of 2 objects)
2^n~outcomes
Binomial Distribution
... n times
TTT\\
TTH\\
THT\\
THH\\
HTT\\
HTH\\
HHT\\
HHH
Example: n = 3, k = 1
\Omega
0\\
1\\
2\\
3\\
X
A = \{HTT, THT, TTH\}
p_X(1) = P(A)
P(A) = P(\{HTT\}) \\+ P(\{THT\}) \\+P(\{TTH\})
P(\{HTT\}) = p(1-p)(1-p)
P(\{THT\}) = (1-p)p(1-p)
P(\{TTH\}) = (1-p)(1-p)p
Binomial Distribution
... n times
Example: n = 3, k = 1
A = \{HTT, THT, TTH\}
= 3 (1-p)^{(3-1)}p^1
p_X(1) = P(A) = 3 (1-p)^2p
= {3 \choose 1} (1-p)^{(3-1)}p^1
Binomial Distribution
... n times
Example: n = 3, k = 2
B = \{HTH, HHT, THH\}
= 3 (1-p)^{(3-2)}p^2
p_X(2) = P(B) = 3 (1-p)p^2
= {3 \choose 2} (1-p)^{(3-2)}p^2
Binomial Distribution
... n times
Observations
n \choose k
favorable outcomes
each of the k successes occur independently with a probability p
each of the n-k failures occur independently with a probability 1 - p
terms in the summation
n \choose k
each term will have the factor
p^k
each term will have the factor
(1-p)^{(n-k)}
n \choose k
p^k
(1-p)^{(n-k)}
Binomial Distribution
... n times
terms in the summation
n \choose k
each term will have the factor
p^k
each term will have the factor
(1-p)^{(n-k)}
n \choose k
p^k
(1-p)^{(n-k)}
p_X(k) =
Parameters:
p, n
the entire distribution is full specified once the values of p and n are known
Example 1: Social distancing
... n times
Suppose 10% of your colleagues from workplace are infected with COVID-19 but are asymptomatic (hence come to office as usual)
Suppose you come in close proximity of 50 of your colleagues. What is the probability of you getting infected?
(Assume you will get infected if you come in close proximity of a person)
Trial: Come in close proximity of a person
p = 0.1Â - probability of success/infection in a single trial
n = 50 trials
Example 1: Social distancing
... n times
Suppose 10% of your colleagues from workplace are infected with COVID-19 but are asymptomatic (hence come to office as usual)
Suppose you come in close proximity of 50 of your colleagues. What is the probability of you getting infected?
n = 50, p = 0.1
P(getting~infected) = P(at~least~one~success)
= 1 - P(0~successes)
= 1 - p_X(0)
= 1 - {50 \choose 0}p^0(1-p)^{50}
= 1 - 1*1*0.9^{50} = 0.9948
Stay at home!!
Example 1: Social distancing
... n times
Suppose 10% of your colleagues from workplace are infected with COVID-19 but are asymptomatic (hence come to office as usual)
What if you interact with only 10 colleagues instead of 50
n = 10, p = 0.1
P(getting~infected) = P(at~least~one~success)
= 1 - P(0~successes)
= 1 - p_X(0)
= 1 - {10 \choose 0}p^0(1-p)^{10}
= 1 - 1*1*0.9^{10} = 0.6513
Still stay at home!!
Example 1: Social distancing
... n times
Suppose 10% of your colleagues from workplace are infected with COVID-19 but are asymptomatic (hence come to office as usual)
What if only 2% of your colleagues are infected instead of 10% ?
n = 10, p = 0.02
P(getting~infected) = P(at~least~one~success)
= 1 - P(0~successes)
= 1 - p_X(0)
= 1 - {10 \choose 0}p^0(1-p)^{10}
= 1 - 1*1*0.98^{10} = 0.1829
Perhaps, still not worth taking a chance!!
Example 2: Linux users
... n times
10% of students in your class use linux. If you select 25 students at random
(a) What is the probability that exactly 3 of them are using linux
(b) What is the probability that between 2 to 6 of them are using linux ?
(c) How would the above probabilities change if instead of 10%, 90% were using linux ?
n = 25, p =0.1, k = 3
n = 25, p =0.1,      k = {2,3,4,5,6}
n = 25, p =0.9,     p = 0.5
Example 2: Linux users
... n times
10% of students in your class use linux. If you select 25 students at random
n = 25, p =0.1, k = 3
n = 25, p =0.1, k = {2,3,4,5,6}
n = 25, p =0.9, p = 0.5
import seaborn as sb
import numpy as np
from scipy.stats import binom
x = np.arange(0, 25)
n=25
p = 0.1
dist = binom(n, p)
ax = sb.barplot(x=x, y=dist.pmf(x))
n \choose k
p^k
(1-p)^{(n-k)}
p_X(k) =
2042975 * 0.1^9 *0.9^{16}
= 0.000378
Example 2: Linux users
... n times
10% of students in your class use linux. If you select 25 students at random
n = 25, p =0.1, k = 3
n = 25, p =0.1, k = {2,3,4,5,6}
n = 25, p =0.9, p = 0.5
import seaborn as sb
import numpy as np
from scipy.stats import binom
x = np.arange(0, 25)
n=25
p = 0.1
dist = binom(n, p)
ax = sb.barplot(x=x, y=dist.pmf(x))n \choose k
p^k
(1-p)^{(n-k)}
p_X(k) =
2042975 * 0.1^{16} *0.9^{9}
= 7.1*10^{-11}
Example 2: Linux users
... n times
10% of students in your class use linux. If you select 25 students at random
n = 25, p =0.1, k = 3
n = 25, p =0.1, k = {2,3,4,5,6}
n = 25, p =0.9, p = 0.5
import seaborn as sb
import numpy as np
from scipy.stats import binom
x = np.arange(0, 25)
n=25
p = 0.1
dist = binom(n, p)
ax = sb.barplot(x=x, y=dist.pmf(x))n \choose k
p^k
(1-p)^{(n-k)}
p_X(k) =
Binomial Distribution
p_X(x) \geq 0
\sum_{i=0}^n p_X(i) = 1 ?
Is Binomial distribution a valid distribution?
... n times
n \choose k
p^k
(1-p)^{(n-k)}
p_X(k) =
Binomial Distribution
\sum_{i=0}^n p_X(i) = 1 ?
... n times
\sum_{i=0}^n p_X(i)
= p_X(0) + p_X(1) + p_X(2) + \cdots + p_X(n)
= {n \choose 0} p^0(1-p)^{n} + {n \choose 1} p^1(1-p)^{(n - 1)} + {n \choose 2} p^2(1-p)^{(n - 2)} + \dots {n \choose n} p^n(1-p)^{0}
(a+b)^n = {n \choose 0} a^0b^{n} + {n \choose 1} a^1b^{(n - 1)} + {n \choose 2} a^2b^{(n - 2)} + \dots {n \choose n} a^n(b)^{0}
a = p, b = 1- p
Bernoulli (a special case of Binomial)
... n times
n \choose k
p^k
(1-p)^{(n-k)}
p_X(k) =
Binomial
Bernoulli
n = 1, k \in \{0, 1\}
p_X(0) = {1 \choose 0} p^0 (1-p)^1 = 1 - p
p_X(1) = {1 \choose 1} p^1 (1-p)^0 = p
Geometric Distribution
\dots \infty~times
The number of tosses until we see the first heads
X:
\mathbb{R}_X = \{1,2,3,4,5, \dots\}
p_X(x) =?
Why would we be interested in such a distribution ?
Geometric Distribution
\dots \infty~times
Hawker selling belts outside a subway station
Why would we be interested in such a distribution ?
Salesman handing pamphlets to passersby
(chance that the first belt will be sold after k trials)
(chance that the k-th person will be the first person to actually read the pamphlet)
A digital marketing agency sending emails
(chance that the k-th person will be the first person to actually read the email)
Useful in any situation involving "waiting times"
independent trials
identical distribution
P(success) = p
Geometric Distribution
\dots \infty~times
Example: k = 5
p_X(5)
F F F F S
P(success) = p
(1-p)
(1-p)
(1-p)
(1-p)
p
\underbrace{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}_{(5-1)}
\underbrace{}_{1}
=(1-p)^{(5-1)}p
p_X(k)=(1-p)^{(k-1)}p
Geometric Distribution
\dots \infty~times
p=0.2
P(success) = p
import seaborn as sb
import numpy as np
from scipy.stats import geom
x = np.arange(0, 25)
p = 0.2
dist = geom(p)
ax = sb.barplot(x=x, y=dist.pmf(x))Geometric Distribution
\dots \infty~times
p=0.9
P(success) = p
import seaborn as sb
import numpy as np
from scipy.stats import geom
x = np.arange(0, 25)
p = 0.9
dist = geom(p)
ax = sb.barplot(x=x, y=dist.pmf(x))Geometric Distribution
\dots \infty~times
P(success) = p
p=0.5
import seaborn as sb
import numpy as np
from scipy.stats import geom
x = np.arange(0, 25)
p = 0.5
dist = geom(p)
ax = sb.barplot(x=x, y=dist.pmf(x))
p_X(k)=(1-p)^{(k-1)}p
p_X(k)=(0.5)^{(k-1)}0.5
p_X(k)=(0.5)^{k}
Geometric Distribution
p_X(x) \geq 0
\sum_{k=1}^\infty p_X(i) = 1 ?
Is Geometric distribution a valid distribution?
p_X(k) = (1 - p)^{(k-1)}p
P(success) = p
= (1 - p)^{0}p + (1 - p)^{1}p + (1 - p)^{2}p + \dots
= \sum_{k=0}^\infty (1 - p)^{k}p
= \frac{p}{1 - (1 - p)} = 1
a, ar, ar^2, ar^3, ar^4, \dots
a=p~and~r=1-p < 1
\dots \infty~times
Example: Donor List
A patient needs a certain blood group which only 9% of the population has?
P(success) = p
What is the probability that the 7th volunteer that the doctor contacts will be the first one to have a matching blood group?
What is the probability that at least one of the first 10 volunteers will have a matching blood type ?
\dots \infty~times
Example: Donor List
A patient needs a certain blood group which only 9% of the population has?
p = 0.09
P(X <=10)
p_X(7) = ?
= 1 - P(X > 10)
= 1 - (1-p)^{10}
\dots \infty~times
Uniform Distribution
Experiments with equally likely outcomes
X:
p_X(x) = \frac{1}{6}~~~\forall x \in \{1,2,3,4,5,6\}
outcome of a die
Uniform Distribution
Experiments with equally likely outcomes
X:
p_X(x) = \begin{cases} \frac{1}{b - a + 1}~~~a \leq x \leq b \\~\\ 0~~~~~~~~~otherwise \end{cases}
outcome of a bingo/housie draw
p_X(x) = \frac{1}{100}~~~1 \leq x \leq 100
\mathbb{R}_X = \{x: a \leq x \leq b\}
Uniform Distribution
Special cases
p_X(x) = \begin{cases} \frac{1}{b - a + 1} = \frac{1}{n}~~~1 \leq x \leq n \\~\\ 0~~~~~~~~~otherwise \end{cases}
a = 1 ~~~~ b = n
p_X(x) = \begin{cases} \frac{1}{b - a + 1} = 1~~~x = c \\~\\ 0~~~~~~~~~otherwise \end{cases}
a = 1 ~~~~ b = c
Uniform Distribution
p_X(x) \geq 0
\sum_{k=1}^\infty p_X(i) = 1 ?
Is Uniform distribution a valid distribution?
p_X(x) = \frac{1}{b - a + 1}
=\sum_{i=a}^b \frac{1}{b-a+1}
=(b-a+1) * \frac{1}{b-a+1} = 1
Expectation
00, 0, 1, 2,3, ..., 36
Does gambling pay off?
Standard pay off - 35:1 if the ball lands on your number
If you play this game a 1000 times how much do you expect to win on average ?
00
0
1
2
3
...
36
Does gambling pay off?
X: profit
\Omega
X
-1
34
p_X(34) =
\frac{1}{38} = 0.026
p_X(-1) = 1 - \frac{1}{38} = 0.974
Does gambling pay off?
P(win) = \frac{\#wins}{\#games}
\frac{1}{36} = 0.026
p_X(-1) = 1 - \frac{1}{38} = 0.974
If you play this game a 1000 times how much do you expect to win on average ?
p_X(34) =
0.026 = \frac{\#wins}{1000}
Avg. gain = ~~~~~~~(26*34 + 974*(-1))
\#wins = 26
\frac{1}{1000}
= -64
(Stop gambling!!)
Expectation: the formula
E[X]
E[X] = \frac{1}{1000}(26*34 + 974*(-1))
E[X] = \frac{26}{1000} * 34 + \frac{974}{1000} * (-1)
E[X] = \sum_{x\in\{-1, 34\}}x*p_X(x)
E[X] = p_X(34)*34 + p_X(-1)*(-1)
E[X] = 0.026*34 + 0.974*(-1)
Expectation: the formula
E[X]
E[X] = \sum_{x\in\mathbb{R}_X}x*p_X(x)
The expected value or expectation of a discrete random variable X whose possible values are
x_1, x_2, \dots, x_n
is denoted by       and computed as
E[X]
E[X] = \sum_{i=1}^n x_i P(X = x_i) = \sum_{i=1}^n x_i*p_X(x_i)
Expectation: Insurance
A person buys a car theft insurance policy of INR 200000 at an annual premium of INR 6000. There is a 2% chance that the car may get stolen.
What is the expected gain of the insurance company at the end of 1 year?
X: profit
X: \{6000, -194000\}
p_X(6000) = 0.98
p_X(-194000) = 0.02
E[X] = \sum_{x \in \mathbb{R}_x} x*p_X(x)
\therefore E[X] = 0.98*6000 + 0.02 * (-194000)
= 2000
Expectation: Insurance
A person buys a car theft insurance policy of INR 200000. Suppose there is a 10% chance that the car may get stolen
What should the premium be so that the expected gain in still INR 2000?
X: profit
X: \{x, -(200000 - x) \}
p_X(x) = 0.90
p_X(x - 200000) = 0.10
E[X] = \sum_{x \in \mathbb{R}_x} x*p_X(x)
\therefore E[X] = 0.9*x + 0.1 * (x - 200000)
\therefore 2000 = 0.9*x + 0.1 * (x - 200000)
\therefore x = 22000
X: \{x, x - 200000 \}
Function of a Random Variable
(1,1)
(1,2)~(2,1)
(1,3)~(2,2)~(3,1)
(1,4)~(2,3)~(3,2)~(4,1)
(1,5)~(2,4)~(3,3)~(4,2)~(5,1)
(1,6)~(2,5)~(3,4)~(4,3)~(5,2)~(6,1)
(2,6)~(3,5)~(4,4)~(5,3)~(6,2)
(3,6)~(4,5)~(5,4)~(6,3)
(4,6)~(5,5)~(6,4)
(5,6)~(6,5)
(6,6)
\Omega
2
3
4
5
6
7
8
9
10
11
12
Y = g(X)
X
Y
1
2
3
E[Y] = ?
Y = \begin{cases}
1~~if~~x < 5 \\
2~~if~~5 \leq x \leq 8 \\
3~~if~~x > 8 \\
\end{cases}
Y = g(X)
Y = \begin{cases}
1~~if~~x < 5 \\
2~~if~~5 \leq x \leq 8 \\
3~~if~~x > 8 \\
\end{cases}
E[Y] = 1*p_Y(1) + 2*p_Y(2) + 3*p_Y(3)
p_Y(1) = \frac{1}{36} + \frac{2}{36} + \frac{3}{36} = \frac{6}{36}
p_Y(2) = \frac{4}{36} + \frac{5}{36} + \frac{6}{36} + \frac{5}{36} = \frac{20}{36}
p_Y(3) = \frac{4}{36} + \frac{3}{36} + \frac{2}{36} + \frac{1}{36} = \frac{10}{36}
\therefore E[Y] = 1*(\frac{1}{36} + \frac{2}{36} + \frac{3}{36})
+ 2*(\frac{4}{36} + \frac{5}{36} + \frac{6}{36} + \frac{5}{36})
+3*(\frac{4}{36} + \frac{3}{36} + \frac{2}{36} + \frac{1}{36}) = \frac{76}{36}
Y = g(X)
\frac{6}{36}
\frac{20}{36}
\frac{30}{36}
p_X(x)
p_Y(y)
1
2
3
Y = g(X)
\therefore E[Y] = 1*(\frac{1}{36} + \frac{2}{36} + \frac{3}{36})
+3*(\frac{4}{36} + \frac{3}{36} + \frac{2}{36} + \frac{1}{36}) = \frac{76}{36}
+ 2*(\frac{4}{36} + \frac{5}{36} + \frac{6}{36} + \frac{5}{36})
\therefore E[Y] = 1*p_X(2) + 1 * p_X(3) + 1 * p_X(4)
+ 2*p_X(5) + 2 * p_X(6) + 2 * p_X(7) + 2 * p_X(8)
+ 3*p_X(9) + 3 * p_X(10) + 3 * p_X(11) + 3 * p_X(12)
Y = g(X)
\therefore E[Y] = 1*(\frac{1}{36} + \frac{2}{36} + \frac{3}{36})
+3*(\frac{4}{36} + \frac{3}{36} + \frac{2}{36} + \frac{1}{36}) = \frac{76}{36}
+ 2*(\frac{4}{36} + \frac{5}{36} + \frac{6}{36} + \frac{5}{36})
\therefore E[Y] = g(2)*p_X(2) + g(3) * p_X(3) + g(4) * p_X(4)
+ g(5)*p_X(5) + g(6) * p_X(6) + g(7) * p_X(7) + g(8) * p_X(8)
+ g(9)*p_X(9) + g(10) * p_X(10) + g(11) * p_X(11) + g(12) * p_X(12)
Y = \begin{cases}
1~~if~~x < 5 \\
2~~if~~5 \leq x \leq 8 \\
3~~if~~x > 8 \\
\end{cases}
\therefore E[Y] = \sum_x g(x)*p_X(x)
E[Y] = \sum_x g(x)*p_X(x)
E[Y] = \sum_y y*p_Y(y)
\equiv
Some properties of expectation
Linearity of expectation
Y = aX + b
E[Y] = \sum_{x \in \mathbb{R}_X} g(x)p_X(x)
= \sum_{x \in \mathbb{R}_X} a*x*p_X(x) +\sum_{x \in \mathbb{R}_X} b*p_X(x)
= a*E[X] + b*1~~(\because \sum_{x \in \mathbb{R}_X} p_X(x) = 1)
= a*\sum_{x \in \mathbb{R}_X} x*p_X(x) +b*\sum_{x \in \mathbb{R}_X} p_X(x)
= \sum_{x \in \mathbb{R}_X} (ax + b)p_X(x)
= aE[X] + b
Expectation of sum of RVs
Given a set of random variables
X_1, X_2, \dots, X_n
E[\sum_{i=1}^{n} X_i] = \sum_{i=1}^n E[X_i]
Expectation as mean of population
n students
\Omega
W: weights
30
40
50
60
E[W] = \sum_{i=1}^np_W(w_i)*w_i
p_W(w_i) = \frac{1}{n}
= \frac{1}{n}\sum_{i=1}^nw_i
(centre of gravity)
Expectation as centre of gravity
A patient needs a certain blood group which only 9% of the population has?
p = 0.09
E[X] = ?
= 1*0.09 + 2 * 0.91*0.09
+ 3 * 0.91 ^2 *.09 + 4 * 0.91^3*0.09 + \dots
= 0.09(\frac{a}{1-r} + \frac{dr}{(1-r)^2})
= \frac{1}{0.09} = \frac{1}{p}
= 0.09(1 + 2 * 0.91 + 3*0.91^2 + 4*0.91^3 + \dots
(a = 1, d = 1, r = 0.91)
= 11.11
Variance of a Random Variable
Variance of a RV
Expectation summarises a random variable Â
(but does not capture the spread of the RV)
E[X] = 0
E[Y] = 0
E[Z] = 0
X
0
1
-1
+1
\frac{1}{2}
\frac{1}{2}
Y
-100
-50
+100
+50
\frac{1}{4}
\frac{1}{4}
\frac{1}{4}
\frac{1}{4}
Z
Recap
Variance
\sigma^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \mu) ^ 2
0
-1
+1
0
-100
-50
+100
+50
0
Variance
Var(X) = E[(X - E(X))^2]
0
-1
+1
-100
-50
+100
+50
0
0
\sigma^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \mu) ^ 2
Variance
Var(X) = E[(X - E(X))^2]
\sigma^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \mu) ^ 2
\therefore Var(X) = E[X^2] -2\mu\cdot\mu + \mu^2
\therefore Var(X) = E[X^2] -2\mu E[ X] + \mu^2
\therefore Var(X) = E[X^2] - \mu^2 = E[X^2] - (E[X])^2
\therefore Var(X) = E[X^2] -E[2\mu X] + E[\mu^2]
\therefore Var(X) = E[(X - \mu)^2] = E[X^2 -2\mu X + \mu^2]
E[X] = \mu
Variance
Var(X) = E[X^2] - (E[X])^2
g(X) = X^2
\therefore E[X^2] = E[g(X)] = \sum_{x} p_X(x)g(x)
= 2^2 * \frac{1}{36} + 3^2 * \frac{2}{36} + 4^2 * \frac{3}{36} + 5^2 * \frac{4}{36} + 6^2 * \frac{5}{36}
+ 11^2 * \frac{2}{36}+ 12^2 * \frac{1}{36}
+ 7^2 * \frac{6}{36}+ 8^2 * \frac{5}{36}+ 9^2 * \frac{4}{36}+ 10^2 * \frac{3}{36}
= 54.83
Variance
Var(X) = E[X^2] - (E[X])^2
= 54.83 - 7^2
= 5.83
Variance: Mutual Funds
MF1 expected (average) returns: 8%
(12%, 2%, 25%, -9%, 10%)
MF2 expected (average) returns: 8%
(7%, 6%, 9%, 12%, 6%)
E[X] = \frac{1}{5}*12+\frac{1}{5}*2+\frac{1}{5}*25+\frac{1}{5}*(-9)+\frac{1}{5}*10 = 8
E[Y] = \frac{1}{5}*7+\frac{1}{5}*6+\frac{1}{5}*9+\frac{1}{5}*12+\frac{1}{5}*6 = 8
E[X^2] = \frac{1}{5}*12^2+\frac{1}{5}*2^2+\frac{1}{5}*25^2+\frac{1}{5}*(-9)^2+\frac{1}{5}*10^2
= 190.8
Variance: Mutual Funds
MF1 expected (average) returns: 8%
(12%, 2%, 25%, -9%, 10%)
MF2 expected (average) returns: 8%
(7%, 6%, 9%, 12%, 6%)
Var(X) = E[X^2] - (E[X])^2
Var(X) = 190.8 - 8^2 = 126.8
Var(Y) = 69.2 - 8^2 = 5.2
Properties of Variance
Var(aX + b) = E[(aX + b - E[aX + b])^2]
= E[(aX + b - aE[X] - b)^2]
= E[(a(X - E[X]))^2]
= E[a^2(X - E[X])^2]
= a^2E[(X - E[X])^2]
=a^2Var(X)
Properties of Variance
Var(X + X) = Var (2X) = 2^2Var(X)
Variance of sum of random variables
\neq Var(X) + Var(X)
In general, variance of the sum of random variables is not equal to the sum of the variances of the random variables
except......
Properties of Variance
P(X = x | Y = y) = P(X = x)~~\forall x \in \mathbb{R}_X, y \in \mathbb{R}_Y
Independent random variables
(X and Y are independent)
X: number~on~first~die
Y: sum~of~two~dice
P(Y = 8 ) = \frac{5}{36}
P(Y = 8 | X = 1) = 0 \neq P(Y = 8)
(X and Y are not independent)
Properties of Variance
We say that n random variables
X_1, X_2, \dots, X_n
are independent if
P(X_1 = x_1, X_2 = x_2, \dots, X_n = x_n)
= P(X_1=x_1)P (X_2=x_2) \dots P(X_n = x_n)
\forall x_1\in\mathbb{R}_{X_1}, x_2\in\mathbb{R}_{X_2}, \dots, x_n\in\mathbb{R}_{X_n}
Given such n random variables
Var(\sum_{i=1}^{n} X_i) = \sum_{i=1}^n Var(X_i)
Summary
Probability Mass Function
\dots n~times
\dots \infty~times
p_X(x) = p^x(1 - p)^{(1- x)}
n \choose k
p^k
(1-p)^{(n-k)}
p_X(k) =
p_X(k) = (1 - p)^{(k-1)}p
p_X(x) = \begin{cases} \frac{1}{b - a + 1}~~~a \leq x \leq b \\~\\ 0~~~~~~~~~otherwise \end{cases}
E[X] = \sum_{x\in\mathbb{R}_X}x*p_X(x)
E[Y] = \sum_x g(x)*p_X(x)
E[Y] = \sum_y y*p_Y(y)
\equiv
Var(X) = E[X^2] - (E[X])^2
Y = aX + b
E[Y]= aE[X] + b
Given n RVs
Var(\sum_{i=1}^{n} X_i) \\= \sum_{i=1}^n Var(X_i)
Var(Y) =a^2Var(X)
Continuous Random Variables
Recap
Probability Mass Function
p_X(x)
1~~~2~~~3~~~4~~~5~~~6
0.2
0.4
0.6
0.8
1.0
x
Cumulative Distribution Function
p_X(x)
1~~~2~~~3~~~4~~~5~~~6
0.2
0.4
0.6
0.8
1.0
x
F_X(x)
1~~~2~~~3~~~4~~~5~~~6
0.2
0.4
0.6
0.8
1.0
x
P(X \leq x)
Cumulative Distribution Function
F_X(x)
p_X(x)
2
3
4
5
6
7
8
9
10
11
12
x
P(X \leq x)
\frac{1}{36}
\frac{2}{36}
\frac{3}{36}
\frac{4}{36}
\frac{5}{36}
\frac{6}{36}
\frac{5}{36}
\frac{4}{36}
\frac{3}{36}
\frac{2}{36}
\frac{1}{36}
\frac{1}{6}
\frac{2}{6}
\frac{3}{6}
\frac{4}{6}
\frac{5}{6}
1 = \frac{6}{6}
2
3
4
5
6
7
8
9
10
11
12
x
\frac{1}{36}
\frac{3}{36}
\frac{6}{36}
\frac{10}{36}
\frac{15}{36}
\frac{21}{36}
\frac{1}{6}
\frac{2}{6}
\frac{3}{6}
\frac{4}{6}
\frac{5}{6}
1 = \frac{6}{6}
\frac{26}{36}
\frac{30}{36}
\frac{33}{36}
\frac{35}{36}
\frac{36}{36}
Recap
p_X(x)
2
3
4
5
6
7
8
9
10
11
12
x
\frac{1}{36}
\frac{2}{36}
\frac{3}{36}
\frac{4}{36}
\frac{5}{36}
\frac{6}{36}
\frac{5}{36}
\frac{4}{36}
\frac{3}{36}
\frac{1}{36}
\frac{1}{6}
\frac{2}{6}
\frac{3}{6}
\frac{4}{6}
\frac{5}{6}
1 = \frac{6}{6}
Probability Mass Function
Total Probability = 1 (unit mass)
PMF: What share of this unit mass does each value take?
What if the random variable can take infinite values?
(continuous random variables)
Continuous Random Variables
Total Probability = 1 (unit mass)
What share of the unit probability mass does each value take?
Rainfall in Chennai:
2 cm ?
2.01, 2.001, 1.99, 1.999
0
Infinite values possible
Continuous Random Variables
Total Probability = 1 (unit mass)
What share of the unit probability mass does each value take?
Rainfall in Chennai:
2 cm ?
2.01, 2.001, 1.99, 1.999
0
Infinite values possible
Continuous Random Variables
Your water intake (in litres )
Does not make sense to ask about the number of days on which you drank exactly 2 litres?
Instead it makes sense to ask about the number of days on which 1.9 < x < 2.1
1.0Â Â Â 1.5Â Â 2.5Â Â Â 2.5Â Â 3.0Â Â 3.5Â Â
\Omega
Continuous Random Variables
Your water intake (in litres )
Does not make sense to ask about the number of days on which you drank exactly 2 litres?
Instead it makes sense to ask about the number of days on which 1.9 < x < 2.1
1.0Â Â Â 1.5Â Â 2.0 Â Â 2.5Â Â 3.0Â Â 3.5Â Â
Continuous Random Variables
Your water intake (in litres )
1.0Â Â Â 1.5Â Â 2.0 Â Â 2.5Â Â 3.0Â Â 3.5Â Â
probability density function
FDS_Random_Variables
By One Fourth Labs
FDS_Random_Variables
PadhAI One: FDS Week 3 (MK)
