# CS6015: Linear Algebra and Random Processes

## Lecture 33:  Expectation, Variance and their properties, Computing expectation and variance of some known distributions

\Omega
X

### 35

p_X(35) =
\frac{1}{38} = 0.026
p_X(-1) = 1 - \frac{1}{38} = 0.974

### Does gambling pay off?

P(win) = \frac{\#wins}{\#games}
\frac{1}{38} = 0.026
p_X(-1) = 1 - \frac{1}{38} = 0.974

### If you play this game a 1000 times how much do you expect to win on average ?

p_X(34) =
0.026 = \frac{\#wins}{1000}
Avg. gain = ~~~~~~~(26*35 + 974*(-1))
\#wins = 26
\frac{1}{1000}
= -0.064

### Expectation: the formula

E[X]
E[X] = \frac{1}{1000}(26*34 + 974*(-1))
E[X] = \frac{26}{1000} * 34 + \frac{974}{1000} * (-1)
E[X] = \sum_{x\in\{-1, 34\}}x*p_X(x)
E[X] = p_X(34)*34 + p_X(-1)*(-1)
E[X] = 0.026*34 + 0.974*(-1)

### Expectation: the formula

E[X]
E[X] = \sum_{x\in\mathbb{R}_X}x*p_X(x)

### The expected value or expectation of a discrete random variable X whose possible values are

x_1, x_2, \dots, x_n

### is denoted by              and computed as

E[X]
E[X] = \sum_{i=1}^n x_i P(X = x_i) = \sum_{i=1}^n x_i*p_X(x_i)

### X: profit

X: \{6000, -194000\}
p_X(6000) = 0.98
p_X(-194000) = 0.02
E[X] = \sum_{x \in \mathbb{R}_x} x*p_X(x)
\therefore E[X] = 0.98*6000 + 0.02 * (-194000)
= 2000

### X: profit

X: \{x, -(200000 - x) \}
p_X(x) = 0.90
p_X(x - 200000) = 0.10
E[X] = \sum_{x \in \mathbb{R}_x} x*p_X(x)
\therefore E[X] = 0.9*x + 0.1 * (x - 200000)
\therefore 2000 = 0.9*x + 0.1 * (x - 200000)
\therefore x = 22000
X: \{x, x - 200000 \}

### Function of a Random Variable

(1,1)
(1,2)~(2,1)
(1,3)~(2,2)~(3,1)
(1,4)~(2,3)~(3,2)~(4,1)
(1,5)~(2,4)~(3,3)~(4,2)~(5,1)
(1,6)~(2,5)~(3,4)~(4,3)~(5,2)~(6,1)
(2,6)~(3,5)~(4,4)~(5,3)~(6,2)
(3,6)~(4,5)~(5,4)~(6,3)
(4,6)~(5,5)~(6,4)
(5,6)~(6,5)
(6,6)
\Omega
2
3
4
5
6
7
8
9
10
11
12

### Y = g(X)

X
Y
1
2
3
E[Y] = ?
Y = \begin{cases} 1~~if~~x < 5 \\ 2~~if~~5 \leq x \leq 8 \\ 3~~if~~x > 8 \\ \end{cases}

### Y = g(X)

Y = \begin{cases} 1~~if~~x < 5 \\ 2~~if~~5 \leq x \leq 8 \\ 3~~if~~x > 8 \\ \end{cases}
E[Y] = 1*p_Y(1) + 2*p_Y(2) + 3*p_Y(3)
p_Y(1) = \frac{1}{36} + \frac{2}{36} + \frac{3}{36} = \frac{6}{36}
p_Y(2) = \frac{4}{36} + \frac{5}{36} + \frac{6}{36} + \frac{5}{36} = \frac{20}{36}
p_Y(3) = \frac{4}{36} + \frac{3}{36} + \frac{2}{36} + \frac{1}{36} = \frac{10}{36}
\therefore E[Y] = 1*(\frac{1}{36} + \frac{2}{36} + \frac{3}{36})
+ 2*(\frac{4}{36} + \frac{5}{36} + \frac{6}{36} + \frac{5}{36})
+3*(\frac{4}{36} + \frac{3}{36} + \frac{2}{36} + \frac{1}{36}) = \frac{76}{36}

\frac{6}{36}
\frac{20}{36}
\frac{30}{36}
p_X(x)
p_Y(y)
1
2
3

### Y = g(X)

\therefore E[Y] = 1*(\frac{1}{36} + \frac{2}{36} + \frac{3}{36})
+3*(\frac{4}{36} + \frac{3}{36} + \frac{2}{36} + \frac{1}{36}) = \frac{76}{36}
+ 2*(\frac{4}{36} + \frac{5}{36} + \frac{6}{36} + \frac{5}{36})
\therefore E[Y] = 1*p_X(2) + 1 * p_X(3) + 1 * p_X(4)
+ 2*p_X(5) + 2 * p_X(6) + 2 * p_X(7) + 2 * p_X(8)
+ 3*p_X(9) + 3 * p_X(10) + 3 * p_X(11) + 3 * p_X(12)

### Y = g(X)

\therefore E[Y] = 1*(\frac{1}{36} + \frac{2}{36} + \frac{3}{36})
+3*(\frac{4}{36} + \frac{3}{36} + \frac{2}{36} + \frac{1}{36}) = \frac{76}{36}
+ 2*(\frac{4}{36} + \frac{5}{36} + \frac{6}{36} + \frac{5}{36})
\therefore E[Y] = g(2)*p_X(2) + g(3) * p_X(3) + g(4) * p_X(4)
+ g(5)*p_X(5) + g(6) * p_X(6) + g(7) * p_X(7) + g(8) * p_X(8)
+ g(9)*p_X(9) + g(10) * p_X(10) + g(11) * p_X(11) + g(12) * p_X(12)
Y = \begin{cases} 1~~if~~x < 5 \\ 2~~if~~5 \leq x \leq 8 \\ 3~~if~~x > 8 \\ \end{cases}
\therefore E[Y] = \sum_x g(x)*p_X(x)
E[Y] = \sum_x g(x)*p_X(x)
E[Y] = \sum_y y*p_Y(y)
\equiv

### Linearity of expectation

Y = aX + b
E[Y] = \sum_{x \in \mathbb{R}_X} g(x)p_X(x)
= \sum_{x \in \mathbb{R}_X} a*x*p_X(x) +\sum_{x \in \mathbb{R}_X} b*p_X(x)
= a*E[X] + b*1~~(\because \sum_{x \in \mathbb{R}_X} p_X(x) = 1)
= a*\sum_{x \in \mathbb{R}_X} x*p_X(x) +b*\sum_{x \in \mathbb{R}_X} p_X(x)
= \sum_{x \in \mathbb{R}_X} (ax + b)p_X(x)
= aE[X] + b

### Given a set of random variables

X_1, X_2, \dots, X_n
E[\sum_{i=1}^{n} X_i] = \sum_{i=1}^n E[X_i]

\Omega

### 60

E[W] = \sum_{i=1}^np_W(w_i)*w_i
p_W(w_i) = \frac{1}{n}
= \frac{1}{n}\sum_{i=1}^nw_i

### A patient needs a certain blood group which only 9% of the population has?

p = 0.09
E[X] = ?
= 1*0.09 + 2 * 0.91*0.09
+ 3 * 0.91 ^2 *.09 + 4 * 0.91^3*0.09 + \dots
= 0.09(\frac{a}{1-r} + \frac{dr}{(1-r)^2})
= \frac{1}{0.09} = \frac{1}{p}
= 0.09(1 + 2 * 0.91 + 3*0.91^2 + 4*0.91^3 + \dots
(a = 1, d = 1, r = 0.91)
= 11.11

E[X] = 0
E[Y] = 0
E[Z] = 0
X
0
1
-1
+1
\frac{1}{2}
\frac{1}{2}
Y
-100
-50
+100
+50
\frac{1}{4}
\frac{1}{4}
\frac{1}{4}
\frac{1}{4}
Z

### Variance

\sigma^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \mu) ^ 2
0
-1
+1
0
-100
-50
+100
+50
0

### Variance

Var(X) = E[(X - E(X))^2]
0
-1
+1
-100
-50
+100
+50
0
0
\sigma^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \mu) ^ 2

### Variance

Var(X) = E[(X - E(X))^2]
\sigma^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \mu) ^ 2
\therefore Var(X) = E[X^2] -2\mu\cdot\mu + \mu^2
\therefore Var(X) = E[X^2] -2\mu E[ X] + \mu^2
\therefore Var(X) = E[X^2] - \mu^2 = E[X^2] - (E[X])^2
\therefore Var(X) = E[X^2] -E[2\mu X] + E[\mu^2]
\therefore Var(X) = E[(X - \mu)^2] = E[X^2 -2\mu X + \mu^2]
E[X] = \mu

### Variance

Var(X) = E[X^2] - (E[X])^2
g(X) = X^2
\therefore E[X^2] = E[g(X)] = \sum_{x} p_X(x)g(x)
= 2^2 * \frac{1}{36} + 3^2 * \frac{2}{36} + 4^2 * \frac{3}{36} + 5^2 * \frac{4}{36} + 6^2 * \frac{5}{36}
+ 11^2 * \frac{2}{36}+ 12^2 * \frac{1}{36}
+ 7^2 * \frac{6}{36}+ 8^2 * \frac{5}{36}+ 9^2 * \frac{4}{36}+ 10^2 * \frac{3}{36}
= 54.83

### Variance

Var(X) = E[X^2] - (E[X])^2
= 54.83 - 7^2
= 5.83

### (7%, 6%, 9%, 12%, 6%)

E[X] = \frac{1}{5}*12+\frac{1}{5}*2+\frac{1}{5}*25+\frac{1}{5}*(-9)+\frac{1}{5}*10 = 8
E[Y] = \frac{1}{5}*7+\frac{1}{5}*6+\frac{1}{5}*9+\frac{1}{5}*12+\frac{1}{5}*6 = 8
E[X^2] = \frac{1}{5}*12^2+\frac{1}{5}*2^2+\frac{1}{5}*25^2+\frac{1}{5}*(-9)^2+\frac{1}{5}*10^2
= 190.8

### (7%, 6%, 9%, 12%, 6%)

Var(X) = E[X^2] - (E[X])^2
Var(X) = 190.8 - 8^2 = 126.8
Var(Y) = 69.2 - 8^2 = 5.2

### Properties of Variance

Var(aX + b) = E[(aX + b - E[aX + b])^2]
= E[(aX + b - aE[X] - b)^2]
= E[(a(X - E[X]))^2]
= E[a^2(X - E[X])^2]
= a^2E[(X - E[X])^2]
=a^2Var(X)

### Properties of Variance

Var(X + X) = Var (2X) = 2^2Var(X)

### Variance of sum of random variables

\neq Var(X) + Var(X)

### Properties of Variance

P(X = x | Y = y) = P(X = x)~~\forall x \in \mathbb{R}_X, y \in \mathbb{R}_Y

### (X and Y are independent)

X: number~on~first~die
Y: sum~of~two~dice
P(Y = 8 ) = \frac{5}{36}
P(Y = 8 | X = 1) = 0 \neq P(Y = 8)

### We say that n random variables

X_1, X_2, \dots, X_n

### are independent if

P(X_1 = x_1, X_2 = x_2, \dots, X_n = x_n)
= P(X_1=x_1)P (X_2=x_2) \dots P(X_n = x_n)
\forall x_1\in\mathbb{R}_{X_1}, x_2\in\mathbb{R}_{X_2}, \dots, x_n\in\mathbb{R}_{X_n}

### Given such n random variables

Var(\sum_{i=1}^{n} X_i) = \sum_{i=1}^n Var(X_i)

### Bernoulli random variable

E[X] = \sum_{x=0}^1 x\cdot p_X(x)
p_X(x) = p^x(1-p)^{(1-x)}
= 0*(1-p) + 1 * p
=p
Var(X) = E[X^2] - (E[X])^2
= \sum_{x=0}^1 x^2 \cdot p_X(x)
- p^2
= p - p^2 = p(1-p)

### Geometric random variable: $$E[X]$$

E[X] = \sum_{x=1}^\infty x\cdot p_X(x)
p_X(x) = (1-p)^{(x-1)}p
E[X] = 1*p + 2(1-p)p + 3(1-p)^2p + \dots
(1-p)E[X] = ~~~~~~~~~ + 1(1-p)p + 2(1-p)^2p + \dots

### Subtraction eqn 2 from eqn 1

pE[X] = p + p(1-p) + p(1-p)^2 + p(1-p)^3 + \dots
E[X] = 1 + (1-p) + (1-p)^2 + (1-p)^3 + \dots
E[X] = \frac{1}{1- (1-p)} = \frac{1}{p}

### Geometric random variable: $$Var(X)$$

Var(X) = E[X^2] - E[X]^2
E[X^2] = \sum_{i=1}^{\infty} i^2 (1-p)^{(i-1)}p
= \sum_{i=1}^{\infty} i^2 q^{(i-1)}p
= \sum_{i=1}^{\infty} (i - 1 + 1)^2 q^{(i-1)}p
= \sum_{i=1}^{\infty} ((i - 1)^2 + 2(i-1) + 1) q^{(i-1)}p

### Substitute j = i-1

= \sum_{j=0}^{\infty} j^2q^{j}p
+ 2\sum_{j=0}^{\infty} jq^{j}p
+ \sum_{j=0}^{\infty} q^{j}p
q = 1-p
= q\sum_{j=0}^{\infty} j^2q^{j-1}p
+ 2q\sum_{j=0}^{\infty} jq^{j-1}p
+ \sum_{j=0}^{\infty} q^{j}p

### Geometric random variable: $$Var(X)$$

Var(X) = E[X^2] - E[X]^2
E[X^2] = \sum_{i=1}^{\infty} i^2 (1-p)^{(i-1)}p
= q\sum_{j=0}^{\infty} j^2q^{j-1}p
+ 2q\sum_{j=0}^{\infty} jq^{j-1}p
+ \sum_{j=0}^{\infty} q^{j}p
\dots
= qE[X^2] + 2qE[X]+p\frac{1}{1-q}
q = 1 - p,~~~p= 1-q
E[X^2]= qE[X^2] + 2qE[X]+1
(1-q)E[X^2]= \frac{2q}{p}+1

### (steps on previous slide)

E[X^2]= \frac{q+1}{p^2}
Var(X) = E[X^2] - E[X]^2
= \frac{2q+p}{p}
= \frac{q + q + p}{p} = \frac{q + 1}{p}
=\frac{q+1}{p^2} - \frac{1}{p^2}
=\frac{q}{p^2}
=\frac{1-p}{p^2}