Recap: Machine Learning

What we saw in the previous chapter?

(c) One Fourth Labs

Repeat the last slide of the previous chapter

Data data everywhere

What is the fuel of Machine Learning?

(c) One Fourth Labs

What is the mathematical formulation of a task?

(c) One Fourth Labs

$x$

$y$

bat

car

dog

cat

Models

$\left[\begin{array}{lcr} 2.1, 1.2, \dots, 5.6, 7.8 \end{array} \right]$

$\left[\begin{array}{lcr} 0, 0, 1,0, 0 \end{array} \right]$

$y = f(x)$ [true relation, unknown]

$\hat{y} = \hat{f}(x)$ [our approximation]

ship

$\left[\begin{array}{lcr} 0, 1, 0, 0, 0 \end{array} \right]$

$\left[\begin{array}{lcr} 0, 0, 0, 0, 1 \end{array} \right]$

$\left[\begin{array}{lcr} 1, 0, 0, 0, 0 \end{array} \right]$

$\left[\begin{array}{lcr} 0, 0, 1, 0, 0 \end{array} \right]$

$\left[\begin{array}{lcr} 0.1, 3.1, \dots, 1.7, 3.4\end{array} \right]$

$\left[\begin{array}{lcr} 0.5, 9.1, 3.7, ......0.8 \end{array} \right]$

$\left[\begin{array}{lcr} 0.2, 0.1, 0.7, ......0.8 \end{array} \right]$

Models

What are the choices for $\hat{f}$ ?

(c) One Fourth Labs

- Show some points sampled from this function

- Say that there is some complex relation between x

- Naively I assumed that its y=mx + c

- no matter how I adjust m and c I can't make f and $\hat{f}$ equal (apparently you can make a video of this in python itself)

- Let's try another function which is a polynomial

- another function...better...better better

$y = mx + c$

$y = ax^2 + bx + c$

$y = \sigma(wx + b)$

$y = Deep\_NN(x)$

$\hat{y} = \hat{f}(x)$ [our approximation]

$\left [\begin{array}{lcr} 0.2\\ 0.1\\ 0.7\\ ....\\0.8 \end{array} \right]$

$\left [\begin{array}{lcr} 2.2\\ 3.1\\ 0.7\\ ....\\4.8 \end{array} \right]$

$x$

$y$

$y = ax^3 + bx^2 + cx + d$

$y = ax^4 + bx^3 + cx + d$

I want to say that the true data was drawn from this function

These should be the points drawn from the function shown

Use Y_hat everywhere

In this course

$y = Deep\_CNN(x)$ ...

$y = RNN(x)$ ...

Models

Why not just use a complex model always ?

(c) One Fourth Labs

$\left [\begin{array}{lcr} 0.2\\ 0.1\\ 0.7\\ ....\\0.8 \end{array} \right]$

$\left [\begin{array}{lcr} 0.4\\ 0.2\\ 1.4\\ ....\\1.6 \end{array} \right]$

$x$

$y$

This will be replaced by a simple line

We will show animation how it will be easy to fit a line but difficult to fit 100 degree polynomial

$y = mx + c$ [true function, simple]

$y = ax^{100} + bx^{99} + ... + c$ [our approximation, very complex]

Later in this course

Bias-Variance Tradeoff

Overfitting

Regularization

Loss Function

How do we know which model is better ?

(c) One Fourth Labs

$\left [\begin{array}{lcr} 0.2\\ 0.1\\ 0.7\\ ....\\0.8 \end{array} \right]$

$\left [\begin{array}{lcr} 0.4\\ 0.2\\ 1.4\\ ....\\1.6 \end{array} \right]$

$x$

$y$

?

Show plots for true f and f_1 f_2 f_3... From the plots it will not be clear but from the columns it will be clear. You can take the points sampled from the sin_x function as in my bias variance lecture. f1, f2, f3 can be 3 different 25 degree polynomial as in that

The functions should be such that on visual inspection it should not be clear which is the closest to the true function

Now, on the next slide we will show the loss function

Then it is clear! Why is it clear? because you are computing some number and numbers are easy to interpret

$\hat{f_1}(x)$

$\left [\begin{array}{lcr} 0.4\\ 0.2\\ 1.4\\ ....\\1.6 \end{array} \right]$

$\hat{f_1}(x) = a_1x^{25} + b_1x^{24} + ... + c_1x + d_1$

$\hat{f_2}(x) = a_1x^{25} + b_1x^{24} + ... + c_1x + d_1$

$\hat{f_3}(x) = a_1x^{25} + b_1x^{24} + ... + c_1x + d_1$

Substitute different values for a1 b1, c1, d1 in the 3 equations

$\begin{array}{lcr} 1\\ 2\\ 3\\ ....\\n \end{array}$

the values for y should come from the sine function

put values for y1, y2, y3 as computed by the 3 functions defined below

$\mathscr{L}_1 = \sum_{i=1}^{n} (y_i - \hat{f}_1(x_i))^2$

$\hat{f_2}(x)$

$\hat{f_3}(x)$

$\mathscr{L}_2 = \sum_{i=1}^{n} (y_i - \hat{f}_2(x_i))^2$

$\mathscr{L}_3 = \sum_{i=1}^{n} (y_i - \hat{f}_3(x_i))^2$

In this course

Square Error Loss

Cross Entropy Loss

KL divergence

Text

cartoon images of

3 friends

Loss Function

How do we know which model is better ?

(c) One Fourth Labs

$\left [\begin{array}{lcr} 0.2\\ 0.1\\ 0.7\\ ....\\0.8 \end{array} \right]$

$\left [\begin{array}{lcr} 0.4\\ 0.2\\ 1.4\\ ....\\1.6 \end{array} \right]$

$x$

$y$

$\hat{f_1}(x)$

$\left [\begin{array}{lcr} 0.4\\ 0.2\\ 1.4\\ ....\\1.6 \end{array} \right]$

$\begin{array}{lcr} 1\\ 2\\ 3\\ ....\\n \end{array}$

the values for y should come from the sine function

put values for y1, y2, y3 as computed by the 3 functions defined below

$\mathscr{L}_1 = \sum_{i=1}^{n} (y_i - \hat{f}_1(x_i))^2$

$\hat{f_2}(x)$

$\hat{f_3}(x)$

$\mathscr{L}_2 = \sum_{i=1}^{n} (y_i - \hat{f}_2(x_i))^2$

$\mathscr{L}_3 = \sum_{i=1}^{n} (y_i - \hat{f}_3(x_i))^2$

In this course

Square Error Loss

Cross Entropy Loss

KL divergence

cartoon images of

3 friends

Show loss function computation only for $\hat{f}_1$ here as an example... Its ok to just show 3-4 terms in the summation (same terms which are there in the array on the LHS)

= some value after the computation shown on RHS

Learning Algorithm

How do we identify parameters of the model?

(c) One Fourth Labs

Animation:

- first the data matrix appears

- then the model equation with a,b,c as parameters appears

- then the friend appears

- then the loss function appears

- now the red cross appears and then the friend disappears

- now the box for the learning algorithm appears

- now the logo for search appears

- now an animation where the values of a,b,c are adjusted till the loss reaches some low value

- data, model and loss function feed int othe pink box

Show a a matrix with 3 inputs: budget* (0 to 1), box office collection* (0 to 1), action scene time+ (0 to 1)

* the unit is 100 crores so 0.1 means 1 crore (mention this in the head row of the table

+ the unit here is 100 minutes

The output is the imdb rating

Show x and y above the header

$\hat{f_1}(x) = 3.5x_1^2 + 2.5x_2^{3} + 1.2x_3^{2}$

cartoon images of 3 friends

$\hat{f_1}(x) = ax_1^2 + bx_2^{3} + cx_3^{2}$

$\mathscr{L}_1 = \sum_{i=1}^{n} (y_i - \hat{f}_1(x_i))^2$

Show a gear box inside this to indicate learning algorithm

Show adjustable scales for a,b,c and create a python video here if you adjust the scale the loss function value changes and you hit some value for which the error is zero. You can actually cheat by creating the y value using some values of a,b,c so that you can then get 0 error for these values of error

logo for search

Learning Algorithm

How do you formulate this mathematically ?

(c) One Fourth Labs

Animation:

- data, model, pink box, loss function, logo for search and animation for a,b,c appears as it if from previous slide

- now the cross and the message appears

- now the text in red appears (but make it black)

Show a a matrix with 3 inputs: budget* (0 to 1), box office collection* (0 to 1), action scene time+ (0 to 1)

* the unit is 100 crores so 0.1 means 1 crore (mention this in the head row of the table

+ the unit here is 100 minutes

The output is the imdb rating

Show x and y above the header

$\hat{f_1}(x) = ax_1^2 + bx_2^{3} + cx_3^{2}$

$\mathscr{L}_1 = \sum_{i=1}^{n} (y_i - \hat{f}_1(x_i))^2$

Show a gear box inside this to indicate learning algorithm

Show adjustable scales for a,b,c and create a python video here if you adjust the scale the loss function value changes and you hit some value for which the error is zero. You can actually cheat by creating the y value using some values of a,b,c so that you can then get 0 error for these values of error

logo for search

In practice, brute force search is infeasible

Find $a, b, c$ such that

is minimized

Learning Algorithm

How do you formulate this mathematically ?

(c) One Fourth Labs

Animation:

Only the green part gets added on this slide (but show it in black except for the tick mark)

Show a a matrix with 3 inputs: budget* (0 to 1), box office collection* (0 to 1), action scene time+ (0 to 1)

* the unit is 100 crores so 0.1 means 1 crore (mention this in the head row of the table

+ the unit here is 100 minutes

The output is the imdb rating

Show x and y above the header

$\hat{f_1}(x) = ax_1^2 + bx_2^{3} + cx_3^{2}$

$\mathscr{L}_1 = \sum_{i=1}^{n} (y_i - \hat{f}_1(x_i))^2$

Show a gear box inside this to indicate learning algorithm

Show adjustable scales for a,b,c and create a python video here if you adjust the scale the loss function value changes and you hit some value for which the error is zero. You can actually cheat by creating the y value using some values of a,b,c so that you can then get 0 error for these values of error

logo for search

Many optimization solvers are available

$min_{a,b,c}$