1.8 Representation Power of functions

Why do we need complex functions ?

Recap: Six jars

What we saw in the previous chapter?

(c) One Fourth Labs

Repeat the slide comparing MP neuron, Perceptron and Sigmoid. Highlight that none of them cannot deal with non-linearly separable data.

Why do we care about continuous functions ?

(c) One Fourth Labs

1. On LHS show the update equations for w and b and write down that \deltaw=partial derivative \deltab=partial derivative

2. On RHS show examples of smooth functions, sigmoid in 2d, asymmetric sigmoid in 3d (w1 !=w2!=0), some other complex function in 3d. For each of the 3 examples contrast it with a non-continuous function. Show the equations also below the plots

Recap: Continuous Functions

Modeling Complex Relations

Why do we need complex functions ?

(c) One Fourth Labs

1. Show x1 axis and x2-axis only first

2. Label x1 as screen size, x2 as cost

3. Now create points in two concentric circles such that phones in the range of screen size 4.5 to 5.5inch and cost 8K to 12k are positive and points outside this range are negative (all points will be black first and then the positive points will become green and negative points as blue

4. Overall the plot should look something like the cookie plot on this URL but with some gap and slightly better separation between + and - points

5. Now on the LHS write the function \hat_y = \hat_f (x_1, x_2)

6. Now on the plot show that the function needs to output 1 (show as green zone) for the +ve points and output 0 (show as red zone) for the -ve points

7. now complete the function by making it smooth and continuous by seamlessly connecting the green and the red zones (in the background I will be saying that what we need continuous functions)

8. the above function should be something that we can write as f(x1, x2; w1, w2, b) and now show that by varying w1, w2, you can get different variants(in bg I will say that and our job is to find one which has the least error)

Modeling Complex Relations

Why do we need complex functions ?

(c) One Fourth Labs

Continue with the above slide and again show that sigmoid cannot fit this data no matter how you adjust this

In the RHS bottom corner show the cartoon person with 2 functions:

1) Are such complex functions seen in most real world examples ?

2) If so, how do I even come up with such complex functions?

Modeling Complex Relations

Are such complex functions seen in most real world examples ?

(c) One Fourth Labs

1. Show a classification example from some Kaggle competition (say, maybe vodafone churn prediction where the task is to predict whether a customer would switch n.w or not)

2. List down some rows from the data with some features in a table and show "..." as there will be many more features

3. Now at the bottom wrote \hat_y = \hat_f(x1, x2, ...., x1000)

Repeat this slide with 2 more examples from Kaggle

Modeling Complex Relations

How do we even come up with such complex functions ?

(c) One Fourth Labs

1.Show 3 different non-linearly separable datasets here in 3d. Each having different complex function to separate the + from - points. One of the plots can be the same as that on slide 3.1

2. Below each plot write the function that corresponds to the plot (a different function in each case)

3. In pink box say "It's hard to come up with such functions. We need a simple approach!"

Modeling Complex Relations

The construction analogy

(c) One Fourth Labs

1. Show the same plots as in the previous slide

2. Now replace the plots by buildings, a small house, a bungalow, a mansion

3. now at the bottom show a brick, some concrete slabs and then show how you combine bricks and/or slabs to construct buildings

Next slide:

1. Now replace the buildings by the plots again and instead of brick show the function f(x1, x2, ...xn) = sigmoid and show the figure

2. Now show the compact form of the function f(\bold_x; \boldw) = sigmoid

3. Now show multiple copies of this function with different w's (w1, w2, wk) placed side by side (ground floor) (below each of these functions show a brick)

4. now show the output of each of these would be h1, h2, ... hk... replace these by \bold h

5. now again show multiple copies of the function f (h, w) and continue (first floor)