Learning Data Science

Lecture 8
Machine Learning

SQL and SQL Databases

Relational Databases

grades

students

courses

APIs

Application Programming Interface

Think of APIs like a waiter at a restaurant:

• You get a list of things you can do
• You request them from an API
• Then the information you want gets delivered back

Monte Carlo (MC) Methods

Use random sampling to estimate a very complicated probability distribution

• scipy.constants

• scipy.stats

• scipy.integrate

• scipy.interpolate

• scipy.optimize

• scipy.fft

SciPy

Tons of tools and highly optimized algorithms for doing math and science

Lecture 8

1. Recap
2. What is Machine Learning?
3. ML Landscape
4. Hands on with scikit-learn

Goals for today

• Understand the basic principles of ML
• Intuitive understanding of major models
• Picking the right ML model for a task

What is ML?

Machine Learning:

• statistical algorithms 
• learn from data
• generalize to unseen data

What is ML?

What is ML?

What is ML?

What is ML?

What is ML?

What is ML?

In most cases, we will never be able to determine the exact function

There are hundreds of algorithms to choose from!

ML Algorithms

Supervised

Supervised learning in a nutshell

Known

Known

Use labeled data to slowly push the unknown function towards correctness

Supervised

Supervised learning in a nutshell

1. Gather some data
2. Feed your data to the algorithm 
3. Let the algorithm guess the output
4. Compare guess to correct answers
5. Adjust algorithm towards a better guess
6. Repeat

Classification

Supervised

Regression

Output is limited set of categories

Output is somewhere along a number line

Supervised

ML Algorithms

Unsupervised

Unsupervised

Unsupervised learning in a nutshell

Known

unknown

Use unlabeled data and allow the algorithm to find patters on its own

Unsupervised

Clustering

Dimensionality

Clustering

Feed it tons of galaxy images

Too many to label by hand!
Why not let an algorithm figure out how many classes there are

Unsupervised

Dimensionality

Reducing the dimensionality of your data

Unsupervised

How can you help a student pick which degree would be most interesting for them?

Dimensionality

Reducing the dimensionality of your data

Unsupervised

Let the algorithm find 'hidden axes' in your data

Supervised

ML Algorithms

Unsupervised

Classification

Regression

Clustering

Dimensionality

Lecture 8

1. Recap
2. What is Machine Learning?
3. ML Landscape
4. Hands on with scikit-learn

Just some of the algorithms that are out there:

Time for today:

1. Linear regression
2. Logistic regression
3. k-nearest neighbors
4. Decision trees
5. k-means clustering
6. Random forests

The standard pipline

Linear Regression

Usually has an analytical solution!

Supervised + Regression

Logistic Regression

Variant of linear regression for binary classification

Supervised + Classification

k-nearest neighbors

Supervised + Classification

k-nearest neighbors

Supervised + Regression

hyperparameters: choosing k

k is a so-called "hyperparameter"

A hyperparameter is a parameter you choose before training

⚠️ Overfitting and Underfitting

• Overfitting: learning hyper-specific patterns that won't recur in the future
• Underfitting: failing to capture important patterns

Decision Trees

Supervised + Regression/Classification

Decision Trees

Supervised + Regression/Classification

k-means clustering

Unsupervised + Classification

Random Forests

Supervised + Regression/Classification

• Among the most powerful and well-used algorithms
• Many variations that increase their power
• Generally a great choice for good performance on tabular data

Random Forests

Supervised + Regression/Classification

Lecture 8

1. Recap
2. What is Machine Learning?
3. ML Landscape
4. Hands on with scikit-learn

To the notebook!

Lecture 8

1. Recap
2. What is Machine Learning?
3. ML Landscape
4. Hands on with scikit-learn