Medicine

CheXNet (Stanford, 2017)

• An ML system trained on 112K labeled chest X-rays
• Detects pneumonia at radiologist-level accuracy
• Heatmap shows where the model "looks"

Input: X-ray image & label (disease / no disease) →

Output: diagnosis + localization

Science

Tabula Sapiens (CZI, 2022)

• 500K+ human cells profiled by gene expression
• Clustering reveals cell types across 24 organs — no predefined labels
• Each dot = one cell; color = organ of origin

Input: gene expression vectors

→ Output: discovered groupings

Robotics

π*0.6 (Physical Intelligence, 2025)

• Vision-language-action model learns from demos + practice
• Makes espresso, folds laundry, assembles boxes
• No task-specific programming — learns by trial and reward

State: camera image → Action: motor commands → Reward: task completion

traditionally

nowadays

In 6.390:

Outline

• Intro to ML
• Regression and Regularization
• Gradient Descent
• Linear Classification
• Features, Neural Networks I
• Neural Networks II (Backprop)
• Convolutional Neural Networks
• Representation Learning
• Transformers
• Markov Decision Processes
• Reinforcement Learning
• Non-parametric Models

Topics in order:

Many other ways to dissect

[These lists are neither exhaustive nor exclusive.]

After 6.390, you can…

• Frame ML problems: problem class, assumptions, evaluation.
• Build baselines and measure generalization (train vs. test).
• Implement and reason about regression and classification.
• Optimize models with gradients (SGD) and regularization.
• Work with representations and neural networks.
• Understand modern LLM mechanisms (transformers) and MDP/RL basics.

Class meetings 

released

due

Hours:

Lec: 1.5 hr

Rec + Lab: 3 hr

Notes + exercise: 2 hr

Homework: 6-7 hr

A typical content week in 6.390:

Grading

• Our objective (and we hope yours) is for you to learn about machine learning — take responsibility for your understanding; we are here to help!
• Grades formula: exercises 5% + homework 20% + labs 15% + midterms 30% + final 30%
• Lateness: 20% penalty per day, applied linearly (so 1 hour late is -0.83%)
• 20 one-day extensions, applied automatically on May 13 to maximize your benefit

• Midterm 1: Thursday, March 12, 7:30–9pm
• Midterm 2: Wednesday, April 15, 7:30–9pm
• Final: scheduled by Registrar (posted in 3rd week). ⚠️ – might be as late as May 21!

Collaboration and How to Get Help

• Understand everything you turn in
• Coding and detailed derivations must be done by you
• See collaboration policy/examples on course web site

Outline

an instance supervised learning known as regression: predicting a continuous number

toy data, for illustration only

temperature \(x\)

\(\left\{\left(x^{(1)}, y^{(1)}\right), \dots, \left(x^{(4)}, y^{(4)}\right)\right\}\)

\(\mathcal{D}_\text{train}:=\)

Training data:

\(x^{(1)} =\begin{bmatrix} x_1^{(1)} \\[4pt] x_2^{(1)}  \end{bmatrix}  \in \mathbb{R}^2\)

label

feature vector

\(y^{(1)} \in \mathbb{R}\)

\(\mathcal{D}_\text{train}:=\)

\(\left\{\left(x^{(1)}, y^{(1)}\right), \dots, \left(x^{(4)}, y^{(4)}\right)\right\}\)

Training data:

\(x^{(1)} =\begin{bmatrix} x_1^{(1)} \\[4pt] x_2^{(1)} \\[4pt] \vdots \\[4pt] x_d^{(1)} \end{bmatrix}  \in \mathbb{R}^d\)

label

feature vector

\(y^{(1)} \in \mathbb{R}\)

\(\mathcal{D}_\text{train}:=\)

\(\left\{\left(x^{(1)}, y^{(1)}\right), \dots, \left(x^{(n)}, y^{(n)}\right)\right\}\)

\(n\) data points, each with \(d\)-dimensional features and scalar label

Training data:

Training data in matrix-vector form:

temperature \(x\)

\(\mathcal{D}_\text{train}\)

Learning algorithm spits out a hypothesis

• Loss

temperature \(x\)

\(h(x)=10\)

• Training error 

• Test error

set of \(h\) we ask the algorithm to search over

Hypothesis class \(\mathcal{H}:\)

Regression

Algorithm

💻

\(\in \mathbb{R}^d \)

\(\in \mathbb{R}\)

\(\mathcal{D}_\text{train}\)

Quick summary:

Classification

Algorithm

💻

🧠⚙️
hypothesis class
loss function
...

classifier

{"Fish", "Grizzly", "Chameleon", ...}

"Fish"

images adapted from Phillip Isola

Looking ahead:

Outline

Why least squares?

• Problem: infer orbit parameters from noisy, partial measurements.
• Idea: pick parameters that make predictions match observations as closely as possible, by minimize the sum of squared residuals.
• Today: still a fast, reliable, interpretable baseline.

Regression

Algorithm

💻

\(\in \mathbb{R}^d \)

\(\in \mathbb{R}\)

\(\mathcal{D}_\text{train}\)

🧠 ⚙️

• hypothesis class
• loss function

General regression

Closed-form formula

💻

\(\in \mathbb{R}^d \)

\(\in \mathbb{R}\)

\(\mathcal{D}_\text{train}\)

🧠 ⚙️

• linear hypothesis class
• squared loss function

Ordinary least squares regression

Linear hypothesis class:

\(h\left(x ; \theta\right)\)

\( = \left[\begin{array}{lllll} \theta_1 & \theta_2 & \cdots & \theta_d\end{array}\right]\) \(\left[\begin{array}{c} x_1 \\ x_2 \\ \vdots \\ x_d\end{array}\right]\) 

for now, ignoring the offset

Ordinary least squares regression

Deriving the OLS solution

1. Write the training error \(J(\theta)\) in scalar form
2. Rearrange into matrix-vector form
3. Set the gradient \(\nabla_\theta J\) to zero 
4. Solve for the optimal parameters 

\( J(\theta) = \frac{1}{4}\big[ ({\theta_1} \cdot {\color{#cc0000}{90}} + {\theta_2} \cdot {\color{#cc0000}{7.2}} - {\color{#cc0000}{45}})^2 + ({\theta_1} \cdot {\color{#0000cc}{20}} + {\theta_2} \cdot {\color{#0000cc}{9.5}} - {\color{#0000cc}{32}})^2 + ({\theta_1} \cdot {\color{#009900}{35}} + {\theta_2} \cdot {\color{#009900}{8.4}} - {\color{#009900}{99}})^2 + ({\theta_1} \cdot {\color{#cc9900}{18}} + {\theta_2} \cdot {\color{#cc9900}{4.3}} - {\color{#cc9900}{39}})^2 \big]\)

1. Write out training error:

• Q: What kind of function is \(J(\theta)\)? 
• A: Quadratic function
• Q: What does \(J(\theta)\) look like?
• A: Typically, looks like a "bowl"

e.g.:

2. Rearrange training error into matrix-vector form

The beauty of

Summary

• When is this \(\theta^*\) not well defined? 
• What can cause this "not well defined"?
• What happens if we are just "close to not well-defined", aka "ill-conditioned"?
• We'll discuss all these next week.

\(\theta^*=\left({X}^{\top} {X}\right)^{-1} {X}^{\top} {Y}\)

Thanks!

We'd love to hear your thoughts.

Reference: Gradient Vector Refresher

(5 important facts for 6.390)

1. The gradient generalizes the concept of a derivative to multiple dimensions.
2. By construction, the gradient's dimensionality always matches the function input.

4. The gradient points in the direction of the (steepest) increase in the function value.

5. The gradient at the function minimizer is necessarily zero.

For \(f: \mathbb{R}^m \rightarrow \mathbb{R}\), its gradient \(\nabla f: \mathbb{R}^m \rightarrow \mathbb{R}^m\) is defined at the point \(p=\left(x_1, \ldots, x_m\right)\) as:​

Sometimes the gradient is undefined or ill-behaved, but today it is well-behaved.

1. The gradient generalizes the concept of a derivative to multiple dimensions.
2. By construction, the gradient's dimensionality always matches the function input.
3. The gradient can be symbolic or numerical.
4. The gradient points in the direction of the (steepest) increase in the function value.
5. The gradient at the function minimizer is necessarily zero.

Extension: How to deal with offset \(\theta_0\)

1. center the data; or
2. append a fake feature of 1

1. "center" the data

when data is centered, the optimal offset is guaranteed to be 0

centering

all column-wise \(\Sigma =0\)

2. Append a "fake" feature of \(1\) 

trick our model: treat the bias as just another feature, always equal to 1.

See recitation 1 for details.

temperature \(x_1\)