Notes of

Machine Learning

Chris Hung, Jin Xun

Content

1. Classical ML
   1. An overview (Chris)
   2. Supervised Learning - Regression (LJX)
   3. Supervised Learning - Classification (LJX)
   4. Reinforcement Learning (RL) (Next)
2. Training
   1. Model Design
   2. Basis of model training
   3. Adjust learning rate (LJX)
   4. Batch normalization(LJX)
3. Quantum Machine Learning
4. Application Examples

An overview of Classical ML

Map of ML

Model Selection

Design Method

Give some features

Predict another feature

(Output one scalar)

Supervised Learning  Regression

Give some features

Classify it

(Output one vector)

Supervised Learning  Classification

把regression重複做

每次乘上不同的w和加上不同的bias

Soft-max

1. Normalization

2. 放大差距(指數)

Model Design

Preprocess data

Basis of Model Training

• loss
• Escape Saddle point and Local Minima
• Batch
• Momentum+Gradient Descent

Train Model

Escape Saddle Point

Batch Size And

Momentum

Help Escape from

Crtical Points

Batch

Batch Size

Momentum

-Gradient(n)+movement(n-1)

想像球從高處滾下來，帶著之前的動量跑

可能可以翻出local minimum

Adjust Learning Rate

Best Optimizer: Adam=RMSProp + Momentum

Adagrad

Adagrad consider all gradients before

Might not be dynamic

RMSProp

Adam

Original paper: https://arxiv.org/pdf/1412.6980.pdf

Learning Rate Scheduling

從頭到尾用一個learning rate, 可能前面累積的參數會影響前進

Batch normalization

Make it easier to train

Self-attention

Input is a set of vectors

may change length

為了考慮前後的資料

Text

No correlation

Better!!