Alvin Chan
Deep Learning Basics
A Crash Course
Personal Journey
Takeaways
- Parts of a DL project
- Basic DL concepts
- Bonus: how the code looks like
Deep Learning
- Branch of machine learning
Tensorflow
- Software library to implement deep learning
- Open-source Software Framework
- Uses CPU or GPU (or TPU)
- Build, Train & Predict with Deep Learning
Tensorflow
What's
CPU
GPU
ALU
CONTROL
CACHE
2000us/sample
216us/sample
10x Faster!
Colab
- https://colab.research.google.com
- Jupyter notebook with libraries
- Free GPU!!
Training a Model
- Dataset ~ Knowledge Source
- Model ~ Learner
- Optimizer ~ Correction mechanism
- Visualization
Dataset
- Image
- Text
- Graphs
- etc
Model
- With layers of 'neurons'
- Dense layer
- Convolutional layer
- Recurrent layer
- etc
Optimizer
- Adjust learned 'knowledge'
- Examples
- SGD
- ADAM
A neuron
input_1
input_2
input_3
output
param_1
param_2
param_3
+
+
=
output
input_1 * param_1 input_2 * param_2 input_3 * param_3
'Rectangle' neuron
length
breadth
brightness
perimeter
x0
x2
x2
length
breadth
Deep learning
input_1
input_2
input_3
output
- Dense Layer
- Convolutional Layer
- Recurrent Layer
- Attention Layer
- Many more..
of neural layer
Types
Dense layers
Dense Layer 1
Dense Layer 2
Example
- https://playground.tensorflow.org/
Dense layers
Dense Layer 1
Dense Layer 2
Convolutional layers
input_1
input_2
input_3
output_1
input_4
input_5
step 1
Convolutional layers
input_1
input_2
input_3
output_1
input_4
input_5
output_2
Step 2
Convolutional layers
input_1
input_2
input_3
input_4
input_5
output_3
step 3
output_1
output_2
Convolutional layers
- Reuse knowledge
- Save memory
Deep learning for images
32 px
32 px
Source: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture6.pdf
32 px
32 px
Convolutional layers
input_1
input_2
input_3
output_1
input_4
input_5
Deep layers
Source: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture6.pdf
Code
https://www.tensorflow.org/tutorials/quickstart/beginner
Resources
- https://playground.tensorflow.org/
- https://www.tensorflow.org/tutorials
- https://www.coursera.org/specializations/deep-learning
- https://developers.google.com/machine-learning/crash-course/
Cheers!
https://github.com/alvinchangw/
twitter: @a1vinchan
Slides @ https://slides.com/alvinchan/dl-basics
Alvin Chan