Alvin Chan

Deep Learning Basics

A Crash Course

Personal Journey

Takeaways

  • Parts of a DL project
  • Basic DL concepts
  • Bonus: how the code looks like

Deep Learning

  • Branch of machine learning

Tensorflow

  • Software library to implement deep learning
  • Open-source Software Framework
  • Uses CPU or GPU (or TPU)
  • Build, Train & Predict with Deep Learning

Tensorflow

What's

CPU

GPU

ALU

CONTROL

CACHE

2000us/sample

216us/sample

10x Faster!

Colab

  • https://colab.research.google.com
  • Jupyter notebook with libraries
  • Free GPU!!

Training a Model

  • Dataset     ~ Knowledge Source
  • Model        ~ Learner
  • Optimizer  ~ Correction mechanism
  • Visualization 

Dataset

  • Image
  • Text
  • Graphs
  • etc

Model

  • With layers of 'neurons'
    • Dense layer
    • Convolutional layer
    • Recurrent layer
    • etc

Optimizer

  • Adjust learned 'knowledge'
  • Examples
    • SGD
    • ADAM

A neuron

input_1

input_2

input_3

output

param_1

param_2

param_3

+

+

=

output

input_1 * param_1    input_2 * param_2    input_3 * param_3

'Rectangle' neuron

length

breadth

brightness

perimeter

x0

x2

x2

length

breadth

Deep learning

input_1

input_2

input_3

output

  • Dense Layer
  • Convolutional Layer
  • Recurrent Layer
  • Attention Layer
  • Many more..

of neural layer

Types

Dense layers

Dense Layer 1

Dense Layer 2

Example

  • https://playground.tensorflow.org/

Dense layers

Dense Layer 1

Dense Layer 2

Convolutional layers

input_1

input_2

input_3

output_1

input_4

input_5

step 1

Convolutional layers

input_1

input_2

input_3

output_1

input_4

input_5

output_2

Step 2

Convolutional layers

input_1

input_2

input_3

input_4

input_5

output_3

step 3

output_1

output_2

Convolutional layers

  • Reuse knowledge
  • Save memory

Deep learning for images

32 px

32 px

Source: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture6.pdf

32 px

32 px

Convolutional layers

input_1

input_2

input_3

output_1

input_4

input_5

Deep layers

Source: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture6.pdf

Code

https://www.tensorflow.org/tutorials/quickstart/beginner

Resources

  • https://playground.tensorflow.org/
  • https://www.tensorflow.org/tutorials
  • https://www.coursera.org/specializations/deep-learning
  • https://developers.google.com/machine-learning/crash-course/

Cheers!

 

https://github.com/alvinchangw/

twitter: @a1vinchan

 

Slides @ https://slides.com/alvinchan/dl-basics

 

Alvin Chan

Deep Learning Basics

By Alvin Chan

Deep Learning Basics

Deep Learning Basics: A Crash Course

  • 406