Deep Learning

 refer to "Deep Learning Tutorial" by Yann LeCun and others

Deep Learning

Wikipedia says: 

“Deep learning is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using multiple processing layers, with complex structures or otherwise, composed of multiple non-linear transformations.”

Machine Learning

Text

SCALE UP

Totally NEW?

Neural Nets
Perception

RNN
CNN

RBM

DBN
D-AE

AlexNet
GoogLeNet

McCulloch&Pitt 1943
Rosenblatt 1958

Grossberg 1973
Fufushima 1979

Hinton 1999

Hinton 2006
Vincent 2008

Alex 2012
Szegedy 2015

Really need DEEP?

Theoretically...2 layers are enough...

...if there are infinite training data 

What is Approach?

multi layers...

but...each layer has 

complex model

with small data

The researchers say even they weren’t sure this new approach (152 layers!) was going to be successful – until it was. 
“We even didn’t believe this single idea could be so significant,” 
said Jian Sun, a principal research manager at Microsoft Research

is DL Omnipotent?

Applications

Scene Recognition (CNN)

Visual Style Recognition (CNN)

Object Detection (R-CNN)

Image Captioning (CNN+LSTM)

Segmentation (DeconvNet)

Deep Visuomotor Control

(CNN)

https://www.youtube.com/watch?v=Q4bMcUk6pcw

Neural Style (CNN)

Text

Text

Deep Learning Trends

• Multi-modal(Image+Voice) -> 1 Label
• DL Best Practice -> Theory

Deep Learning Tools

Refer to

TensorFlow

Refer to "https://www.udacity.com/course/deep-learning--ud730"

How many Dimension need?

for Basic Image Processing

X = [B, W, H, C]

Basic Logistic Regression

a example of Machine Learning

AlexNet

a example of Deep Learning

GoogLeNet

a example of Deep Learning

Image Recognition Demo

Prerequisite

• Python
• Development Environment
• Numpy
• Image Handling
• TensorFlow
• Training & Testing Data Prepartion
• General Machine Learning Methodology
• Deep Learning Practice

Python Scientific Ecosystem

Python Ecosystem

Development Environment

Development Environment

NumPy = N-Dim Array

• Python List
  • 여러가지 타입의 원소
  • 메모리 용량이 크고 속도가 느림
  • nesting 가능
  • 전체 연산 불가
• NumPy Array
  • 동일 타입의 원소
  • 메모리 최적화, 계산 속도 향상
  • 크기(dimension)이 명확하게 정의
  • 전체 연산 가능

Live Coding Practice

• Image Handling 
• Training & Testing Data Prepartion
• General Machine Learning Practice
• Deep Learning Practice