Deep Learning
refer to "Deep Learning Tutorial" by Yann LeCun and others
Deep Learning
Wikipedia says:
“Deep learning is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using multiple processing layers, with complex structures or otherwise, composed of multiple non-linear transformations.”
Machine Learning
Text
SCALE UP
Totally NEW?
Neural Nets
Perception
RNN
CNN
RBM
DBN
D-AE
AlexNet
GoogLeNet
McCulloch&Pitt 1943
Rosenblatt 1958
Grossberg 1973
Fufushima 1979
Hinton 1999
Hinton 2006
Vincent 2008
Alex 2012
Szegedy 2015
Learning Representation
-
ML/AI: how do we learn features?
- What is the fundamental rule?
- What is the learning algorithms?
- CogSci: how does the mind learn abstract concepts on top of less abstract ones?
- Neuroscience: how does the cortex learn perception?
- Deep learning : addresses the problem of learning hierarchical representation with a single algorithm.
Text
Trainable Feature Transform
Inspired by Nature?
L'Avion III de Clement Ader, 1897
Do not go TOO FAR
4 Types of Deep Architecture
Feed-Forward : multilayer neural nets, convolutional nets
Feed-Back : Stacked Sparse Coding, Deconvolutional Nets
Bi-Directional : Deep Boltzman Machine, Stacked Auto-Encoders
Recurrent : Recurrent Nets, Long-Short Term Memory
Really need DEEP?
What is Approach?
multi layers...
but...each layer has
complex model
with small data
The researchers say even they weren’t sure this new approach (152 layers!) was going to be successful – until it was.
“We even didn’t believe this single idea could be so significant,”
said Jian Sun, a principal research manager at Microsoft Research
is DL Omnipotent?
Applications
Scene Recognition (CNN)
Visual Style Recognition (CNN)
Object Detection (R-CNN)
Image Captioning (CNN+LSTM)
Segmentation (DeconvNet)
Deep Visuomotor Control
(CNN)
Neural Style (CNN)
Text
Text
Deep Learning Trends
- Multi Rresource(Image+Voice) -> 1 Label
- DL Best Practice -> Theory
How to study deep learning?
관심있는 연구주제에 대한 Data를 얻을 수 있는지 먼저 볼것!!!
Deep Learning Tools
Refer to
TensorFlow
Refer to "https://www.udacity.com/course/deep-learning--ud730"
TensorFlow : N-Dimension Arrays
- Scala : 0-D Tensor
- Vector : 1-D Tensor
- Matrix : 2-D Tensor
- Tensor : over 3-D
How many Dimension need?
for Basic Image Processing
X = [B, W, H, C]
SGD
정해진 입력데이터 Random하게 무한히 반복하여 트래이닝해도 결과에 영향을 주지 않는다.
Basic Logistic Regression
a example of Machine Learning
Basic Logistic Regression
a example of Machine Learning
Basic Logistic Regression
a example of Machine Learning
AlexNet
a example of Deep Learning
GoogLeNet
a example of Deep Learning
A simple ReLU Network를 As Matrix Operation으로 처리 가능하다.