CS6910: Fundamentals of Deep Learning
Module 4: From Cats to Convolutional Neural Networks
Hubel & Wiesel Experiment on Cat
Observed electrical activity from individual neurons in the brains of cats. They used a slide projector to show specific patterns to the cats and noted that specific patterns stimulated activity in specific parts of the brain.
H & W experiment
1959
http://braintour.harvard.edu/archives/portfolio-items/hubel-and-wiesel
Fukushima and Neocognitron
Dr. Kunihiko Fukushima was inspired by Hubel and Wiesel’s work on simple and complex cells, and proposed the “neocognitron” model (original paper: “Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position”).
https://wcci2020.org/speaker/kunihiko-fukushima/
H & W experiment
NeoCognitron
1959
1979
LeNet5
The first work on modern CNNs done by Yann LeCun et al., in their paper “Gradient-Based Learning Applied to Document Recognition” (now cited 17,588 times) demonstrated that a CNN model which aggregates simpler features into progressively more complicated features can be successfully used for handwritten character recognition.
H & W experiment
NeoCognitron
1959
1979
1998
Lenet5
http://yann.lecun.com/exdb/lenet/
LeNet5 contd.
LeCun et al. trained a CNN using the MNIST database of handwritten digits (MNIST pronounced “EM-nisst”)
H & W experiment
NeoCognitron
1959
1979
1998
Lenet5
https://glassboxmedicine.com/2019/04/13/a-short-history-of-convolutional-neural-networks/
AlexNet
In 2012, Alex Krizhevsky released AlexNet which was a deeper and much wider version of the LeNet and won by a large margin the difficult ImageNet competition.
H & W experiment
NeoCognitron
1959
1979
1998
Lenet5
2012
AlexNet
https://neurohive.io/en/popular-networks/alexnet-imagenet-classification-with-deep-convolutional-neural-networks/
CNN since 2012
https://towardsdatascience.com/a-reading-guide-about-deep-learning-with-cnns-3a0e0fc99b78
