CS6910: Fundamentals of Deep Learning

Module 4: From Cats to Convolutional Neural Networks

Hubel & Wiesel Experiment on Cat

Observed electrical activity from individual neurons in the brains of cats. They used a slide projector to show specific patterns to the cats and noted that specific patterns stimulated activity in specific parts of the brain.

H & W experiment

1959

http://braintour.harvard.edu/archives/portfolio-items/hubel-and-wiesel

Fukushima and Neocognitron

Dr. Kunihiko Fukushima was inspired by Hubel and Wiesel’s work on simple and complex cells, and proposed the “neocognitron” model (original paper: “Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position”).

https://wcci2020.org/speaker/kunihiko-fukushima/

H & W experiment

NeoCognitron

1959

1979

LeNet5

The first work on modern CNNs done by Yann LeCun et al., in their paper “Gradient-Based Learning Applied to Document Recognition” (now cited 17,588 times) demonstrated that a CNN model which aggregates simpler features into progressively more complicated features can be successfully used for handwritten character recognition.

H & W experiment

NeoCognitron

1959

1979

1998

Lenet5

http://yann.lecun.com/exdb/lenet/

LeNet5 contd.

 LeCun et al. trained a CNN using the MNIST database of handwritten digits (MNIST pronounced “EM-nisst”)

H & W experiment

NeoCognitron

1959

1979

1998

Lenet5

AlexNet

In 2012, Alex Krizhevsky released AlexNet which was a deeper and much wider version of the LeNet and won by a large margin the difficult ImageNet competition.

H & W experiment

NeoCognitron

1959

1979

1998

Lenet5

2012

AlexNet

https://neurohive.io/en/popular-networks/alexnet-imagenet-classification-with-deep-convolutional-neural-networks/

CNN since 2012

https://towardsdatascience.com/a-reading-guide-about-deep-learning-with-cnns-3a0e0fc99b78

An algorithm inspired by an experiment on cats is today used to detect cats in videos and Visualize them :-)