Understanding CNNS in NLP
- Convolutional Neural Network
- Has Convolutions (Photoshop idea)
- CNNs are Convolutions in a Neural Network
- Image classification is explained first
- Applied to NLP
What is a Convolution?
A sliding window function applied to a matrix.
Taking the difference between pixels and its neighbours detects edges
Averaging each pixel with it's neighbours gives blurring
ReLu and Tanh
- Layers of convolution with non-linear activation functions
- Two function examples are ReLU and Tanh
- In a traditional feed-forward network - we connect each input neuron to each output neuron in the next layer
- In CNNs we use convolutions over the input layer to compute the output
- Local connections - each region of input connected to a neuron in output
- Each layer applies different filters (100s or 1000s) and combines them
A CNN automatically learns the values of its filters based on the task you want it to perform.
A CNN for image classification may learn to detect edges in the first layer -> shapes -> facial shapes. The final layer will be a classifier that uses these high level features.
Location invariance and compositionality
- Don't care where an elephant is in an image
- Each filter composes a local patch of lower level features into a higher level representation.
- Pixels -> Edges -> Shapes -> Complex Shapes
CNNs in NLP