Understanding CNNS in NLP

Peadar Coyle

Contents

  • Convolutional Neural Network
     
  • Has Convolutions (Photoshop idea)
     
  • CNNs are Convolutions in a Neural Network
     
  • Image classification is explained first
     
  • Applied to NLP 

What is a Convolution?

A sliding window function applied to a matrix.

Edge Detection

Taking the difference between pixels and its neighbours detects edges

Blurring

Averaging each pixel with it's neighbours gives blurring

ReLu and Tanh

CNN

  • Layers of convolution with non-linear activation functions
  • Two function examples are ReLU and Tanh
  • In a traditional feed-forward network - we connect each input neuron to each output neuron in the next layer

CNN (2)

  • In CNNs we use convolutions over the input layer to compute the output
  • Local connections - each region of input connected to a neuron in output
  • Each layer applies different filters (100s or 1000s) and combines them

Training

A CNN automatically learns the values of its filters based on the task you want it to perform.

Example

A CNN for image classification may learn to detect edges in the first layer -> shapes -> facial shapes. The final layer will be a classifier that uses these high level features. 

2 Aspects

Location invariance and compositionality

 

  • Don't care where an elephant is in an image

     
  • Each filter composes a local patch of lower level features into a higher level representation.

     
  • Pixels -> Edges -> Shapes -> Complex Shapes

CNNs in NLP

By springcoil

CNNs in NLP

  • 594
Loading comments...

More from springcoil