Sequence Modeling with RNNs

Suriyadeepan Ramamoorthy

Who Am I?

  • AI Research Engineer, SAAMA Tech.
  • I work in NLU
  • I am a Free Software Evangelist
  • I'm interested in
    • AGI
    • Community Networks
    • Data Visualization
    • Generative Art
    • Creative Coding
  • And I have a blog

At SAAMA

 

  • Adverse Drug Event Extraction
  • Pharmacovigilance
  • Drug-Drug Interaction
  • Protein-Protein Interaction
  • Clinical Trial Optimization
  • Inclusion-Exclusion Criteria
  • OCR for Clinical Notes
  • Machine Learning on EHR
  • Genome Sequence Modeling

ADE Extraction

The Deep Learning Revolution

  • Neural Networks
    • Representation Power
    • Specialized Networks
  • Availability of labelled data

The Deep Learning Revolution

  • Automatic Feature Learning
  • Hierarchical Representation

The Deep Learning Revolution

The Deep Learning Revolution

  • Automatic Feature Learning
  • Hierarchical Representation
  • Gradient Descent based Optimization
    • Differentiable Networks

The Deep Learning Revolution

The Deep Learning Revolution

Sequence Modeling

 \(y = softmax(o)\)

\(o = tanh(W_2h_1 + b_2)\)

\(h_1 = tanh(W_1x + b_1)\)

Recurrence

Recurrent Neural Network

Vector to Sequence

Sequence to Sequence

Bidirectional RNN

Stacked RNN

Flavors of RNN

Vanilla

LSTM

Long Short Term Memory

LSTM

LSTM

GRU

Gated Recurrent Unit

Made with Slides.com