Sequence Modeling with RNNs
Suriyadeepan Ramamoorthy
Who Am I?
- AI Research Engineer, SAAMA Tech.
- I work in NLU
- I am a Free Software Evangelist
- I'm interested in
- AGI
- Community Networks
- Data Visualization
- Generative Art
- Creative Coding
- And I have a blog
At SAAMA
- Adverse Drug Event Extraction
- Pharmacovigilance
- Drug-Drug Interaction
- Protein-Protein Interaction
- Clinical Trial Optimization
- Inclusion-Exclusion Criteria
- OCR for Clinical Notes
- Machine Learning on EHR
- Genome Sequence Modeling
ADE Extraction
The Deep Learning Revolution
- Neural Networks
- Representation Power
- Specialized Networks
- Availability of labelled data
The Deep Learning Revolution
- Automatic Feature Learning
- Hierarchical Representation
The Deep Learning Revolution
The Deep Learning Revolution
- Automatic Feature Learning
- Hierarchical Representation
- Gradient Descent based Optimization
- Differentiable Networks
The Deep Learning Revolution
The Deep Learning Revolution
Sequence Modeling
\(y = softmax(o)\)
\(o = tanh(W_2h_1 + b_2)\)
\(h_1 = tanh(W_1x + b_1)\)
Recurrence
Recurrent Neural Network
Vector to Sequence
Sequence to Sequence
Bidirectional RNN
Stacked RNN
Flavors of RNN
Vanilla
LSTM
Long Short Term Memory
LSTM
LSTM
GRU
Gated Recurrent Unit
