Sentence Ordering and Coherence Modeling Using Recurrent Neural Networks
Lajanugen Logeswaran, Honglak Lee, Dragomir Radev
presented by
Andrianna Polydouri
The Problem
Given a set of sentences, order them in a coherent text.
Relevant tasks
- Order discrimination task
- Order sentences from abstracts of scientific articles
Motivation
Model text coherence (!)
RNN Basics...
Recurrent Neural Networks are mainly used for sequence processing
Typical sequences: text, video
RNN basics...
One-to-many (e.g. image captioning):
RNN Basics...
Many-to-one (e.g. sentiment analysis):
RNN Basics...
Sequence-to-sequence =
Many-to-one(encoder) + One-to-many (decoder)
Applied on tasks like: machine translation, sentence ordering ...
Variable-size input and output.
RNN Basics...
The proposed model
- Sentence Encoder : produce word embeddings
- Set Encoder: identifies a soft input order
- Decoder: selects the sentences sequentially
Results(1)
Order discrimination task:
Results(2)
Order sentences from abstracts of scientific articles
"Kendall's tau metric": τ = 1 - 2*N/(n per 2)
where N = number of pairs in the predicted sequence,
n = sequence length
Results(3)
Thank you !
Any questions?
deck
By andrian
