Neural Natural Language Generation.
A short tutorial
by Angelo Basile
based on Yoav Goldberg's tutorial at INLG2018.
http://u.cs.biu.ac.il/~yogo/INLG2018.pdf
Goals
- share with you my work
- disseminate what I learned at INLG
- show you how to build seq2seq models quickly (without writing code!)
- give you a summary on NLG (if you don't have time to read this 118 page review article)
- ...
- not annoy you!
Natural Language Generation (NLG)
- machine translation
- paraphrasing
- summarisation
- spell correction
- ...
text-to-text
Natural Language Generation (NLG)
- image captioning
- soccer reports
- weather and financial reports
- ???
data-to-text
Hot Stuff in Natural Language Generation (NLG)
- style/personality (neural, non-neural)
- affect
- politeness
- persona-based NLG (also this)
- persuasive text
- motivational NLG
NLG in Industry
Industrial NLG
Industrial NLG
Industrial NLG
Industrial NLG
Industrial NLG
Industrial NLG
Academic NLG
- University of Aberdeen
- University of Edinburgh
- Harvard University
- University of Malta
- University of Tilburg
Neural NLG
...the basics of seq2seq models
Neural NLG
INPUT
OUTPUT
Neural NLG - Output
INPUT
An English sentence. Or a whole paragraph. Or a document.
Neural NLG - Input
An English sentence. Or a whole paragraph. Or a document.
Syntax
{key:value}
Neural NLG - Input
An English sentence. Or a whole paragraph. Or a document.
Syntax
{key:value}
STRINGS
Tools
- python 3
- windows
- GPU
- character-based models
- actively maintained
- bleeding-edge research
- yaml configuration
TOOLS
Warning
Don't use this!
Ready!
Task: style-constrained NLG
Tool: xNMT (char+bilstm+attention)
Data: E2E extended
Evaluation: BLEU
SCREEN TIME
...AND BACK!
Deletion, repetition, allucination
FAILS
WHAT ABOUT NON-NEURAL STUFF?
THE END
Thanks for your attention and for letting me attend INLG!