Neural Natural Language Generation.
A short tutorial
by Angelo Basile
![](https://media2.giphy.com/media/PLsbyZkfc1z0VMqGaN/giphy.gif)
based on Yoav Goldberg's tutorial at INLG2018.
http://u.cs.biu.ac.il/~yogo/INLG2018.pdf
Goals
- share with you my work
- disseminate what I learned at INLG
- show you how to build seq2seq models quickly (without writing code!)
- give you a summary on NLG (if you don't have time to read this 118 page review article)
- ...
- not annoy you!
![](https://media1.giphy.com/media/Xg4BqZezlViaA/giphy.gif)
![](https://media1.giphy.com/media/EgkNhBeY289z2/giphy.gif)
Natural Language Generation (NLG)
- machine translation
- paraphrasing
- summarisation
- spell correction
- ...
text-to-text
Natural Language Generation (NLG)
- image captioning
- soccer reports
- weather and financial reports
- ???
data-to-text
Hot Stuff in Natural Language Generation (NLG)
- style/personality (neural, non-neural)
- affect
- politeness
- persona-based NLG (also this)
- persuasive text
- motivational NLG
NLG in Industry
Industrial NLG
Industrial NLG
Industrial NLG
Industrial NLG
Industrial NLG
Industrial NLG
Academic NLG
- University of Aberdeen
- University of Edinburgh
- Harvard University
- University of Malta
- University of Tilburg
Neural NLG
![](https://media0.giphy.com/media/3o7TKzNcmYMe0hVsUU/giphy.gif)
...the basics of seq2seq models
![](https://media0.giphy.com/media/l4CD0LCe5r82lqRyM/giphy.gif)
Neural NLG
INPUT
OUTPUT
Neural NLG - Output
INPUT
An English sentence. Or a whole paragraph. Or a document.
Neural NLG - Input
An English sentence. Or a whole paragraph. Or a document.
Syntax
{key:value}
Neural NLG - Input
An English sentence. Or a whole paragraph. Or a document.
Syntax
{key:value}
STRINGS
![](https://media2.giphy.com/media/3oFzmhSdxPiyRyR5vi/giphy.gif)
Tools
![](https://s3.amazonaws.com/media-p.slid.es/uploads/421921/images/5502867/marian.png)
![](https://media3.giphy.com/media/3o7abKhOpu0NwenH3O/giphy.gif)
- python 3
- windows
- GPU
- character-based models
- actively maintained
- bleeding-edge research
- yaml configuration
TOOLS
Warning
Don't use this!
![](https://media0.giphy.com/media/MeFiwDSGDApHy/giphy.gif)
Ready!
Task: style-constrained NLG
![](https://media3.giphy.com/media/LN6AfV6TxxJtu/giphy.gif)
Tool: xNMT (char+bilstm+attention)
Data: E2E extended
Evaluation: BLEU
SCREEN TIME
...AND BACK!
![](https://media1.giphy.com/media/uRcoPXfNZUMp2/giphy.gif)
Deletion, repetition, allucination
FAILS
![](https://s3.amazonaws.com/media-p.slid.es/uploads/421921/images/5612961/extra-1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/421921/images/5612960/extra-2.png)
WHAT ABOUT NON-NEURAL STUFF?
![](https://media3.giphy.com/media/Mp4hQy51LjY6A/giphy.gif)
THE END
Thanks for your attention and for letting me attend INLG!
NLG
By Angelo
NLG
- 801