Neural Natural Language Generation.

A short tutorial

by Angelo Basile

based on Yoav Goldberg's tutorial at INLG2018.

http://u.cs.biu.ac.il/~yogo/INLG2018.pdf

Goals

  1. share with you my work
  2. disseminate what I learned at INLG
  3. show you how to build seq2seq models quickly (without writing code!)
  4. give you a summary on NLG (if you don't have time to read this 118 page review article)
  5. ...
  6. not annoy you!

Natural Language Generation (NLG)

  • machine translation
  • paraphrasing
  • summarisation
  • spell correction
  • ...

text-to-text

Natural Language Generation (NLG)

  • image captioning
  • soccer reports
  • weather and financial reports
  • ???

data-to-text

Hot Stuff in Natural Language Generation (NLG)

NLG in Industry

Industrial NLG

Industrial NLG

Industrial NLG

Industrial NLG

Industrial NLG

Industrial NLG

Academic NLG

  • University of Aberdeen
  • University of Edinburgh
  • Harvard University
  • University of Malta
  • University of Tilburg

Neural NLG

...the basics of seq2seq models

Neural NLG

INPUT

OUTPUT

Neural NLG - Output

INPUT

An English sentence. Or a whole paragraph. Or a document.

Neural NLG - Input

An English sentence. Or a whole paragraph. Or a document.

Syntax

{key:value}

Neural NLG - Input

An English sentence. Or a whole paragraph. Or a document.

Syntax

{key:value}

STRINGS

Tools

  • python 3
  • windows
  • GPU
  • character-based models
  • actively maintained
  • bleeding-edge research
  • yaml configuration

TOOLS

Warning

Don't use this!

Ready!

Task: style-constrained NLG

Tool: xNMT (char+bilstm+attention)

Data: E2E extended

Evaluation: BLEU

SCREEN TIME

...AND BACK!

Deletion, repetition, allucination

FAILS

WHAT ABOUT NON-NEURAL STUFF?

THE END

Thanks for your attention and for letting me attend INLG!

NLG

By Angelo