Dead Poets Soc ty
AI
RNN - LSTM/GRU Text Generation
source of everything: Martin Gorner, https://goo.gl/zTDm7D
??!!!?!!,,,
EDGAR ALLAN NO
- Forgot to log accuracy
- 300k characters
- 100 iterations
- 2 GPUs
- 2 hours?
WHITMAN
- 60k characters
- 100 iterations
- 2 GPUs
- 15 minutes ???
- 97% accuracy
- ???!?!?!
WALT
WILL. I. AM.
SHAKESPEARE
- Whole night + more
- 8 CPUs
- No GPU
- Martin Gorner's work
- Complete works
Around and around we go...
RNN? WTF?
#######################
# LE KERAS CODE - WOW #
#######################
# Around 60-ish lines of code
# Around 1000000 headaches
# 40 chars at a time, chars = total no of unique lowercase chars
model.add(LSTM(128, input_shape=(40, len(chars))))
# Softmax layer, just as many neurons as chars then get highest prob
model.add(Dense(len(chars)))
model.add(Activation('softmax'))
Meth
MATH!
TMMDR: Use gates to transform shiite.
Takeaways
- Applications: speech recog, lang translation, text prediction
- Improve: Time. RNNs take a lot of computing power and time to train.
- Wish we had PH data to work with
- ML is weird. We got 97% accuracy (accdg to Keras) on Walt Whitman.
- Clean your data too????!!!???,,,,.
- ML is hard but fun