Loading
eiffl
This is a live streamed presentation. You will automatically follow the presenter and see the slide they're currently on.
Follow slides live at: slides.com/eiffl/nn-ts/live
Francois Lanusse @EiffL
Credit: PLAsTiCC team
What we will cover today:
Illustrations from this excellent blog: https://colah.github.io/posts/2015-08-Understanding-LSTMs/
The main idea: Preserve the information by default, update if necessary
1) Control of the state
2) Cell Output
Compared to the LSTM:
LSTM
RNN
RNN
RNN
RNN
Dense
The simplest RNN regression model
import tensorflow as tf
# Create model instance
model = tf.keras.Sequential()
# Add layers to your model
model.add(layers.LSTM(128, input_shape=(10,)))
model.add(layers.Dense(32))
# Compile the model with specific optimizer and loss function
model.compile(optimizer='rmsprop', loss='mse')
RNN
RNN
RNN
RNN
Dense
Let's go deeper! Stacked RNNs
RNN
RNN
RNN
RNN
RNN
RNN
RNN
RNN
RNN
RNN
RNN
RNN
Dense
Causality is overrated! Bi-directional RNNs
RNN
RNN
RNN
RNN
Pooling
Credit: https://arxiv.org/abs/1809.04356
Several problems of this approach:
For a temporal convolution W is a causal filter
Thank you!
Bonus: Check out a complete example of star/quasar classification by LSTM here