So what is Deep Learning?

• It's Machine Learning with neural networks
• "Deep" reflects the anatomy of these nets
• Mostly supervised learning
• Ideal for fuzzy data (images, audio, text)
• Outperfoms other approaches
• New easy-to-use frameworks for DL

Yes, it's a hype

#googletrends

#GWF2017

#alphago

Images

#deeposm

#terrapattern

#orbitalinsight

Convolutional Nets

#neuralnetworkzoo

(c) Fjodor Van Veen, Asimov Institute

Input Cell

Kernel

Convolutional

Hidden Cell

Output Cell

Text Mining

Text

Text

#datexis

#tasty

Memory Networks

#colah_on_LSTMs

=

The entire net repeated and interconnected

Sequences?

Single values (many-to-one)

Sequences (many-to-many)

Location?

Test Data

• ~ 130 inductive loops (speed, occupancy etc.)
• 9 years of data (atm testing only 1 month of 2014)
• 80% training, 20% test

Time Series

Workflow

• Define a time window (e.g. 15 min of historical data)
• Take moving average from each sensors as input ...
• ... to predict a future value for one sensor (e.g. 15 min ahaed)
• Narrow down to neighbours
• Feed input vectors into network
• Use RMSE as loss function

Spatial Correlation?

#makeyourownneuralnetwork

pos. correlation

neg. correlation

Adjacency Matrix

Python Code (Tensor Flow)

@tf_attributeLock
def prediction(self):
    c.debugInfo(__name__,"Adding Prediction nodes to the graph")
    with tf.name_scope('layer1'):
        weights = tf.Variable(tf.truncated_normal(
                (self.n_input,self.n_hidden),stddev=0.1
            ), name="lay1_weights")
        bias = tf.Variable(tf.constant(
                0.1,shape=[self.n_hidden]
            ), name = "lay1_bias")
        out_layer1 = tf.nn.sigmoid(tf.matmul(
                self.data,weights
            )+bias, name = "lay1_output")
    with tf.name_scope('layer2'):
        weights = tf.Variable(tf.truncated_normal(
                (self.n_hidden,self.n_output),stddev=0.1
            ), name="lay2_weights")
        bias = tf.Variable(tf.constant(
                0.1,shape=[self.n_output]
            ), name="lay2_bias")
        out_layer2 = tf.nn.sigmoid(tf.matmul(
                out_layer1,weights
            )+bias, name = "lay2_output")
    return out_layer2

Visual Representation

#tensorboard

Caveats?

• It needs A LOT of training data
• Computationally intense = waste of energy?
• DKWGO, but it works!?
• So many hyperparameters to tune
• How often do I need to train the net?

Alternatives?

• Autoregressive methods (STARIMA etc.)
• Support-Vector Machines