pybrain

April 10, 2015

Lightning Talk

Predict Weather with

 @hobsonlane

PyCon 2015

Hobson Lane

Neural Nets

Hidden Layer Weights

ColorMaps(trainer)

Output

Input

Hidden

pug-ann

pybrain.Network pug.ann.util.build_neural_net():

def build_neural_net(N_inp=7, N_hid=2):
    nn = pb.structure.FeedForwardNetwork()

    # layers
    inlay = pb.structure.LinearLayer(N_inp, name='input')
    nn.addInputModule(inlay)
    outlay = pb.structure.LinearLayer(1, name='output')
    nn.addOutputModule(outlay)

    # connections
    if N_hid:
        hidlay = pb.structure.LinearLayer(N_hid, name='hidden')
        nn.addModule(hidlay)
        in_to_hid = pb.structure.FullConnection(inlay, hidlay)
        hid_to_out = pb.structure.FullConnection(hidlay, outlay)
        nn.addConnection(in_to_hid)
        nn.addConnection(hid_to_out)
    else:
        in_to_out = pb.structure.FullConnection(inlay, outlay)
        nn.addConnection(in_to_out)

    nn.sortModules()
    return nn

def pybrain_dataset_from_dataframe(df, 
        inputs=['Max Humidity', ' Mean Humidity', ' Min Humidity'],
        outputs=['Max TemperatureF']):
    N_inp = len(inputs)
    N_out = len(outputs)

    ds = pb.datasets.SupervisedDataSet(N_inp, N_out)
    for sample in df[inputs + outputs].values:
        ds.addSample(sample[:N_inp], sample[N_inp:])
    return ds

pybrain.Dataset pug.ann.util.pybrain_dataset_from_dataframe():

from pug.ann.example import train_weather_predictor
trainer, dataframe = train_weather_predictor(
        location='PDX',
        delays=(1, 2, 3),
        epochs=1000,
        use_cache=True,
        )

All together now:

pip install pug-ann

epoch    995  total error       5.2621   avg weight         1.008
epoch    996  total error       5.2555   avg weight         1.008
epoch    997  total error       5.2483   avg weight         1.008
epoch    998  total error         5.24   avg weight         1.008
epoch    999  total error        5.231   avg weight         1.008

5 minutes later...

RMSE = 5.2 deg C 

Looks OK!

Not So Fast

Momentum "Trading"?

Predictor just extrpolates trend

Extroplate in space as well as time...

Add "nearby" Locations: 

• Seattle?

• Vancouver?

• Hawaii?

• Boise?

• Sacramento?

Automatic Feature Selection

• Mutual information
• Entropy
• Autocorrelation
• Delayed cross-correlation

Auto-Restructuring

• Prune

  • Near-zero weights

  • Low entropy

• ​Meta learning

  • Reinforcement

  • Genetic

• Convolution Structure

  • By name

  • By units

  • By stats