Gregor Lenz
TU Delft, 10.3.2023
https://open-neuromorphic.org
https://github.com/open-neuromorphic
import tonic
transform = tonic.transforms.ToFrame(sensor_size=(128, 128, 2), time_window=3000)
dataset = tonic.datasets.DVSGesture(save_to="data", transform=transform)
from torch.utils.data import DataLoader
testloader = DataLoader(
dataset,
batch_size=8,
collate_fn=tonic.collation.PadTensors(batch_first=True),
)
frames, targets = next(iter(testloader))
Leaky integrate and fire neurons
animation from https://norse.github.io/norse/pages/working.html
Event-based vs clocked computation
EXODUS: Stable and Efficient Training of Spiking Neural Networks
Bauer, Lenz, Haghighatshoar, Sheik, 2022
https://lenzgregor.com/posts/train-snns-fast/
import torch.nn as nn
import sinabs.layers as sl
model = nn.Sequential(
nn.Conv2d(2, 8, 3),
sl.IAF(),
nn.AvgPool2d(2),
nn.Conv2d(8, 16, 3),
sl.IAF(),
nn.AvgPool2d(2),
nn.Flatten(),
nn.Linear(128, 10),
sl.IAF(),
)
# training...
from sinabs.backend.dynapcnn import DynapcnnNetwork
dynapcnn_net = DynapcnnNetwork(
snn=model,
input_shape=(2, 30, 30)
)
dynapcnn_net.to("speck2b")
https://lenzgregor.com
Gregor Lenz
Next talks