Open source software at SynSense: motivation, challenges and examples

Gregor Lenz

13. 12. 2022

SynSense

  • Founded 2017 in Zurich
     
  • Spin off from UZH/ETH
     
  • Ning Qiao and Prof. Giacomo Indiveri
     
  • Offices in Zurich, Chengdu, Shanghai

Core tech teams at SynSense

  • Hardware team: Circuit design for our chips
     
  • System-integration team: interface between hardware and higher-level software. Allows to configure chips
     
  • Algorithms team: Train spiking neural networks for different tasks

Technical challenges

  • Dataloading / dataset curation
     
  • Training robust models
     
  • Deployment to custom hardware

Dataloading and datasets

SNN training and deployment for vision models

SNN training and deployment for audio models

Accelerated neuron models

rockpool.ai

sinabs.ai

tonic.readthedocs.io/

SynSense Github!

Why we write our own libraries

  • We started in 2017
     
  • The current list of features on top of PyTorch is not very long - manageable time effort
     
  • Need high degree of control to move fast
     
  • Features that we care about without the need for compromises
     
  • Tailored to our hardware

Dataloading and datasets

  • Dataloading is a serious bottleneck when training models

Dataloading and datasets

https://lenzgregor.com/posts/train-snns-fast/

Dataloading and datasets

  • Dataloading is a serious bottleneck when training models
     
  • We create our own datasets - how to manage them?
     
  • Event-based filtering / transformations

SNN training

  • Anti-features: intricate neuron models, biologically-plausible learning rules
     
  • Focus on training speed and robustness

SNN training

EXODUS: Stable and Efficient Training of Spiking Neural Networks

Bauer, Lenz, Haghighatshoar, Sheik, 2022

SNN training

Supervised training of spiking neural networks for robust deployment on mixed-signal neuromorphic processors

Büchel, Zendrikov, Solinas, Indiveri & Muir, 2021

Model definition and deployment to Speck

import torch.nn as nn
import sinabs.layers as sl

model = nn.Sequential(
    nn.Conv2d(2, 8, 3),
    sl.IAF(),
    nn.AvgPool2d(2),
    nn.Conv2d(8, 16, 3),
    sl.IAF(),
    nn.AvgPool2d(2),
    nn.Flatten(),
    nn.Linear(128, 10),
    sl.IAF(),
)

# training...

from sinabs.backend.dynapcnn import DynapcnnNetwork

dynapcnn_net = DynapcnnNetwork(
    snn=model,
    input_shape=(2, 30, 30)
)

dynapcnn_net.to("speck2b")

Bit-precise simulation of hardware

# model definition...

from rockpool.transform import quantize_methods as q
from rockpool.devices.xylo import config_from_specification, XyloSim
from rockpool.devices import xylo as x

# map the graph to Xylo HW architecture
spec = x.mapper(model.as_graph(), weight_dtype='float')
# quantize the parameters to Xylo HW constraints
spec.update(q.global_quantize(**spec))

xylo_conf, _, _ = config_from_specification(**spec)

XyloSim_model = XyloSim.from_confiq(xylo_conf)

Conclusions

  • Our research progress is publicly available
     
  • Neuron models are based on hardware capabilities, unfortunately no standardized neuron models
     
  • We offer mature frameworks that are battle-tested and are open to contributions

Algorithms team

lenzgregor.com

Gregor Lenz

Made with Slides.com