Suivi de 2ème année
Gregor Lenz

Titre de la thèse: Neuromorphic Systems for Mobile Computing
sous la direction de Dr Sio-Hoi Ieng et Pr Ryad Benosman

Pr Bruno Gas
Directeur du Département
"Master Sciences de l'Ingénieur"
Sorbonne Université

Dr Camille Simon Chane
École Nationale Supérieure de l’Électronique et de ses Applications
ETIS UMR 8051, CY University, ENSEA, CNRS

Mobile Computing

• limited power capacity
• growing demands of functionality
• need for efficient computing

How does it scale?

• limited power capacity:
  ~5% battery improvement / year
• need for efficient computing: more transistors / area
• growing demands of functionality: cloud computing

• computes extremely efficiently (20 W)
• completely different mechanisms of computation
• copy it by recreating the basic components

Neuromorphic Engineering

• Artificial neurons
• Computing with spikes
• Asynchronous communication

Neuromorphic System

1. Sensor

2. Algorithm

3. Hardware

Goal: low power system for mobile devices

1. Connecting a Neuromorphic Camera to a Mobile Device

Mobile Phone Android framework

• Motivation: assist elderly and visually impaired people with hand gestures and voice commands
• Prototype that connects a small
  event camera
• Paper in preparation

• integrates different modalities
  (gestures / speech)
• displays event camera output
  in real time
• detects 4 directional gestures

Mobile Phone Android framework

Maro, Lenz, Reeves and Benosman, Event-based Visual Gesture Recognition with Background Suppression running on a smart-phone, 14th ICAG 2019. Best demo award.

Mobile Phone Android framework

Maro, Lenz, Reeves and Benosman, Event-based Visual Gesture Recognition with Background Suppression running on a smart-phone, 14th ICAG 2019. Best demo award.

2. Efficient neuromorphic algorithm for camera data

Event-based Face Detection Using the Dynamics of Eye Blinks

• Lenz et al., Frontiers of Neuroscience 2020
  

• tracking with μs precision and in difficult light situations
   
• lower power than gold standard methods
   
• robust to multiple faces and partial occlusions

Event-based Face Detection Using the Dynamics of Eye Blinks

Lenz et al., Frontiers of Neuroscience 2020

Event-based Face Detection Using the Dynamics of Eye Blinks

3. Algorithm execution on neuromorphic hardware

Neuromorphic Hardware

• Loihi is a neuromorphic research test chip designed by Intel Labs that uses an asynchronous spiking neural network (SNN)
   
• 128k neurons

Neural Computation on Loihi

• Almost all spiking neural
  networks use rate coding
   
• Temporal encoding exists:
  Time To First Spike (TTFS)
  but very inaccurate
   
• We propose alternative
  encoding scheme based
  on inter spike intervals (ISI)
   
• paper under submission

Neural Computation on Loihi

• Composable networks for general purpose computing using artificial neurons

Neural Computation on Loihi

• Conversion of networks trained on GPUs for efficient inference on Loihi

Conclusions

• New computing principles for low-power devices
   
• Brain-like processing (spikes) eases human-machine interfaces
   
• Neuromorphic computing can potentially extend the capabilities of current devices

Summary of contributions

• 1 journal article published

• 1 paper under submission

• 1 paper in preparation

• 110 page thesis manuscript draft

Other contributions:

Maro, Lenz, Reeves and Benosman, Event-based Visual Gesture Recognition with Background Suppression running on a smart-phone, 14th ICAG 2019.

Haessig, Lesta, Lenz, Benosman and Dudek, A Mixed-Signal Spatio-Temporal Signal Classifier for On-Sensor Spike Sorting, ISCAS 2020.

Oubari, Exarchakis, Lenz, Benosman and Ieng, Computationally efficient learning on very large event-based datasets, to be submitted 2020

20 Watt for 6 years:

1 MWh

Can We Learn From the Brain?

1 GWh