Mobile Computing

• limited power capacity
• growing demands of functionality
• need for efficient computing

How does it scale?

• limited power capacity:
  ~5% battery improvement / year
• need for efficient computing: more transistors / area
• growing demands of functionality: cloud computing

• computes extremely efficiently (20 W)
• completely different mechanisms of computation
• copy it by recreating the basic components

Neuromorphic Engineering

• Artificial neurons
• Computing with spikes
• Asynchronous communication

Neuromorphic System

1. Sensor

2. Algorithm

3. Hardware

Goal: low power system for mobile devices

1. Connecting a Neuromorphic Camera to a Mobile Device

Mobile Phone Android framework

• Motivation: assist elderly and visually impaired people with hand gestures and voice commands
• Prototype that connects a small
  event camera
• Paper in preparation

• integrates different modalities
  (gestures / speech)
• displays event camera output
  in real time
• detects 4 directional gestures

Mobile Phone Android framework

Maro, Lenz, Reeves and Benosman, Event-based Visual Gesture Recognition with Background Suppression running on a smart-phone, 14th ICAG 2019. Best demo award.

Mobile Phone Android framework

Maro, Lenz, Reeves and Benosman, Event-based Visual Gesture Recognition with Background Suppression running on a smart-phone, 14th ICAG 2019. Best demo award.

2. Efficient neuromorphic algorithm for camera data

Event-based Face Detection Using the Dynamics of Eye Blinks

• Lenz et al., Frontiers of Neuroscience 2020
  

• tracking with μs precision and in difficult light situations
   
• lower power than gold standard methods
   
• robust to multiple faces and partial occlusions

Event-based Face Detection Using the Dynamics of Eye Blinks

Lenz et al., Frontiers of Neuroscience 2020

Event-based Face Detection Using the Dynamics of Eye Blinks

3. Algorithm execution on neuromorphic hardware

Neuromorphic Hardware

• Loihi is a neuromorphic research test chip designed by Intel Labs that uses an asynchronous spiking neural network (SNN)
   
• 128k neurons

Neural Computation on Loihi

• Almost all spiking neural
  networks use rate coding
   
• Temporal encoding exists:
  Time To First Spike (TTFS)
  but very inaccurate
   
• We propose alternative
  encoding scheme based
  on inter spike intervals (ISI)
   
• paper under submission

Neural Computation on Loihi

• Composable networks for general purpose computing using artificial neurons

Neural Computation on Loihi

• Conversion of networks trained on GPUs for efficient inference on Loihi

Conclusions

• New computing principles for low-power devices
   
• Brain-like processing (spikes) eases human-machine interfaces
   
• Neuromorphic computing can potentially extend the capabilities of current devices

Summary of contributions

• 1 journal article published

• 1 paper under submission

• 1 paper in preparation

• 110 page thesis manuscript draft

Other contributions:

Maro, Lenz, Reeves and Benosman, Event-based Visual Gesture Recognition with Background Suppression running on a smart-phone, 14th ICAG 2019.

Haessig, Lesta, Lenz, Benosman and Dudek, A Mixed-Signal Spatio-Temporal Signal Classifier for On-Sensor Spike Sorting, ISCAS 2020.

Oubari, Exarchakis, Lenz, Benosman and Ieng, Computationally efficient learning on very large event-based datasets, to be submitted 2020