Programmable Resistances
from Inaccurate Resistors
via Random Subset-Sum Theory
Emanuele Natale, Laurent Viennot and Arthur C. W. da Cunha
16 November 2022
The Deep Learning Revolution
Turing test (1950)
Today
1st AI winter (1974–1980)
2nd AI winter (1974–1980)
"A hand lifts a cup"
Use of GPUs in AI (2011)
Today, most AI heavy lifting is done in the cloud due to the concentration of large data sets and dedicated compute, especially when it comes to the training of machine learning (ML) models. But when it comes to the application of those models in real-world inferencing near the point where a decision is needed, a cloud-centric AI model struggles. [...] When time is of the essence, it makes sense to distribute the intelligence from the cloud to the edge.
Inference Energy
Chakraborty, I. et al. Resistive crossbars as approximate hardware building blocks for machine learning: Opportunities and challenges. Proc. IEEE 108, 2276–2310 (2020).
AI Accelerators
Resistive Crossbar Device
Analog MVM via crossbars of programmable resistances
Problem: Making precise programmable resistances is hard
Cfr. ~10k flops for digital 100x100 MVM
The Strong Lottery Ticket Hypothesis
da Cunha, A., Natale, E. & Viennot, L. Proving the Strong Lottery Ticket Hypothesis for Convolutional Neural Networks. ICLR 2022
A network with random weights contains sub-networks that can approximate any given sufficiently-smaller neural network
"Résistance équivalente modulable
à partir de résistances imprécises"
INRIA Patent deposit FR2210217
Leverage noise itself
to increase precision
RSS
Theorem
Programmable
effective resistance
Random Subset Sum Theory
bits of precision for any target value are linear w.r.t. number of resistances
Worst case among
2.5k instances