Emanuele Natale, Laurent Viennot and Arthur C. W. da Cunha
16 November 2022
Turing test (1950)
1st AI winter (1974–1980)
2nd AI winter (1974–1980)
"A hand lifts a cup"
Use of GPUs in AI (2011)
Today, most AI heavy lifting is done in the cloud due to the concentration of large data sets and dedicated compute, especially when it comes to the training of machine learning (ML) models. But when it comes to the application of those models in real-world inferencing near the point where a decision is needed, a cloud-centric AI model struggles. [...] When time is of the essence, it makes sense to distribute the intelligence from the cloud to the edge.
Chakraborty, I. et al. Resistive crossbars as approximate hardware building blocks for machine learning: Opportunities and challenges. Proc. IEEE 108, 2276–2310 (2020).
Analog MVM via crossbars of programmable resistances
Problem: Making precise programmable resistances is hard
Cfr. ~10k flops for digital 100x100 MVM
da Cunha, A., Natale, E. & Viennot, L. Proving the Strong Lottery Ticket Hypothesis for Convolutional Neural Networks. ICLR 2022
A network with random weights contains sub-networks that can approximate any given sufficiently-smaller neural network
INRIA Patent deposit FR2210217
Leverage noise itself
to increase precision
bits of precision for any target value are linear w.r.t. number of resistances
Worst case among