The combination of Hebbian and predictive plasticity learns invariant object representations in deep sensory networks
Research salon
https://www.biorxiv.org/content/10.1101/2022.03.17.484712v1
This paper is about
a self-supervised learning rule that
- has a predictive component for temporally contiguous features
- has a Hebbian component that prevents activity collapse
Ideas based on...
VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning
https://arxiv.org/abs/2105.04906
Adrien Bardes, Jean Ponce and Yann LeCun
'cake analogy' by LeCun, NIPS 2016 / ISSCC 2019
self-supervised learning by LeCun at ISSCC 2019
Bienenstock Cooper Munro (BCM) theory
Latent Predictive Learning (LPL)
learning rule
Self supervised learning in a DNN
Local layer learning vs. end-to-end
Summary
- + self-supervised learning using an apparently biologically plausible rule
- + does not need contrastive examples
- + possibility to link this work to spike timing
- - limited to toy problems
- - no spiking in this work
LPL research salon
By Gregor Lenz
