russtedrake PRO
Roboticist at MIT and TRI
System Identification: from linear models to neural networks
MIT 6.832: Underactuated Robotics
Spring 2021, Lecture 22
https://keypointnet.github.io/
https://nanonets.com/blog/human-pose-estimation-2d-guide/
Dense Object Nets
Core technology: dense correspondences
(built on Schmidt, Newcombe, Fox, RA-L 2017)
Peter R. Florence*, Lucas Manuelli*, and Russ Tedrake. Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation. CoRL, 2018.
Dense Object Nets
dense 3D reconstruction
+ pixelwise contrastive loss
Learn descriptor keypoint dynamics + trajectory MPC
Learn descriptor keypoint dynamics + trajectory MPC
System
..., u_{-1}, u_0, u_1, ...
..., y_{-1}, y_0, y_1, ...
Auto-regressive (ARX)
Lagrangian mechanics,
Recurrent neural networks (e.g. LSTM), ...
Feed-forward networks (e.g. \(y_n\)= image)
input
output
State-space
x_{n+1} = f(n, x_n, u_n, w_n, \theta) \\
\quad y_n = g(n, x_n, u_n, w_n, \theta)
y_{n+1} = f(n, u_n, u_{n-1}, ..., \\ \qquad \qquad y_n, y_{n-1}, ..., \\ \qquad \qquad w_n, w_{n-1}, ..., \theta)
Deep vs "physics-based" models
The failings of our physics-based models are mostly due to the unreasonable burden of estimating the "Lagrangian state" and parameters.
For e.g. onions, laundry, peanut butter, ...
The failings of our deep models are mostly due to our inability to due efficient/reliable planning, control design and analysis.
Lecture 22: System Identification (from linear to neural)
By russtedrake
Lecture 22: System Identification (from linear to neural)
MIT Underactuated Robotics Spring 2021 http://underactuated.csail.mit.edu
