Russ Meeting
2022/08/30
Agendas
-
NeRF Real-To-Sim Project
-
JMLR Submission
-
Contact-Aware Control
NERF Project
Assumptions:
- Known state (pose of the object is known)
- Rigid Object Assumption
- Unknown Shape, implicitly given some occupancy measure representation
Approach 1. Occupancy-Based Scene Graph
Finding Witness Points for the Algorithm
1. Given to measures
2. Find the first moment of the product measure
This point denotes the "point of action" for the two measures.
\rho^A_t, \rho^B_t
\bar{x} = \int x \rho^A_t(x)\rho^B_t(x)dx
NOTE:
All computations taken on Eulerian grid. Lots of memory, but fast compute.
Approach 1. Occupancy-Based Scene Graph
Finding Witness Points for the Algorithm
3. Find the average normal weighted by the product measure.
3.1 Note that points in the normal direction.
3.2 So we can average normals weighted by the product measure
\nabla \rho^A_t
\hat{n}^A = \int \frac{1}{2}\left[\nabla \rho^A_t(x) - \nabla \rho^B_t(x)\right] \rho^A_t(x)\rho^B_t(x)dx
\hat{n}^A = -\hat{n}^B
Approach 1. Occupancy-Based Scene Graph
Finding Witness Points for the Algorithm
4. Line search along the direction and find the find intersection where occupancy is above some probability
5. Pass the witness points into a point-contact based simulator (e.g. Pang's quasistatic)
Approach 2. Overlap Volume Method
Overlap Volume Method
(No dissipation formulation) Apply spring-like forces proportional to the overlap volume of the two measures, then use Newton-Euler.
F = \hat{n}^A \cdot \int \rho^A_t(x)\rho^B_t(x)dx
(No dissipation formulation) Apply spring-like forces proportional to the overlap volume of the two measures, then use Newton-Euler.
Limitations
Limitations
1. Non-convex geometry (peg-in-hole) may have non-intuitive properties
2. Two pairs of geometry should not be summarized into single point without moments (box-on-ground simulation, no moment balance)
Next Steps
3D extension (Does the grid-based formulation compute fast enough?)
Plug it to rigid-body simulation
How to connect it back to gradient-based optimization?
JMLR Ideas
How do we extend what we have right now?
In some sense, the question is always about "how to do policy gradient the right way". We've considered variation in terms of gradients, but there are other variations?
1. Where should we inject noise / take expectations to smooth out the irregular landscape? (dynamics, policy output, parameters)
Contact-Aware Control
What does stability and feedback control mean in the context of manipulation?
Copy of DeepContactDynamics
By Terry Suh
