Russ Meeting

2022/08/30

Agendas 

  • NeRF Real-To-Sim Project

  • JMLR Submission

  • Contact-Aware Control 

NERF Project

Assumptions:

- Known state (pose of the object is known)

- Rigid Object Assumption

- Unknown Shape, implicitly given some occupancy measure representation

Approach 1. Occupancy-Based Scene Graph

Finding Witness Points for the Algorithm

 

1. Given to measures

2. Find the first moment of the product measure

 

This point denotes the "point of action" for the two measures. 

 

 

 

 

 

 

\rho^A_t, \rho^B_t
\bar{x} = \int x \rho^A_t(x)\rho^B_t(x)dx

NOTE:

All computations taken on Eulerian grid. Lots of memory, but fast compute. 

Approach 1. Occupancy-Based Scene Graph

Finding Witness Points for the Algorithm

3. Find the average normal weighted by the product measure.

      3.1 Note that               points in the normal direction. 

      3.2 So we can average normals weighted by the product measure

\nabla \rho^A_t
\hat{n}^A = \int \frac{1}{2}\left[\nabla \rho^A_t(x) - \nabla \rho^B_t(x)\right] \rho^A_t(x)\rho^B_t(x)dx
\hat{n}^A = -\hat{n}^B

Approach 1. Occupancy-Based Scene Graph

Finding Witness Points for the Algorithm

4. Line search along the direction and find the find intersection where occupancy is above some probability

    

5. Pass the witness points into a point-contact based simulator (e.g. Pang's quasistatic)

    

Approach 2. Overlap Volume Method

Overlap Volume Method   

(No dissipation formulation) Apply spring-like forces proportional to the overlap volume of the two measures, then use Newton-Euler.    

F = \hat{n}^A \cdot \int \rho^A_t(x)\rho^B_t(x)dx

(No dissipation formulation) Apply spring-like forces proportional to the overlap volume of the two measures, then use Newton-Euler.    

Limitations

Limitations

1. Non-convex geometry (peg-in-hole) may have non-intuitive properties 

2. Two pairs of geometry should not be summarized into single point without moments (box-on-ground simulation, no moment balance)

Next Steps

3D extension (Does the grid-based formulation compute fast enough?)

Plug it to rigid-body simulation

How to connect it back to gradient-based optimization?

JMLR Ideas

How do we extend what we have right now? 


In some sense, the question is always about "how to do policy gradient the right way". We've considered variation in terms of gradients, but there are other variations?


1. Where should we inject noise / take expectations to smooth out the irregular landscape? (dynamics, policy output, parameters)



Contact-Aware Control

What does stability and feedback control mean in the context of manipulation?