Title Text

Code as Policies: Language Model Programs for Embodied Control

Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, Andy Zeng

*Robotics at Google

TALK TO THE ROBOT

GENERAL-PURPOSE ROBOTS

DISCUSSION

DEMOS

• Robot programming is a classic way to control a robot
• Large language models (LLMs) have trained on millions of lines of code
• LLMs can convert natural language instructions into robot code

• An alternative approach towards generalist robots that can do many tasks
• Compose code libraries in different ways to solve new problems
• Broadly applicable to other applications

Learn more at:

code-as-policies.github.io

• Human-centered robots: code is interpretable
• Safety should be built-in within control code
• Limitation: restricted by scope of APIs
• Limitation: "build a house of candies"

  "How many fruits are on the table?" Robot: "2"

  "Follow the convex hull containing the chairs"

  "Draw the sun and a pyramid on the ground. Then erase."

  "Circle the blocks in the scene."

  "Throw away the coke can and the apple."