MIT 6.4210/2: Robotic Manipulation
Fall 2026, Lecture 1
Matthew T. Mason. Toward Robotic Manipulation. Annual Review of Control, Robotics, and Autonomous Systems, 1:1-28, 2018.
Website: http://manipulation.mit.edu/Fall2026
Website: http://manipulation.mit.edu/Fall2026
Communications Instructors from Comparative Media Studies/Writing
Recitations Fridays (starting Sept 18)
For communication tasks
Read the full policy
For homework and project implementation- unlimited
But we expect you to understand what is generated. Exam questions based on material.
Read the policy
TODO
Links
Joints (Revolute, Prismatic)
Sensors
Proprioception (position, torque)
Force and/or tactile
Vision: RGB(D), Laser Scan
Actuation
Motors (electric, hydraulic)
Transmission (gear reduction, backlash, friction)
Motor drivers (position control, torque control)
Exploring robot arms
https://www.infineon.com/cms/en/discoveries/robotics-systems-safety-security/
https://spectrum.ieee.org/robotics/industrial-robots/rethink-robotics-baxter-robot-factory-worker
Universal Robots
Flexiv
Kuka iiwa
Kinova Jaco
ABB Yumi
Franka Panda
Schunk WSG 50
Force-controlled planar gripper
Sandia Hand
iHY (i-Robot, Harvard, Yale)
Robotiq 3-fingered gripper
Allegro Hand
Shadow hand
Rainbow RBY1
Figure
To be clear: we're not just controlling the arm
state of the robot x state of the environment
Low-level feedback control from cameras / sensors
(continuous time/state/action)
Scene-understanding +
Task-level Planning
(discrete/symbolic)
From Kris Hauser - Robotic Systems (online)
Levine*, Finn*, Darrel, Abbeel, JMLR 2016
perception network
(often pre-trained)
policy network
other robot sensors
learned state representation
actions
Manipulation is much more than "pick and place"!
C. Garrett, C. Paxton, T. Lozano-Perez, L. Kaelbling, D. Fox 2020
Denny Freeman
Drake models computations as Systems, wired together in a Diagram.
Example: Simple LeafSystem
class DoubleSystem(LeafSystem):
def __init__(self):
super().__init__()
self.DeclareVectorInputPort('u', 1)
self.DeclareVectorOutputPort('y', 1, self.CalcY)
def CalcY(self, context: Context, output: BasicVector) -> None:
u = self.get_input_port('u').Eval(context)
output.SetFromVector([2*u[0]])a = BasicVector(3) # 3-d vector
a.SetFromVector(array) # from numpy array
Diagram Composition
Diagrams allow composition of multiple subsystems.
- Each system has input/output ports.
- Ports are connected to build larger systems.
builder = DiagramBuilder()
builder.AddSystem(sys1)
builder.AddSystem(sys2)
builder.Connect(sys1.get_output_port(0), sys2.get_input_port(0))
diagram = builder.Build()class VerticalMassUnderGravity(LeafSystem):
"""
Continuous-time 1D vertical mass:
ż = v
v̇ = g + f/m (constant gravity)
State vector: x = [z, v]
"""
def __init__(self, gravity=-9.81, mass=1.0):
super().__init__()
self._g = float(gravity)
self._m = float(mass)
# Continuous state: q (z), v (ż)
self.DeclareContinuousState(2)
# Input force f
self.DeclareVectorInputPort("force", 1)
# Output: position
self.DeclareVectorOutputPort("position", 1, self.CalcPosition)
def CalcPosition(self, context: Context, output: BasicVector) -> None:
x = context.get_continuous_state_vector()
output.SetFromVector([x[0]])
def DoCalcTimeDerivatives(self, context:Context, derivatives:ContinuousState) -> None:
x = context.get_continuous_state_vector().CopyToVector()
f = self.get_input_port(0).Eval(context)
z, v = x
z_dot = v
v_dot = self._g + float(f)/self._m
derivatives.get_mutable_vector().SetFromVector([z_dot, v_dot])if __name__ == "__main__":
builder = DiagramBuilder()
mass_sys = builder.AddSystem(VerticalMassUnderGravity())
# Two ways of doing the logger
logger = LogVectorOutput(mass_sys.get_output_port(0), builder)
# logger = builder.AddSystem(VectorLogSink(1))
# builder.Connect(mass_sys.get_output_port(0), logger.get_input_port(0))
diagram = builder.Build()
ctx = diagram.CreateDefaultContext()
sim = Simulator(diagram, ctx)
ctx.get_mutable_continuous_state().SetFromVector([10.0, 0.0])
sys_ctx = mass_sys.GetMyContextFromRoot(ctx)
mass_sys.get_input_port(0).FixValue(sys_ctx, 10.0)
sim.AdvanceTo(2.0) # simulate 2 seconds
z_final = mass_sys.get_output_port(0).Eval(sys_ctx)
print("Final position [z] =", z_final)
plot(sim)
Drake's Physics Engine: MultibodyPlant
MultibodyPlant is Drake's class for modeling robots and mechanisms.
builder = DiagramBuilder()
plant, scene_graph = AddMultibodyPlantSceneGraph(builder, time_step=1e-4)
parser = Parser(plant, scene_graph)
parser.AddModelsFromUrl("package://drake_models/iiwa_description/urdf/iiwa14_primitive_collision.urdf")
plant.Finalize()
diagram = builder.Build()