Lecture 4: Trust Regions

Prof. Sarah Dean

MW 1:25-2:40pm
155 Olin Hall

Reminders

Homework
- PSet 1 due tonight
- Project 1 due 9/11
Enrollment: join waitlist, come to lecture, do assignments, there should be plenty of space
Unofficial audit: email for Canvas access

1. Why Gradients?

2. KL & Trust Regions

3. NPG Derivation

4. PPO Derivation

How should we update the policy parameters to improve the expected reward?

\begin{align*} \arg\max_{\|u\|_2\leq \epsilon} c + v^\top u \end{align*}

What is the solution to the following optimization problem?

By Sarah Dean

asst prof in CS at Cornell