The Mathematics of Optimization
Christopher Makler
Stanford University Department of Economics
Econ 50: Lecture 9
pollev.com/chrismakler
How many indifference curves are there of the utility function
\(u(x_1,x_2) = 2x_1 + x_2\)
if we're looking at a graph showing (0,0) to (10,10)?
Choice space:
all possible options
Feasible set:
all options available to you
Optimal choice:
Your best choice(s) of the ones available to you
Constrained Optimization
Choice Space
(all colleges plus alternatives)
Feasible Set
(colleges you got into)
Your optimal choice!
Preferences
Preferences describe how the agent ranks all options in the choice space.
For example, we'll assume that you could rank all possible colleges
(and other options for what to do after high school) based upon your preferences.
Preference Ranking
Found a startup
Harvard
Stanford
Play Xbox in parents' basement
Cal
Choice space
Feasible set
Optimal
choice!
Found a startup
Stanford
Cal
Harvard
Play XBox in parents' basement
Optimal choice is the highest-ranking option in the feasible set.
Today's Agenda
- Unconstrained optimization with one variable
- Constrained optimization with one variable
- Unconstrained optimization with two variables
- Constrained optimization with two variables
- Interpreting the Lagrange multiplier
Unconstrained Optimization
Think about maximizing each of these functions subject to the constraint \(0 \le x \le 10\).
Plot the graph on that interval; then find and plot the derivative \(f'(x)\) on that same interval.
Which function(s) reach their maximum in the domain [0, 10] at a point where \(f'(x) = 0\)?
f(x) = 5 + 4x - x^2
f(x) = 10 - |2-x|
f(x) = 9 - (x-11)^2
f(x) = 1 + \tfrac{1}{5}(x-5)^2
f(x) = 10 - x
f(x) = 3
pollev.com/chrismakler
Sufficient conditions for an interior optimum characterized by \(f'(x)=0\) with constraint \(x \in [0,10]\)
- \(f'(0) > 0\)
- \(f'(10) < 0\)
- \(f'(x)\) continuous and strictly decreasing on \([0,10]\)
f(x)
f'(x)
x
x
10
0
10
Incorporating the Constraint into the Optimization Problem: The Lagrange Multiplier Method
\max f(x)
\text{s.t. }g(x) \le k
OBJECTIVE FUNCTION
CONSTRAINT
EXAMPLE
\max u(x) = 144\sqrt{x}
\text{s.t. }px \le m
OBJECTIVE FUNCTION
CONSTRAINT
\max u(x) = 144\sqrt{x}
\text{s.t. }px \le m
\max u(x) = 144\sqrt{x}
\text{s.t. }m - px \ge 0
Step 1: rewrite the constraint as a nonnegativity constraint
Step 2: create a new combined objective function that "punishes" you for increasing \(x\)
\mathcal{L}(x,\lambda) =
144\sqrt{x}
+ \lambda (
)
m - px
Step 3: take the derivative of this with respect to \(x\) and \(\lambda\) and set them equal to zero
{\partial \mathcal{L}(x,\lambda) \over \partial x} =
{\partial \mathcal{L}(x,\lambda) \over \partial \lambda} =
{72 \over \sqrt{x}}
m - px
= 0
- \lambda
p
= 0
\mathcal{L}(x,\lambda) =
144\sqrt{x}
+ \lambda (
)
m - px
{\partial \mathcal{L}(x,\lambda) \over \partial x} =
{\partial \mathcal{L}(x,\lambda) \over \partial \lambda} =
{72 \over \sqrt{x}}
m - px
= 0
- \lambda
p
= 0
\lambda = {72/\sqrt{x} \over p}
x = {m \over p}
= {f'(x) \over g'(x)}
\max f(x)
\text{s.t. }g(x) \le k
OBJECTIVE FUNCTION
CONSTRAINT
\max u(x) = 144\sqrt{x}
\text{s.t. }px \le m
Optimization in Two Variables
Constrained Optimization
Canonical Constrained Optimization Problem
f(x_1,x_2)
\text{s.t. }
g(x_1,x_2) = k
k - g(x_1,x_2) = 0
\mathcal{L}(x_1,x_2,\lambda)=
\displaystyle{\max_{x_1,x_2}}
f(x_1,x_2)
k - g(x_1,x_2)
+ \lambda\ (
)
Suppose \(g(x_1,x_2)\) is monotonic (increasing in both \(x_1\) and \(x_2\)).
Then \(k - g(x_1,x_2)\) is negative if you're outside of the constraint,
positive if you're inside the constraint,
and zero if you're along the constraint.
OBJECTIVE
FUNCTION
CONSTRAINT
\mathcal{L}(x_1,x_2,\lambda)=
f(x_1,x_2)
k - g(x_1,x_2)
+ \lambda\ (
)
\displaystyle{\partial \mathcal{L} \over \partial x_1} =
FIRST ORDER CONDITIONS
\displaystyle{\partial \mathcal{L} \over \partial x_2} =
\displaystyle{\partial \mathcal{L} \over \partial \lambda} =
\displaystyle{\partial f \over \partial x_1}
\displaystyle{\partial f \over \partial x_2}
k - g(x_1,x_2)
=0
- \lambda\ \times
\displaystyle{\partial g \over \partial x_1}
\displaystyle{\partial g \over \partial x_2}
- \lambda\ \times
=0
=0
3 equations, 3 unknowns
Solve for \(x_1\), \(x_2\), and \(\lambda\)
How does the Lagrange method work?
It finds the point along the constraint where the
level set of the objective function passing through that point
is tangent to the constraint
\mathcal{L}(x_1,x_2,\lambda)=
f(x_1,x_2)
k - g(x_1,x_2)
+ \lambda\ (
)
\displaystyle{\partial \mathcal{L} \over \partial x_1} =
FIRST ORDER CONDITIONS
\displaystyle{\partial \mathcal{L} \over \partial x_2} =
\displaystyle{\partial \mathcal{L} \over \partial \lambda} =
\displaystyle{\partial f \over \partial x_1}
\displaystyle{\partial f \over \partial x_2}
k - g(x_1,x_2)
=0
- \lambda\ \times
\displaystyle{\partial g \over \partial x_1}
\displaystyle{\partial g \over \partial x_2}
- \lambda\ \times
=0
=0
\displaystyle{\Rightarrow \lambda\ = {{\partial f /\partial x_1} \over {\partial g/\partial x_1}}}
\displaystyle{\Rightarrow \lambda\ = {{\partial f /\partial x_2} \over {\partial g/\partial x_2}}}
\displaystyle{\Rightarrow {{\partial f /\partial x_1} \over {\partial f/\partial x_2}} = {{\partial g /\partial x_1} \over {\partial g/\partial x_2}}}
TANGENCY
CONDITION
CONSTRAINT
Example: Fence Problem
You have 40 feet of fence and want to enclose the maximum possible area.
Example: Fence Problem
You have 40 feet of fence and want to enclose the maximum possible area.
f(x_1,x_2)
\text{s.t. }
g(x_1,x_2) = k
k - g(x_1,x_2) = 0
\mathcal{L}(L,W,\lambda)=
\displaystyle{\max_{x_1,x_2}}
L \times W
40 - 2L - 2W
+ \lambda\ (
)
OBJECTIVE
FUNCTION
CONSTRAINT
L \times W
40 - 2L - 2W = 0
2L + 2W = 40
\mathcal{L}(L,W,\lambda)=
L \times W
40 - 2L - 2W
+ \lambda\ (
)
\displaystyle{\partial \mathcal{L} \over \partial L} =
FIRST ORDER CONDITIONS
\displaystyle{\partial \mathcal{L} \over \partial W} =
\displaystyle{\partial \mathcal{L} \over \partial \lambda} =
W
L
40 - 2L - 2W
=0
- \lambda\ \times
2
2
- \lambda\ \times
=0
=0
\displaystyle{\Rightarrow \lambda\ = {W \over 2}}
\displaystyle{\Rightarrow \lambda\ = {L \over 2}}
W = L
TANGENCY
CONDITION
CONSTRAINT
2L + 2W = 40
W = L
TANGENCY
CONDITION
CONSTRAINT
2L + 2W = 40
Meaning of the Lagrange multiplier
Suppose you have \(F\) feet of fence instead of 40.
\displaystyle{\partial \mathcal{L} \over \partial L} =
\displaystyle{\partial \mathcal{L} \over \partial W} =
\displaystyle{\partial \mathcal{L} \over \partial \lambda} =
W
L
F - 2L - 2W
=0
- \lambda\ \times
2
2
- \lambda\ \times
=0
=0
\displaystyle{\Rightarrow \lambda\ = {W \over 2}}
\displaystyle{\Rightarrow \lambda\ = {L \over 2}}
W = L
TANGENCY
CONDITION
CONSTRAINT
2L + 2W = F
SOLUTIONS
L^* = {F \over 4}
W^* = {F \over 4}
\lambda^* = {F \over 8}
Meaning of the Lagrange multiplier
Suppose you have \(F\) feet of fence instead of 40.
SOLUTIONS
L^* = {F \over 4}
W^* = {F \over 4}
\lambda^* = {F \over 8}
Maximum enclosable area as a function of F:
A^*(F) = L^*(F) \times W^*(F) = {F \over 4} \times {F \over 4} = {F^2 \over 16}
Meaning of the Lagrange multiplier
Suppose you have \(F\) feet of fence instead of 40.
\mathcal{L}(L,W,\lambda)=
L \times W
40 - 2L - 2W
+ \lambda\ (
)
A Note on the Homework
- The last two problems show you situations in which the Lagrange method may fail
- Use the exercise you got in section but didn't have time to get to to derive some intuition for what's going on...
Next Week
- Use this technique to find Chuck's optimal choice along his PPF
- Review and prepare for the midterm
- SEND IN YOUR OAE LETTERS NOW!!!!
Econ 50 | Lecture 09
By Chris Makler
Econ 50 | Lecture 09
The Mathematics of Optimization
