The Mathematics of Optimization

Christopher Makler

Stanford University Department of Economics

Econ 50: Lecture 9

pollev.com/chrismakler

How many indifference curves are there of the utility function
\(u(x_1,x_2) = 2x_1 + x_2\)
if we're looking at a graph showing (0,0) to (10,10)?

Choice space:
all possible options

Feasible set:
all options available to you

Optimal choice:
Your best choice(s) of the ones available to you

Constrained Optimization

Choice Space
(all colleges plus alternatives)

Feasible Set
(colleges you got into)

Your optimal choice!

Preferences

Preferences describe how the agent ranks all options in the choice space.

For example, we'll assume that you could rank all possible colleges
(and other options for what to do after high school) based upon your preferences.

Preference Ranking

Found a startup

Harvard

Stanford

Play Xbox in parents' basement

Cal

Choice space

Feasible set

Optimal
choice!

Found a startup

Stanford

Cal

Harvard

Play XBox in parents' basement

Optimal choice is the highest-ranking option in the feasible set.

Today's Agenda

  • Unconstrained optimization with one variable
  • Constrained optimization with one variable
  • Unconstrained optimization with two variables
  • Constrained optimization with two variables
  • Interpreting the Lagrange multiplier

Unconstrained Optimization

Think about maximizing each of these functions subject to the constraint \(0 \le x \le 10\).

Plot the graph on that interval; then find and plot the derivative \(f'(x)\) on that same interval.

Which function(s) reach their maximum in the domain [0, 10] at a point where \(f'(x) = 0\)?

f(x) = 5 + 4x - x^2
f(x) = 10 - |2-x|
f(x) = 9 - (x-11)^2
f(x) = 1 + \tfrac{1}{5}(x-5)^2
f(x) = 10 - x
f(x) = 3

pollev.com/chrismakler

Sufficient conditions for an interior optimum characterized by \(f'(x)=0\) with constraint \(x \in [0,10]\)

  • \(f'(0) > 0\)
  • \(f'(10) < 0\)
  • \(f'(x)\) continuous and strictly decreasing on \([0,10]\)
f(x)
f'(x)
x
x
10
0
10

Incorporating the Constraint into the Optimization Problem: The Lagrange Multiplier Method

\max f(x)
\text{s.t. }g(x) \le k

OBJECTIVE FUNCTION

CONSTRAINT

EXAMPLE

\max u(x) = 144\sqrt{x}
\text{s.t. }px \le m

OBJECTIVE FUNCTION

CONSTRAINT

\max u(x) = 144\sqrt{x}
\text{s.t. }px \le m
\max u(x) = 144\sqrt{x}
\text{s.t. }m - px \ge 0

Step 1: rewrite the constraint as a nonnegativity constraint

Step 2: create a new combined objective function that "punishes" you for increasing \(x\)

\mathcal{L}(x,\lambda) =
144\sqrt{x}
+ \lambda (
)
m - px

Step 3: take the derivative of this with respect to \(x\) and \(\lambda\) and set them equal to zero

{\partial \mathcal{L}(x,\lambda) \over \partial x} =
{\partial \mathcal{L}(x,\lambda) \over \partial \lambda} =
{72 \over \sqrt{x}}
m - px
= 0
- \lambda
p
= 0
\mathcal{L}(x,\lambda) =
144\sqrt{x}
+ \lambda (
)
m - px
{\partial \mathcal{L}(x,\lambda) \over \partial x} =
{\partial \mathcal{L}(x,\lambda) \over \partial \lambda} =
{72 \over \sqrt{x}}
m - px
= 0
- \lambda
p
= 0
\lambda = {72/\sqrt{x} \over p}
x = {m \over p}
= {f'(x) \over g'(x)}
\max f(x)
\text{s.t. }g(x) \le k

OBJECTIVE FUNCTION

CONSTRAINT

\max u(x) = 144\sqrt{x}
\text{s.t. }px \le m

Optimization in Two Variables

Constrained Optimization

Canonical Constrained Optimization Problem

f(x_1,x_2)
\text{s.t. }
g(x_1,x_2) = k
k - g(x_1,x_2) = 0
\mathcal{L}(x_1,x_2,\lambda)=
\displaystyle{\max_{x_1,x_2}}
f(x_1,x_2)
k - g(x_1,x_2)
+ \lambda\ (
)

Suppose \(g(x_1,x_2)\) is monotonic (increasing in both \(x_1\) and \(x_2\)).

Then \(k - g(x_1,x_2)\) is negative if you're outside of the constraint,
positive if you're inside the constraint,
and zero if you're along the constraint.

OBJECTIVE

FUNCTION

CONSTRAINT

\mathcal{L}(x_1,x_2,\lambda)=
f(x_1,x_2)
k - g(x_1,x_2)
+ \lambda\ (
)
\displaystyle{\partial \mathcal{L} \over \partial x_1} =

FIRST ORDER CONDITIONS

\displaystyle{\partial \mathcal{L} \over \partial x_2} =
\displaystyle{\partial \mathcal{L} \over \partial \lambda} =
\displaystyle{\partial f \over \partial x_1}
\displaystyle{\partial f \over \partial x_2}
k - g(x_1,x_2)
=0
- \lambda\ \times
\displaystyle{\partial g \over \partial x_1}
\displaystyle{\partial g \over \partial x_2}
- \lambda\ \times
=0
=0

3 equations, 3 unknowns

 

Solve for \(x_1\), \(x_2\), and \(\lambda\)

How does the Lagrange method work?

It finds the point along the constraint where the
level set of the objective function passing through that point
is tangent to the constraint

\mathcal{L}(x_1,x_2,\lambda)=
f(x_1,x_2)
k - g(x_1,x_2)
+ \lambda\ (
)
\displaystyle{\partial \mathcal{L} \over \partial x_1} =

FIRST ORDER CONDITIONS

\displaystyle{\partial \mathcal{L} \over \partial x_2} =
\displaystyle{\partial \mathcal{L} \over \partial \lambda} =
\displaystyle{\partial f \over \partial x_1}
\displaystyle{\partial f \over \partial x_2}
k - g(x_1,x_2)
=0
- \lambda\ \times
\displaystyle{\partial g \over \partial x_1}
\displaystyle{\partial g \over \partial x_2}
- \lambda\ \times
=0
=0
\displaystyle{\Rightarrow \lambda\ = {{\partial f /\partial x_1} \over {\partial g/\partial x_1}}}
\displaystyle{\Rightarrow \lambda\ = {{\partial f /\partial x_2} \over {\partial g/\partial x_2}}}
\displaystyle{\Rightarrow {{\partial f /\partial x_1} \over {\partial f/\partial x_2}} = {{\partial g /\partial x_1} \over {\partial g/\partial x_2}}}

TANGENCY
CONDITION

CONSTRAINT

Example: Fence Problem

You have 40 feet of fence and want to enclose the maximum possible area.

Example: Fence Problem

You have 40 feet of fence and want to enclose the maximum possible area.

f(x_1,x_2)
\text{s.t. }
g(x_1,x_2) = k
k - g(x_1,x_2) = 0
\mathcal{L}(L,W,\lambda)=
\displaystyle{\max_{x_1,x_2}}
L \times W
40 - 2L - 2W
+ \lambda\ (
)

OBJECTIVE

FUNCTION

CONSTRAINT

L \times W
40 - 2L - 2W = 0
2L + 2W = 40
\mathcal{L}(L,W,\lambda)=
L \times W
40 - 2L - 2W
+ \lambda\ (
)
\displaystyle{\partial \mathcal{L} \over \partial L} =

FIRST ORDER CONDITIONS

\displaystyle{\partial \mathcal{L} \over \partial W} =
\displaystyle{\partial \mathcal{L} \over \partial \lambda} =
W
L
40 - 2L - 2W
=0
- \lambda\ \times
2
2
- \lambda\ \times
=0
=0
\displaystyle{\Rightarrow \lambda\ = {W \over 2}}
\displaystyle{\Rightarrow \lambda\ = {L \over 2}}
W = L

TANGENCY
CONDITION

CONSTRAINT

2L + 2W = 40
W = L

TANGENCY
CONDITION

CONSTRAINT

2L + 2W = 40

Meaning of the Lagrange multiplier

Suppose you have \(F\) feet of fence instead of 40.

\displaystyle{\partial \mathcal{L} \over \partial L} =
\displaystyle{\partial \mathcal{L} \over \partial W} =
\displaystyle{\partial \mathcal{L} \over \partial \lambda} =
W
L
F - 2L - 2W
=0
- \lambda\ \times
2
2
- \lambda\ \times
=0
=0
\displaystyle{\Rightarrow \lambda\ = {W \over 2}}
\displaystyle{\Rightarrow \lambda\ = {L \over 2}}
W = L

TANGENCY
CONDITION

CONSTRAINT

2L + 2W = F

SOLUTIONS

L^* = {F \over 4}
W^* = {F \over 4}
\lambda^* = {F \over 8}

Meaning of the Lagrange multiplier

Suppose you have \(F\) feet of fence instead of 40.

SOLUTIONS

L^* = {F \over 4}
W^* = {F \over 4}
\lambda^* = {F \over 8}

Maximum enclosable area as a function of F:

A^*(F) = L^*(F) \times W^*(F) = {F \over 4} \times {F \over 4} = {F^2 \over 16}

Meaning of the Lagrange multiplier

Suppose you have \(F\) feet of fence instead of 40.

\mathcal{L}(L,W,\lambda)=
L \times W
40 - 2L - 2W
+ \lambda\ (
)

A Note on the Homework

  • The last two problems show you situations in which the Lagrange method may fail
  • Use the exercise you got in section but didn't have time to get to to derive some intuition for what's going on...

Next Week

  • Use this technique to find Chuck's optimal choice along his PPF
  • Review and prepare for the midterm
  • SEND IN YOUR OAE LETTERS NOW!!!!