pollev.com/chrismakler

The Mathematics of Optimization

Christopher Makler

Stanford University Department of Economics

Econ 50: Lecture 7

Today's Agenda

  • Solution functions and comparative statics
  • Unconstrained optimization
  • Constrained optimization and the Lagrange method

Solution Functions and Comparative Statics

Math problems with numbers

Math problems with parameters

Plot the line \(y = 5 - \frac{1}{2}x\)

10

5

4

3

2

7

6

1

9

12

11

2

1

4

3

6

5

8

7

x = 0 \Rightarrow y = 5
y = 0 \Rightarrow x = 10

Plot the line \(y = a - bx\)

x = 0 \Rightarrow y = a
y = 0 \Rightarrow x = \frac{a}{b}
a
\tfrac{a}{b}
y = a - bx
y = 5 - \tfrac{1}{2}x

Math problems with numbers

Math problems with parameters

Find the intersection of the lines \(y = 5 - \frac{1}{2}x\) and \(y = 2x\)

10

5

4

3

2

7

6

1

9

12

11

2

1

4

3

6

5

8

7

\text{Solve for }x: 5 - \tfrac{1}{2}x = 2x \Rightarrow x^* = 2
\text{Plug into }y = 2x \Rightarrow y^* = 4

Find the intersection of the lines \(y = a - bx\) and \(y = cx\)

a - bx = cx \Rightarrow x^*(a,b,c) = {a \over b + c}
y = cx \Rightarrow y^*(a,b,c) = \frac{ac}{b+c}
\tfrac{ac}{b + c}
\tfrac{a}{b+c}

Find the intersection of the lines \(y = 5 - \frac{1}{2}x\) and \(y = 2x\)

a
\tfrac{a}{b}
y = a - bx
y = cx
y = 5 - \tfrac{1}{2}x
y = 2x
X^* = (2,4)
X^*(a,b,c) = \left({a \over b+c},{ac \over b + c}\right)

[SOLUTIONS]

[SOLUTION FUNCTIONS]

x^*(a,b,c) = {a \over b + c}
y^*(a,b,c) = \frac{ac}{b+c}

What happens to the intersection when \(a\), \(b\), or \(c\) increases?

Comparative Statics

  • When you solve a parameterized problem,
    the result is a solution function: the endogenous variables as functions of the exogenous parameters
     
  • Comparative statics is the analysis of the behavior of these solution functions: how does a change in the parameters of the problem affect its solution?

Unconstrained Optimization

Constrained Optimization

Think about maximizing each of these functions subject to the constraint \(0 \le x \le 10\).

Plot the graph on that interval; then find and plot the derivative \(f'(x)\) on that same interval.

Which function(s) reach their maximum in the domain [0, 10] at a point where \(f'(x) = 0\)?

f(x) = 5 + 4x - x^2
f(x) = 10 - |2-x|
f(x) = 9 - (x-11)^2
f(x) = 1 + \tfrac{1}{5}(x-5)^2
f(x) = 10 - x
f(x) = 3

pollev.com/chrismakler

Sufficient conditions for an interior optimum characterized by \(f'(x)=0\) with constraint \(x \in [0,10]\)

  • \(f'(0) > 0\)
  • \(f'(10) < 0\)
  • \(f'(x)\) continuous and strictly decreasing on \([0,10]\)
f(x)
f'(x)
x
x
10
0
10

Canonical Constrained Optimization Problem

f(x_1,x_2)
\text{s.t. }
g(x_1,x_2) = k
k - g(x_1,x_2) = 0
\mathcal{L}(x_1,x_2,\lambda)=
\displaystyle{\max_{x_1,x_2}}
f(x_1,x_2)
k - g(x_1,x_2)
+ \lambda\ (
)

Suppose \(g(x_1,x_2)\) is monotonic (increasing in both \(x_1\) and \(x_2\)).

Then \(k - g(x_1,x_2)\) is negative if you're outside of the constraint,
positive if you're inside the constraint,
and zero if you're along the constraint.

OBJECTIVE

FUNCTION

CONSTRAINT

\mathcal{L}(x_1,x_2,\lambda)=
f(x_1,x_2)
k - g(x_1,x_2)
+ \lambda\ (
)
\displaystyle{\partial \mathcal{L} \over \partial x_1} =

FIRST ORDER CONDITIONS

\displaystyle{\partial \mathcal{L} \over \partial x_2} =
\displaystyle{\partial \mathcal{L} \over \partial \lambda} =
\displaystyle{\partial f \over \partial x_1}
\displaystyle{\partial f \over \partial x_2}
k - g(x_1,x_2)
=0
- \lambda\ \times
\displaystyle{\partial g \over \partial x_1}
\displaystyle{\partial g \over \partial x_2}
- \lambda\ \times
=0
=0

3 equations,
3 unknowns

 

Solve for \(x_1\), \(x_2\), and \(\lambda\)

How does the Lagrange method work?

It finds the point along the constraint where the
level set of the objective function passing through that point
is tangent to the constraint

\mathcal{L}(x_1,x_2,\lambda)=
f(x_1,x_2)
k - g(x_1,x_2)
+ \lambda\ (
)
\displaystyle{\partial \mathcal{L} \over \partial x_1} =

FIRST ORDER CONDITIONS

\displaystyle{\partial \mathcal{L} \over \partial x_2} =
\displaystyle{\partial \mathcal{L} \over \partial \lambda} =
\displaystyle{\partial f \over \partial x_1}
\displaystyle{\partial f \over \partial x_2}
k - g(x_1,x_2)
=0
- \lambda\ \times
\displaystyle{\partial g \over \partial x_1}
\displaystyle{\partial g \over \partial x_2}
- \lambda\ \times
=0
=0
\displaystyle{\Rightarrow \lambda\ = {{\partial f /\partial x_1} \over {\partial g/\partial x_1}}}
\displaystyle{\Rightarrow \lambda\ = {{\partial f /\partial x_2} \over {\partial g/\partial x_2}}}
\displaystyle{\Rightarrow {{\partial f /\partial x_1} \over {\partial f/\partial x_2}} = {{\partial g /\partial x_1} \over {\partial g/\partial x_2}}}

TANGENCY
CONDITION

CONSTRAINT

Example: Fence Problem

You have 40 feet of fence and want to enclose the maximum possible area.

Example: Fence Problem

You have 40 feet of fence and want to enclose the maximum possible area.

f(x_1,x_2)
\text{s.t. }
g(x_1,x_2) = k
k - g(x_1,x_2) = 0
\mathcal{L}(L,W,\lambda)=
\displaystyle{\max_{x_1,x_2}}
L \times W
40 - 2L - 2W
+ \lambda\ (
)

OBJECTIVE

FUNCTION

CONSTRAINT

L \times W
40 - 2L - 2W = 0
2L + 2W = 40
\mathcal{L}(L,W,\lambda)=
L \times W
40 - 2L - 2W
+ \lambda\ (
)
\displaystyle{\partial \mathcal{L} \over \partial L} =

FIRST ORDER CONDITIONS

\displaystyle{\partial \mathcal{L} \over \partial W} =
\displaystyle{\partial \mathcal{L} \over \partial \lambda} =
W
L
40 - 2L - 2W
=0
- \lambda\ \times
2
2
- \lambda\ \times
=0
=0
\displaystyle{\Rightarrow \lambda\ = {W \over 2}}
\displaystyle{\Rightarrow \lambda\ = {L \over 2}}
W = L

TANGENCY
CONDITION

CONSTRAINT

2L + 2W = 40
W = L

TANGENCY
CONDITION

CONSTRAINT

2L + 2W = 40

Meaning of the Lagrange multiplier

Suppose you have \(F\) feet of fence instead of 40.

\displaystyle{\partial \mathcal{L} \over \partial L} =
\displaystyle{\partial \mathcal{L} \over \partial W} =
\displaystyle{\partial \mathcal{L} \over \partial \lambda} =
W
L
F - 2L - 2W
=0
- \lambda\ \times
2
2
- \lambda\ \times
=0
=0
\displaystyle{\Rightarrow \lambda\ = {W \over 2}}
\displaystyle{\Rightarrow \lambda\ = {L \over 2}}
W = L

TANGENCY
CONDITION

CONSTRAINT

2L + 2W = F

SOLUTION

FUNCTIONS

L^*(F) = {F \over 4}
W^*(F) = {F \over 4}
\lambda^*(F) = {F \over 8}

Meaning of the Lagrange multiplier

SOLUTION

FUNCTIONS

L^*(F) = {F \over 4}
W^*(F) = {F \over 4}
\lambda^*(F) = {F \over 8}

Maximum enclosable area as a function of F:

A^*(F) =
L^*(F) \times W^*(F)
= {F \over 4} \times {F \over 4}
= {F^2 \over 16}

Meaning of the Lagrange multiplier

In a constrained optimization problem,
the constraint may be determined by a parameter:
how much labor you have, how much money you have,
how many units of a good you want to produce, etc.

The Lagrange multiplier tells you how much
the optimized value of the objective function will change
due to a change in that parameter.

"How much more utility could you get
if you had another hour of labor,
and used it optimally?"

Next Steps

  • Tomorrow night: math homework due
  • Rest of this week: use these techniques
    to solve for the optimal division of labor
    between two competing uses

Meaning of the Lagrange multiplier

SOLUTION FUNCTIONS

L^* = {F \over 4}
W^* = {F \over 4}
\lambda^* = {F \over 8}