(Supporting slides)
Shen Shen
Feb 23, 2021
ORF363/COS323 Computing and Optimization
Def: A set Ω is a convex set if
λx+(1−λ)y∈Ω, ∀x,y∈Ω,∀λ∈[0,1]
convex combination of x and y
convex combination coefficient
Convex Sets
Non-convex Sets
because...
- Hyperplanes: {x∣aTx=b}(a∈Rn,b∈R,a=0)
- Halfspaces: {x∣aTx≤b}(a∈Rn,b∈R,a=0)
n=3:3x1+4x2+5x3=1
n=2:3x1+4x2+≤2
- Euclidean balls: {x∣∣∣x−xc∣∣≤r} (xc∈Rn,r∈R,∥.∥ 2-norm)
- Ellipsoids: {x∣(x−xc)TP(x−xc)≤r}(xc∈Rn,r∈R,P≻0)
n=2
n=3
- The set of PSD matrices
- The set of co-positive matrices (HW)
A common recipe
1. Pick two arbitrary members, say x and y, from Ω
2. Write out the convex combination c
3. Make use of that x and y are from Ω, and λ∈[0,1]
4. Claim the c also belongs in Ω
Proof of the PSD claim:
Pick A⪰0,B⪰0
N.t.s λA+(1−λ)B⪰0
i.e., xT(λA+(1−λ)B)x≥0∀x
=λxTAx+(1−λ)xTBx
xT(λA+(1−λ)B)x
≥0
≥0
≥0
Useful fact: Ω1 convex, Ω2 convex ⇒Ω1∩Ω2 convex.
- Proof in last lecture.
- The union of convex sets may not be convex
- Polyhedrons: {x∣Ax≤b} are convex sets
e.g. m =4 (4 halfspaces), n=2 (2d)
A=−10010−111,b=0013
Def: A function f:Rn→R is convex if its domain is a convex set and
f(λx+(1−λ)y)≤λf(x)+(1−λ)f(y) ∀x,y∈domain(f),∀λ∈[0,1]
Simple examples
Convex functions
Non-convex functions
f is called a concave function if −f is convex
- All affine functions f(x)=aTx+b (for any a∈Rn,b∈R)
- Some quadratic functions (details later)
- All norms
Recall norm is a function f that satisfies:
a. f(αx)=∣α∣f(x),∀α∈R
b. f(x+y)≤f(x)+f(y)
c. f(x)≥0,∀x,f(x)=0⇒x=0
f(λx+(1−λ)y)
=λf(x)+(1−λ)f(y)
b.c. (b)
b.c. (a) and λ≥0
Proof (convexity of any norm):
⩽ f(λx)+f((1−λ)y)
Theorem: If a function f:Rn→R is convex, then all its sublevelsets are convex sets.
- Recall the sub-levelsets definition:
The α -sublevel set of a function f:Rn→R is the set Sα={x∈domain(f)∣f(x)≤α}
Theorem: If a function f:Rn→R is convex, then all its sublevelsets are convex sets.
- Recall the sub-levelsets definition:
The α -sublevel set of a function f:Rn→R is the set Sα={x∈domain(f)∣f(x)≤α}
- Instead, a function whose sub-levelsets are convex sets is called quasiconvex.
- The converse is not true:
Theorem: Suppose f:Rn→R is twice differentiable over its domain.
Then, the following are equivalent:
(i) f is convex.
(ii) f(y)≥f(x)+∇fT(x)(y−x),∀x,y∈dom(f)
(iii) ∇2f(x)⪰0,∀x∈dom(f) (i.e., the Hessian is psd ∀x∈dom(f)).
Intuition for (ii) f(y)≥f(x)+∇fT(x)(y−x),∀x,y∈dom(f)
Intuition for (iii) ∇2f(x)⪰0,∀x∈dom(f) (i.e., the Hessian is psd ∀x∈dom(f)).
Function has nonnegative curvature everywhere: "it curves up"
Recall we said, some quadratic functions are convex, this second order characterization makes identifying the "some" really easy
In one dimension: f′′(x)≥0,∀x∈dom(f)
Recall theorem says f convex ⇔∇2f(x)⪰0,∀x∈dom(f)
Consider a quadratic function f(x)=xTAx+bx+c (A symmetric)
when is f convex?
we know ∇2f(x)=2A
Text
f convex ⇔A⪰0 (independent of b,c)
(for convex functions)
If f1,…,fn are convex functions and ω1,…,ωn≥0, then
f(x)=ω1f1(x)+⋯+ωnfn(x) is also convex.
If f1,…,fm are convex functions then their pointwise maximum
f(x)=max{f1(x),f2(x),…fm(x)}
with dom(f)=dom(f1)∩dom(f2)∩⋯∩dom(fm), is also convex.
Suppose f:Rn→R,A∈Rn×m and b∈Rn. Define g:Rm→R by
g(x)=f(Ax+b)
with dom(g)={x∣Ax+b∈dom(f)}. Then, if f is convex, so is g; if f is concave so is g.
Let f:Rn→R be a convex function and fix some a,b,∈Rn. Then, the function g:R→R given by g(x)=f(ax+b) is convex.