PhD Oral Presentation
An Investigation on the use of Analytical Tools
Victor Sanches Portella
July 30, 2024
, and Martingales
Privacy
Experts
,
Analytical Methods in CS
(and Back!)
Martingales
Privacy
Experts
∫abf(x)dx
\int_a^b f(x) \mathrm{d}x
Stochastic Calc.
& Online Learning
Vector Calculus and Probability Theory
Stochastic Calculus
Privacy
Differential Privacy
M
\displaystyle
\mathcal{M}
M
\displaystyle
\mathcal{M}
Output 1
Output 2
Not too far apart
What does it mean for an algorithm M to be private?
Differential Privacy: M does not rely heavily on any individual
(ε,δ)-Diff. Privacy
ε≡ "Privacy leakage", small constant
δ≡ "Chance of failure", usually O(1/#samples)
{
\displaystyle
\Bigg \{
Covariance Estimation
x1,x2,…,xn∼N(0,Σ)
\displaystyle
x_1, x_2, \dotsc, x_n \sim \mathcal{N}(0, \Sigma)
Σ≻0
\displaystyle
\Sigma \succ 0
Unknown Covariance Matrix
X∈Rd×n
\displaystyle
X \in \mathbb{R}^{d \times n}
(ε,δ)-differentially private M to estimate Σ
on Rd
Goal:
Required even without privacy
Required even for d=1
Is this tight?
Exists (ε,δ)-DP M such that
n=O~(α2d2+εlog(1/δ)+αεd2)
\displaystyle
n = \tilde O\Big(\frac{d^2}{\alpha^2} + \frac{\log(1/\delta)}{\varepsilon} + \frac{d^2}{\alpha \varepsilon}\Big)
E[∥M(X)−Σ∥F2]≤α2
\displaystyle
\mathbb{E}[\lVert\mathcal{M}(X) - \Sigma\rVert_F^2] \leq \alpha^2
samples
Known algorithmic results
with
Our Results - New Lower Bounds
Theorem
For any (ε,δ)-DP algorithm M such that
E[∥M(X)−Σ∥F2]≤α2=O(d)
\displaystyle
\mathbb{E}\big[\lVert\mathcal{M}(X) - \Sigma\rVert_F^2\big] \leq \alpha^2 = O(d)
and
δ=O(nlnn1)
\displaystyle
\delta = O\Big( \frac{1}{n \ln n}\Big)
we have
n=Ω(αεd2)
\displaystyle
n = \Omega\Big(\frac{d^2}{\alpha\varepsilon}\Big)
Our results generalize both of them
Nearly highest reasonable value
δ=O~(d21)=o(n1)
\displaystyle
\delta = \tilde O\Big(\frac{1}{d^2}\Big) = o\Big(\frac{1}{n}\Big)
α=O(1)
\displaystyle
\alpha = O(1)
[Kamath et al. 22]
Previous lower bounds required
n=Ω(αεd2)
\displaystyle
n = \Omega\big(\tfrac{d^2}{\alpha\varepsilon}\big)
{
\displaystyle
\Bigg \{
[Narayanan 23]
OR
Lower Bounds via Fingerprinting
A measure of the correlation between z and M(X)
Correlation statistic
A(z,M(X))
\displaystyle
\mathcal{A}(z, \mathcal{M}(X))
E[∣A(z,M(X))∣]
\displaystyle
\mathbb{E}[|\mathcal{A}(z, \mathcal{M}(X))|]
If z∼N(0,Σ) indep. of X
small
E[A(xi,M(X))]
\displaystyle
\mathbb{E}[\mathcal{A}(x_i, \mathcal{M}(X))]
If M is accurate
large
Fingerprinting Lemma
Σ
\displaystyle
\Sigma
zz⊺
\displaystyle
zz^{\intercal}
zz⊺
\displaystyle
zz^{\intercal}
zz⊺
\displaystyle
zz^{\intercal}
zz⊺
\displaystyle
zz^{\intercal}
M(X)
\displaystyle
\mathcal{M}(X)
Σ
\displaystyle
\Sigma
x3x3⊺
\displaystyle
x_3 x_3^{\intercal}
x1x1⊺
\displaystyle
x_1 x_1^{\intercal}
x2x2⊺
\displaystyle
x_2 x_2^{\intercal}
x4x4⊺
\displaystyle
x_4 x_4^{\intercal}
M(X)
\displaystyle
\mathcal{M}(X)
Approx. equal by privacy
Approx. equal by privacy
Score Statistic
A(z,M(X))=⟨M(X)−Σ,∇ΣlogpN(z∣Σ)⟩
\displaystyle
\mathcal{A}(z, \mathcal{M}(X)) = \big\langle\mathcal{M}(X) - \Sigma, \nabla_{\Sigma} \log p_{\mathcal{N}}(z \;|\; \Sigma)\big\rangle
Score function
Score Attack Statistic
To get a fingerprinting lemma, we need to randomize Σ so that
E[A(xi,M(X))]
\displaystyle
\mathbb{E}[\mathcal{A}(x_i, \mathcal{M}(X))]
is large
Previous work
Σ with "small radius" ⟹ Weak FP Lemma
Σ with "large radius" ⟹ hard to bound E[∣A(z,M(X))∣]
A(z,M(X))=⟨M(X)−Σ,zz⊺−Σ⟩
\displaystyle
\mathcal{A}(z, \mathcal{M}(X)) = \langle\mathcal{M}(X) - \Sigma, z z^{\intercal} - \Sigma \rangle
zz⊺
\displaystyle
zz^{\intercal}
M(X)
\displaystyle
\mathcal{M}(X)
Σ
\displaystyle
\Sigma
New Fingerprinting Lemma
Fingerprinting Lemma
Need to Lower Bound
E[i,j∑∂ijg(Σ)ij]
\displaystyle \mathbb{E}\Big[
\sum_{i,j }\partial_{ij} \; g(\Sigma)_{ij}\Big]
g(Σ)=E[M(X)]
\displaystyle
g(\Sigma) = \mathbb{E}[\mathcal{M}(X)]
Σ∼ Wishart leads to elegant analysis
Stein-Haff Identity
"Move the derivative" from g to p with integration by parts
divg(Σ)
\displaystyle
\mathrm{div} g(\Sigma)
E[divg(Σ)]=∫divg(Σ)⋅p(Σ)dΣ
\displaystyle
\mathbb{E}[
\mathrm{div} g(\Sigma)] =
\int \mathrm{div} g(\Sigma) \cdot p(\Sigma) \mathrm{d}\Sigma
Stokes' Theorem
i=1∑nE[A(xi,M(X))]
\displaystyle
\sum_{i = 1}^n \mathbb{E}[\mathcal{A}(x_i, \mathcal{M}(X))]
≥Ω(d2)
\displaystyle
\geq \Omega(d^2)
O(nαε)≥
\displaystyle
O(n \alpha \varepsilon) \geq
FP Lemma
Upper Bound
Experts
Player
Adversary
n Experts
0.5
0.1
0.3
0.1
Probabilities
pt
p_t
1
-1
0.5
-0.3
Gains
gt∈[−1,1]n
g_t \in [-1,1]^n
Player's gain:
⟨gt,pt⟩
\langle g_t, p_t \rangle
E[gt(i)]
\mathbb{E}[g_t(i)]
Prediction with Expert's Advice
Regret(T)=i=1,…,nmaxt=1∑Tgt(i)−t=1∑T⟨gt,pt⟩
\displaystyle
\mathrm{Regret}(T) = \max_{i = 1, \dotsc, n} \sum_{t = 1}^T
g_t(i) - \sum_{t = 1}^T \langle g_t, p_t \rangle
Gain of Best Expert
Player's Gain
Quantile Regret
Best Expert
Best Experts
ε-fraction
}
\displaystyle
\Bigg \}
Multiplicative Weights Update:
≲Tln(1/ε)
\displaystyle
\lesssim \sqrt{T \ln (1/\varepsilon)}
Needs knowledge of ε
We design an algorith with Tln(1/ε) quantile regret
for all ε and best known leading constant
t=1∑Tgt(iε)−t=1∑T⟨gt,pt⟩
\displaystyle
\sum_{t = 1}^T
g_t(i_{\varepsilon}) - \sum_{t = 1}^T \langle g_t, p_t \rangle
Gain of
top εn expert
ε-Quantile Regret
Moving to Continuous Time
Analysis often becomes clean
Sandbox for design of optimization algorithms
Key Question: How to model non-smooth (online) optimization in continuous time?
Why go to ?
continuous time
Discrete Time
Gt(i)=s=1∑tgs(i)
\displaystyle
G_t(i) = \sum_{s = 1}^t g_s(i)
Useful perspective: G(i) is a realization of a random walk
Continuous Time
Gt(i) is a realization of a Brownian Motion
Gt(i)=Bt(i)
\displaystyle
G_t(i) = B_t(i)
Worst-case =
Probability 1
Improved Quantile Regret
Potential based players
p(t)∝∇Φ(t,R(t))
\displaystyle
p(t) \propto \nabla \Phi(t, R(t))
Stochastic Calculus suggests Φ that satisfy the Backwards Heat Equation
Φ+Φ=0
\;\;\;\;\Phi + \;\;\;\;\;\;\; \Phi = 0
∂t
\displaystyle
\partial_t
21∂xx
\displaystyle
\tfrac{1}{2}\partial_{xx}
QuantRegret(T,ε)≤22Tln(1/ε)+6T
\displaystyle
\mathrm{QuantRegret}(T, \varepsilon) \leq 2\sqrt{2 T \ln(1/\varepsilon)} + 6 \sqrt{T}
For all ε
Using this potential*, we get
Best leading constant
Discrete time analysis is IDENTICAL to continuous time analysis
*(with a slightly bigger cnst. in the BHE)
Anytime Experts
Question:
Are the minimax regret with and without knowledge of T different?
fixed-time
anytime
anytime
fixed-time
2Tlnn
\displaystyle
\sqrt{2 T \ln n}
Theorem: In Continuous Time, both are equal if Brownian Motions are independent.
MinmaxRegret
2Tlnn
\displaystyle
2\sqrt{T \ln n}
≤
\displaystyle
\leq
Can we get better lower bounds?
=
?
Martingales
Motivation: Expected Regret
Regret(τ)=∥Gτ∥∞−Aτ
\displaystyle
\mathrm{Regret}(\tau) = \lVert \phantom{G_\tau} \rVert_{\infty} - \phantom{A_\tau}
Gτ
\displaystyle
G_\tau
Aτ
\displaystyle
A_\tau
Player's Total Gain
Vector of the Experts' Gains
E[Aτ]=0
\displaystyle
\mathbb{E}[A_\tau] = 0
High expected regret ⟹ anytime lower bound
Max expected anytime regret without independent experts?
Anytime Regret ≡ τ is a stopping time
E[τ]E[Regret(τ)]
\displaystyle
\frac{\mathbb{E}[\mathrm{Regret}(\tau)]}{\mathbb{E}[\sqrt{\tau}]}
How big can
be?
Norm of High-Dimensional Martingales
For a martingale (Gt)t≥0, find upper and lower bounds to
E[τ]E[∥Gτ∥∞]
\displaystyle
\frac{\mathbb{E}[\lVert G_{\tau}\rVert_{\infty}]}{\mathbb{E}[\sqrt{\tau}]}
sup
{
\displaystyle
\Bigg\{
:
\displaystyle
:
τ
\displaystyle
\tau
is a stopping time
}
\displaystyle
\Bigg\}
Kn=
\displaystyle
K_n =
Kn
\displaystyle
K_n
≤λ(n−1)
\displaystyle
\leq \lambda(n-1)
cn≤
\displaystyle
c_n \leq
∼2lnn
\displaystyle
\sim \sqrt{2 \ln n}
2lnn∼
\displaystyle
\sqrt{2 \ln n} \sim
No assumptions
on the dependency between coordinates
Theorem
If Gt(i) is a Brownian motion for all i=1,…,n, then
Evidence that Anytime Lower Bounds for
continuous experts needs new techniques
Other Results
For a martingale (Gt)t≥0, find upper and lower bounds to
E[τ]E[∥Gτ∥∞]
\displaystyle
\frac{\mathbb{E}[\lVert G_{\tau}\rVert_{\infty}]}{\mathbb{E}[\sqrt{\tau}]}
sup
{
\displaystyle
\Bigg\{
:
\displaystyle
:
τ
\displaystyle
\tau
is a stopping time
}
\displaystyle
\Bigg\}
Kn=
\displaystyle
K_n =
Similar upper bounds when Gt(i) has smooth quadratic variation
If Gt(i) is a discrete martingale with increments in [−1,1], we have
Kn≤λ(n−1)
\displaystyle
K_n\leq \lambda(n-1)
Beyond Brownian Motion
Discrete Martingales
Discrete Ito's Lemma
Main Ideas of the Analysis
Goal:
E[∥Gτ∥∞−βτ]≤0
\displaystyle
\mathbb{E}\big[\lVert G_\tau \rVert_{\infty} - \beta \sqrt{\tau} \big] \leq 0
=E[∥G0∥−β0]
\displaystyle
= \mathbb{E}\big[\lVert G_{0} \rVert - \beta \sqrt{0} \big]
non-smooth ⟹ hard to show it is a supermartingale
Idea:
Design a smooth function Φ such that
(Φ(t,Gt))t≥0
is a supermartingale
and
∥x∥∞−βt≤Φ(t,x)
\displaystyle
\lVert x \rVert_{\infty} - \beta \sqrt{t} \leq \Phi(t, x)
Backwards Heat Eq.
Tune Constants
Summary
Martingales
Tight bounds on expected norm for a large family of martingales. Nearly tight bounds and implications for the experts problem.
Experts
Continuous-time model for the experts' problem and new algorithms. Sandbox for online learning algorithms.
Privacy
New lower-bounds for private covariance estimation. Techniques suggest a proof strategy for lower bounds in DP.
PhD Oral Presentation
An Investigation on the use of Analytical Tools
Victor Sanches Portella
July 30, 2024
, and Martingales
Privacy
Experts
,
Fingerprinting and Stein's Identity
A(z,M(X))=⟨M(X)−Σ,∇ΣlogpN(z∣Σ)⟩
\displaystyle
\mathcal{A}(z, \mathcal{M}(X)) = \big\langle\mathcal{M}(X) - \Sigma, \nabla_{\Sigma} \log p_{\mathcal{N}}(z \;|\; \Sigma)\big\rangle
Score function
If z∼N(0,Σ) indep. of X
For x1,…,xn from X
E[∣A(z,M(X))∣]≤λmin(Σ)α
\displaystyle
\mathbb{E}[|\mathcal{A}(z, \mathcal{M}(X))|] \leq \frac{\alpha}{\lambda_{\min}(\Sigma)}
i=1∑nE[A(xi,M(X))]=i,j∑∂Σij∂E[M(X)ij]
\displaystyle
\sum_{i = 1}^n \mathbb{E}[\mathcal{A}(x_i, \mathcal{M}(X))] = \sum_{i,j} \frac{\partial}{\partial_{\Sigma_{ij}}} \mathbb{E}[\mathcal{M}(X)_{ij}]
=Θ(d2) if E[M(X)]=Σ
Lower Bounds via Fingerprinting
Correlation statistic
A(z,M(X))=⟨M(X)−Σ,zz⊺−Σ⟩
\displaystyle
\mathcal{A}(z, \mathcal{M}(X)) = \langle\mathcal{M}(X) - \Sigma, z z^{\intercal} - \Sigma \rangle
E[∣A(z,M(X))∣]
\displaystyle
\mathbb{E}[|\mathcal{A}(z, \mathcal{M}(X))|]
If z∼N(0,Σ) indep. of X
small
E[A(xi,M(X))]
\displaystyle
\mathbb{E}[\mathcal{A}(x_i, \mathcal{M}(X))]
If M is accurate
large
M(X)
\displaystyle
\mathcal{M}(X)
zz⊺
\displaystyle
zz^{\intercal}
Σ
\displaystyle
\Sigma
zz⊺
\displaystyle
zz^{\intercal}
zz⊺
\displaystyle
zz^{\intercal}
zz⊺
\displaystyle
zz^{\intercal}
M(X)
\displaystyle
\mathcal{M}(X)
x1x1⊺
\displaystyle
x_1 x_1^{\intercal}
Σ
\displaystyle
\Sigma
x3x3⊺
\displaystyle
x_3 x_3^{\intercal}
x2x2⊺
\displaystyle
x_2 x_2^{\intercal}
x4x4⊺
\displaystyle
x_4 x_4^{\intercal}
Approx. equal by privacy
Approx. equal by privacy
Fingerprinting Lemma
for covariance estimation
A(z,M(X))
leads to limited lower bounds
Lower Bounds via Fingerprinting
Correlation statistic
A(z,M(X))=⟨M(X)−Σ,zz⊺−Σ⟩
\displaystyle
\mathcal{A}(z, \mathcal{M}(X)) = \langle\mathcal{M}(X) - \Sigma, z z^{\intercal} - \Sigma \rangle
E[∣A(z,M(X))∣]
\displaystyle
\mathbb{E}[|\mathcal{A}(z, \mathcal{M}(X))|]
If z∼N(0,Σ) indep. of X
small
E[A(xi,M(X))]
\displaystyle
\mathbb{E}[\mathcal{A}(x_i, \mathcal{M}(X))]
If M is accurate
large
M(X)
\displaystyle
\mathcal{M}(X)
zz⊺
\displaystyle
zz^{\intercal}
Σ
\displaystyle
\Sigma
zz⊺
\displaystyle
zz^{\intercal}
zz⊺
\displaystyle
zz^{\intercal}
zz⊺
\displaystyle
zz^{\intercal}
M(X)
\displaystyle
\mathcal{M}(X)
x1x1⊺
\displaystyle
x_1 x_1^{\intercal}
Σ
\displaystyle
\Sigma
x3x3⊺
\displaystyle
x_3 x_3^{\intercal}
x2x2⊺
\displaystyle
x_2 x_2^{\intercal}
x4x4⊺
\displaystyle
x_4 x_4^{\intercal}
Approx. equal by privacy
Approx. equal by privacy
Fingerprinting Lemma
for covariance estimation
A(z,M(X))
leads to limited lower bounds
Online Learning and Experts
Prediction with Expert's Advice
Player
Adversary
n Experts
0.5
0.1
0.3
0.1
Probabilities
xt
x_t
1
-1
0.5
-0.3
Costs
ℓt
\ell_t
Player's loss:
⟨ℓt,xt⟩
\langle \ell_t, x_t \rangle
Adversary knows the strategy of the player
E[ℓt(i)]
\mathbb{E}[\ell_t(i)]
Performance Measure - Regret
Regret(T)=t=1∑T⟨ℓt,pt⟩−i=1,…,nmint=1∑Tℓt(i)
\displaystyle
\mathrm{Regret}(T) = \sum_{t = 1}^T \langle \ell_t, p_t \rangle - \min_{i = 1, \dotsc, n} \sum_{t = 1}^T
\ell_t(i)
Loss of Best Expert
Player's Loss
Regret(T)≤2Tlnn
\displaystyle
\mathrm{Regret}(T) \leq \sqrt{2 T \ln n}
Optimal!
E[Regret(T)]≥2Tlnn(1−o(1))
\displaystyle
\mathbb{E}[\mathrm{Regret}(T)] \geq \sqrt{2 T \ln n}(1 - o(1))
For random ±1 costs
Multiplicative Weights Update:
pt+1(i)∝pt(i)⋅exp(−ηℓt(i))
\displaystyle
p_{t+1}(i) \propto p_t(i) \cdot \exp(-\eta \ell_t(i) )
(Hedge)
Why Learning with Experts?
Boosting in ML
Understanding sequential prediction
Universal Optimization
Solving SDPs, TCS, Learning theory...
Motivating Problem - Fixed Time vs Anytime
MWU regret
Regret(T)≤2Tlnn
\displaystyle
\mathrm{Regret}(T) \leq \sqrt{2 T \ln n}
when T is known
Regret(T)≤2Tlnn
\displaystyle
\mathrm{Regret}(T) \leq 2\sqrt{T \ln n}
when T is not known
{
\displaystyle
\Bigg\{
anytime
fixed-time
Does knowing T gives the player an advantage?
[Harvey, Liaw, Perkins, Randhawa '23]
Random cost are (probably) too easy to show separation
[VSP, Liaw, Harvey '22]
Anytime > Fixed time for 2 experts + optimal algorithm
+ new algorithms for quantile regret!
With stochastic calculus:
Continuous Experts' Problem
Modeling Adversarial Costs in Continuous Time
Analysis often becomes clean
Sandbox for design of optimization algorithms
Gradient flow is useful for smooth optimization
∂txt=−∇f(xt)
\displaystyle
\partial_t x_t = - \nabla f(x_t)
How to model non-smooth (adversarial) optimization in continuous time?
Why go to ?
continuous time
Modeling Adversarial Costs in Continuous Time
Total loss of expert i:
Lt(i)=s=1∑tℓs(i)
\displaystyle
L_t(i) = \sum_{s = 1}^t \ell_s(i)
Useful perspective: L(i) is a realization of a random walk
realization of a Brownian Motion
Probability 1 = Worst-case
Discrete Time
Continuous Time
Lt(i)=Bt(i)
\displaystyle
L_t(i) = B_t(i)
The Continuous Time Model
Discrete time
Continuous time
Regret(t)=imaxRt(i)
\displaystyle
\mathrm{Regret}(t) = \max_{i} R_t(i)
Rt(i)=At−Lt(i)
\displaystyle
R_t(i) = A_t - L_t(i)
Lt(i)=Bt(i)
\displaystyle
L_t(i) =B_t(i)
Lt(i)=Rand.Walk
\displaystyle
L_t(i) = \mathrm{Rand. Walk}
Cummulative loss
Player's cummulative loss
t=1∑T⟨pt,ΔLt⟩
\displaystyle
\sum_{t = 1}^T\langle p_t, \Delta L_t\rangle
∫0T⟨pt,dLt⟩
\displaystyle
\int_0^T\langle p_t, \mathrm{d} L_t \rangle
At
\displaystyle
A_t
At
\displaystyle
A_t
Player's loss per round
⟨pt,ℓt⟩
\displaystyle
\langle p_t, \ell_t\rangle
⟨pt,dLt⟩
\displaystyle
\langle p_t, \mathrm{d} L_t \rangle
[Freund '09]
ΔLt
\displaystyle
\Delta L_t
MWU in Continuous Time
Potential based players
p(t)∝∇xΦ(t,Rt)
\displaystyle
p(t) \propto \nabla_x \Phi(t, R_t)
Φ(t,R(t))=ln(i∑e−ηtRt(i))
\displaystyle
\Phi(t, R(t)) = \ln \left( \sum_{i} e^{-\eta_t R_t(i)} \right)
pt(i)∝e−ηtLt(i)
\displaystyle
p_t(i) \propto e^{-\eta_t L_t(i)}
⟹
\displaystyle
\implies
Regret bounds
Regret(T)≤2Tlnn
\displaystyle
\mathrm{Regret}(T) \leq \sqrt{2 T \ln n}
when T is known
Regret(T)≤2Tlnn
\displaystyle
\mathrm{Regret}(T) \leq 2\sqrt{T \ln n}
when T is not known
anytime
fixed-time
{
\displaystyle
\Bigg\{
MWU!
Same as discrete time!
Idea: Use stochastic calculus to guide the algorithm design
with prob. 1
A Peek Into the Analysis
Ito's Lemma
(Fundamental Theorem of Stochastic Calculus)
Φ(T,RT)−Φ(0,R0)
\displaystyle
\Phi(T, R_T) - \Phi(0, R_0)
+∫0T(∂tΦ(t,Rt)+21∂xxΦ(t,Rt))dt
\displaystyle
+ \int_0^T \Big( \partial_t \Phi(t, R_t) + \frac{1}{2}\partial_{xx} \Phi(t, R_t) \Big) \mathrm{d} t
=∫0T∇xΦ(t,Rt)dRt
\displaystyle
= \int_0^T \nabla_x \Phi(t, R_t) \mathrm{d} R_t
f(Bt)−f(B0)=∫0Tf′(Bt)dBt
\displaystyle
f(B_t) - f(B_0) = \int_{0}^T f'(B_t) \mathrm{d} B_t
+ 21∫0Tf′′(Bt)dt
\displaystyle
+ ~\frac{1}{2} \int_{0}^T f'' (B_t) \mathrm{d} t
Bt is very non-smooth ⟹ second-order terms matter
n=1
Ito's Lemma
Potential does not change too much
=0
\displaystyle
= 0
≈0
\displaystyle
\approx 0
Would be great
A Peek Into the Analysis
Potential based players
p(t)∝∇Φ(t,R(t))
\displaystyle
p(t) \propto \nabla \Phi(t, R(t))
Regret(T)≤2Tlnn+o(1)
\displaystyle
\mathrm{Regret}(T) \leq \sqrt{2 T \ln n} + o(1)
Matches fixed-time!
Stochastic calculus suggests pontential that satisfy the Backwards Heat Equation
Φ+Φ=0
\;\;\;\;\Phi + \;\;\;\;\;\;\; \Phi = 0
∂t
\displaystyle
\partial_t
21∂xx
\displaystyle
\tfrac{1}{2}\partial_{xx}
This new anytime algorithm has good regret!
Does not translate easily to discrete time
need to add correlation between experts
Take away: independent experts cannot give better lower-bounds (in continuous-time)
Beyond i.i.d. Experts
dL1(t)=w1,1(t)dB1(t)+w1,2(t)dB2(t)+
\mathrm{d}L_1(t) = \phantom{w_{1,1}(t)} \mathrm{d} B_1(t) + \phantom{w_{1,2}(t)} \mathrm{d} B_2(t)\; +
dL2(t)=w2,1(t)dB1(t)+w2,2(t)dB2(t)+
\displaystyle
\mathrm{d}L_2(t) = \phantom{w_{2,1}(t)} \mathrm{d} B_1(t) + \phantom{w_{2,2}(t)} \mathrm{d} B_2(t) \; +
dLn(t)=wn,1(t)dB1(t)+wn,2(t)dB2(t)+
\displaystyle
\mathrm{d}L_n(t) = \phantom{w_{n,1}(t)} \mathrm{d} B_1(t) + \phantom{w_{n,2}(t)} \mathrm{d} B_2(t) \; +
⋮
\displaystyle
\vdots
+w2,n(t)dBn(t)
\displaystyle
+ \; \phantom{w_{2,n}(t)} \mathrm{d} B_n(t)
+wn,n(t)dBn(t)
\displaystyle
+ \; \phantom{w_{n,n}(t)} \mathrm{d} B_n(t)
⋮
\displaystyle
\vdots
⋮
\displaystyle
\vdots
+w1,n(t)dBn(t)
\displaystyle
+ \; \phantom{w_{1,n}(t)}\mathrm{d} B_n(t)
⋮
\displaystyle
\vdots
⋮
\displaystyle
\vdots
w1,1(t)
w_{1,1}(t)
w1,2(t)
w_{1,2}(t)
w1,n(t)
w_{1,n}(t)
w2,1(t)
w_{2,1}(t)
w2,2(t)
w_{2,2}(t)
w2,n(t)
w_{2,n}(t)
wn,1(t)
w_{n,1}(t)
wn,2(t)
w_{n,2}(t)
wn,n(t)
w_{n,n}(t)
Discrete time analysis is IDENTICAL to continuous time analysis
Improved anytime algorithms with bounds
quantile regret
Discrete Ito's
Lemma
Online Linear Optimization
Based on work by Zhang, Yang, Cutkosky, Paschalidis
Online Linear Optimization
Player
Adversary
xt∈Rn
x_t \in \mathbb{R}^n
Unconstrained
ft(x)=⟨gt,x⟩
f_t(x) = \langle g_t, x \rangle
Linear functions
∥gt∥≤1
\displaystyle
\lVert g_t\rVert \leq 1
Player's loss:
ft(xt)
f_t(x_t)
Regret(T,u)=t=1∑T⟨gt,xt⟩−t=1∑T⟨gt,u⟩
\displaystyle
\mathrm{Regret}(T,\phantom{u}) = \sum_{t = 1}^T \langle g_t, x_t \rangle - \sum_{t = 1}^T \langle g_t, \phantom{u} \rangle
u
\displaystyle u
u
\displaystyle u
Loss of Fixed u
Player's Loss
Parameter-Free Online Linear Optimization
Goal:
Regret(T,u)=O(∥u∥Tlog(∥u∥))
\displaystyle
\mathrm{Regret}(T,u) = O(\lVert u\rVert \sqrt{T \log(\lVert u \rVert )})
Parameter-Free = No knowledge of ∥u∥
Even better:
O(∥u∥VTlog(∥u∥))
\displaystyle
O(\lVert u\rVert \sqrt{V_T \log(\lVert u \rVert )})
VT=t=1∑T∥gt∥2
\displaystyle
V_T = \sum_{t = 1}^T \lVert g_t \rVert^2
"Adaptive" = Adapts to gradient norm
A One Dimensional Continuous Time Model
t=1∑Txt⋅gt−t=1∑Tu⋅gt
\displaystyle \sum_{t = 1}^T x_t \cdot g_t - \sum_{t = 1}^T u \cdot g_t
∫t=0Tx(t,Gt)dGt−u⋅Gt
\displaystyle
\int_{t = 0}^T x(t, G_t )\mathrm{d} G_t - u \cdot G_t
Discrete Regret
Continuos Regret
Theorem:
x(t,Gt)=∂xΦ(t,−Gt)
\displaystyle
x(t, G_t) = \partial_x \Phi(t, - G_t)
If Φ satisfies the BHE and
ContRegret(T,u)≤Φ(0,0)+Φ∗([G]t,u)
\displaystyle
\mathrm{ContRegret}(T, u) \leq \Phi(0,0) + \Phi^*([G]_t, u)
Going to higher dim:
Continuous time analogue
VT=∑t=1Tgt2
V_T = \sum_{t = 1}^T g_t^2
of
Learn direction and scale separately
Use refined discretization
Discretizing:
Conclusion and Open Questions
Continuous Time Model for Experts and OLO
Thanks!
Main References:
[VSP, Liaw, Harvey '22] Continuous prediction with experts' advice.
[Zhang, Yang, Cutkosky, Paschalidis '24] Improving adaptive online learning using refined discretization.
[Freund '09] A method for hedging in continuous time.
[Harvey, Liaw, Perkins, Randhawa '23] Optimal anytime regret with two experts.
How to discretize the algorithm for indep. experts?
?
Improve LB for anytime experts?
?
High-dim continuous time OLO?
?
PhD Oral Presentation An Investigation on the use of Analytical Tools Victor Sanches Portella July 30, 2024 , and Martingales Privacy Experts ,
PhD Oral Presentation
By Victor Sanches Portella
PhD Oral Presentation
- 160
