Sarah Dean PRO
asst prof in CS at Cornell
Prof. Sarah Dean
MW 2:55-4:10pm
255 Olin Hall
1. Recap: MAB
2. Contextual Bandits
3. Linear Model
4. LinUCB
A simplified setting for studying exploration
Multi-Armed Bandits
Explore-then-Commit
Upper Confidence Bound
For \(t=1,...,T\):
Explore for \(N \approx T^{2/3}\),
\(R(T) \lesssim T^{2/3}\)
\(R(T) \lesssim \sqrt{T}\)
1. Recap: MAB
2. Contextual Bandits
3. Linear Model
4. LinUCB
Example: online advertising
Journalism
Programming
"Arms" are different job ads:
But consider different users:
CS Major
English Major
Example: online shopping
"Arms" are various products
But what about search queries, browsing history, items in cart?
Example: social media feeds
"Arms" are various posts: images, videos
Personalized to each user based on demographics, behavioral data, etc
Contextual Bandits
Contextual Bandits
1. Recap: MAB
2. Contextual Bandits
3. Linear Model
4. LinUCB
tempo
lyricism
tempo
lyricism
X
X
1. Recap: MAB
2. Contextual Bandits
3. Linear Model
4. LinUCB
LinUCB
tempo
lyricism
X
X
X
LinUCB
By Sarah Dean