Contextual Bandits
Contextual bandit algorithms extend the classic multi-armed bandit problem by incorporating side information
(context) about each decision. This allows for more sophisticated decision-making in real-world applications
like content recommendation, clinical trials, and online advertising.
Key Resources
- Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems - Bubeck & Cesa-Bianchi survey, Chapter 4 on contextual bandits
- Introduction to Multi-Armed Bandits - Aleksandrs Slivkins, 2019. Excellent modern introduction with contextual bandits coverage.
- Bandit Algorithms textbook - Lattimore & Szepesvári. Chapters 19-20 on contextual bandits.
Fundamental Papers
- A Contextual-Bandit Approach to Personalized News Article Recommendation - Li et al., 2010. Introduced LinUCB algorithm.
- Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits - Agarwal et al., 2014. Practical algorithm for large action spaces.
- Improved Algorithms for Agnostic Pool-based Active Classification - Chaudhuri et al., 2015. Modern theoretical results for active learning.
- Thompson Sampling for Contextual Bandits with Linear Payoffs - Agrawal & Goyal, 2013. Bayesian approach to contextual bandits.
Practical Applications
- Recommendation Systems: News articles, products, content personalization
- Online Advertising: Ad selection, auction bidding, campaign optimization
- Clinical Trials: Adaptive treatment assignment, precision medicine
- Resource Allocation: Network routing, job scheduling, A/B testing
mv