Multi Armed Bandits

An A/B test waits until the end to act on what it learned; a multi-armed bandit keeps allocating trials while learning which arm pays off. The page starts with a book, then bandits in recommender systems, then the business case against conventional A/B testing, and ends with Thompson sampling.

The same notes are in A/B Testing, Contextual Bandits, and Reinforcement Learning. The same notes are in Recommender Systems.

The reference to start from is the Book — bandit algorithms, the free online edition of Bandit Algorithms; its content is the same as the Cambridge University Press print edition, with minor typos corrected. Where the theory meets product, Bandits for recommender systems is Eugene Yan on industry examples, exploration strategies, warm-starting, off-policy evaluation, and more.

The same notes are in Recommender Systems.

The business argument for bandits is speed. Rolling out multi armed bandits for fast adaptive experimentation is about how to outperform conventional A/B testing when scaling up personalized messages and services. The algorithm most of that rests on is Thompson sampling for multi-armed bandit, a walkthrough of Thompson Sampling for decision-making under uncertainty; it was originally introduced by William R. Thompson in 1933 and is used in online advertising, clinical trials, and recommendation systems.