Contextual Bandits

A plain bandit picks arms from their history alone; a contextual bandit chooses actions using side information too. The page opens with an overview deck and a personalization talk, then a Tools section with the Python packages that implement the idea.

The same notes are in A/B Testing, Multi Armed Bandits, and Reinforcement Learning.

The overview is Contextual bandits, the Contextual Bandits.pdf deck. It came via personalization in practice, the personalization talk it belongs to.

Tools

After the overview, the question is what to run. contextualbandits.com is the hosted option — adapt your creative according to context and outcomes automatically without A/B-sitting your campaigns. For code, Striatum is ntucllab/striatum on GitHub, contextual bandits in python.

The fuller package is Contextual bandits, in python by David Cortes, and its README says what it is for:

"This Python package contains implementations of methods from different papers dealing with contextual bandit problems, as well as adaptations from typical multi-armed bandits strategies. It aims to provide an easy way to prototype and compare ideas, to reproduce research papers that don't provide easily-available implementations of their proposed algorithms, and to serve as a guide in learning about contextual bandits. For details about the implementations, or if you would like to cite this in your research, see "Adapting multi-armed bandits policies to contextual bandits scenarios"."

The figure below is from that package.

Contextual bandits package.

Deprecated links

These links and images no longer work. The original wording is kept here. A same-resource copy, when one was checked, is used above.

• contextualbandits.com — adapt your creative according to context and outcomes automatically without A/B-sitting your campaigns. This address no longer opens: https://www.contextualbandits.com/