With amazing help from Jules Kruijswijk we are heavily progressing with “STREAMPY” (not sure we are sticking to the name though). STREAMPY is a python module that allows easy setup of a REST server (using Tornado & Redis) to solve MAB problems in data-streams. Basically, the REST server provides two calls:
- BASE_URL/getAction.json: A call to retrieve the suggested action given a context and the current parameters.
- BASE_URL/setReward.json: A call to store the rewards and update parameters.
The code executed at each call (e.g., Thompson sampling to decide on an action, and an online update of a Bayesian probit model when rewards come in) is fully customizable. Hence, the module aims to make it very easy to test different policies to solve contextual MAB problems and evaluate them in (web) applications.
We are proud to have the first working examples going, but are putting in a bunch of effort to test the whole thing, and provide some interfaces.
More soon 😉