MAB seems to find a local maxima subject to input biases whereas an AB test is aimed to figure out a scientific truth and isolates out all potential biases in the system. I'd be curious to hear where a MAB approach and an AB test did not yield the same results and why that happened.
The multi-armed bandit problem (2012)
41–50 of 82 posts
Re: The multi-armed bandit problem (2012)
#42We (Facebook) just released our Adaptive Experimentation platform as open source: https://github.com/facebook/Ax. It supports bandit optimization for discrete sets of candidates and bayesian optimization for continuous/very large search spaces.
Re: The multi-armed bandit problem (2012)
#43Earlier quoted context omitted.
Statistical significance isn't necessary for deriving value from information! The point of multi-armed bandit is that you use the best information you currently have, while also not taking that information too seriously. In a context where experiments have a cost and you need results, this makes more sense than gathering more and more data until you meet statistical significance thresholds.
If random noise is more likely to explain the validity of one hypothesis over another, then your information has very little value.
Go take a look at some solutions to the multi-armed bandit problem. A common trait is assigning value (ie. monetary) to each hypothesis. A weaker positive-outcome hypothesis is worth less. The method takes that into account.
Re: The multi-armed bandit problem (2012)
#44Bandit algorithms can also be approached from a Bayesian point of view! This lets you quantify the uncertainty in your estimates (e.g. how uncertain you are than ad A has a higher CTR than ad B), which a lot of other bandit methods don't offer.
To toot my own horn a bit, I wrote a blog post about Bayesian bandits: https://eigenfoo.xyz/bayesian-bandits/
Re: The multi-armed bandit problem (2012)
#45Earlier quoted context omitted.
Statistical significance isn't necessary for deriving value from information! The point of multi-armed bandit is that you use the best information you currently have, while also not taking that information too seriously. In a context where experiments have a cost and you need results, this makes more sense than gathering more and more data until you meet statistical significance thresholds.
If random noise is more likely to explain the validity of one hypothesis over another, then your information has very little value.
Guessing based on uncertain data seems better than guessing randomly? Also if the signal to noise ratio is low my data is likely just noise, but at the same time the decision doesn't matter much. The clearer the difference the more important the decision and the more likely my data is right.
Re: The multi-armed bandit problem (2012)
#46Great to see Multi-Armed Bandits getting coverage here. I wrote an article last year to help with gaining a deeper understanding on the topic:
https://towardsdatascience.com/bandits-for-recommender-syste...
Re: The multi-armed bandit problem (2012)
#47I published this work in CHI on the use of multiarmed bandits in educational games. My biggest takeaway was the importance of choosing the right metric -- because our optimization worked too well.
https://www.researchgate.net/publication/301935710_Interface...
Re: The multi-armed bandit problem (2012)
#48The purpose of an A/B test isn't to always show the best performing result, it's to perform a _controlled scientific experiment_ with a control group, from which you can learn things. Also, I work in this field and I will just say that people _do_ behave differently based on traffic source: i.e. users coming from Facebook behave alike, but different than traffic from Reddit who act similarly to each other. If you wer…
> The purpose of an A/B test isn't to always show the best performing result The key part is 'not always'. Using A/B tests just to find the better performing version is a valid use case too. And if you have the traffic, multi-armed bandits are a nice way of automating the whole procedure. Even scientifically speaking there is nothing wrong with them. Their biggest issue is that they require a lot of traffic for signi…
In practice, MAB should require less traffic than an A/B test for the same level of significance. Although it's much more difficult to rigorously describe that significance level with a MAB.
Re: The multi-armed bandit problem (2012)
#49The purpose of an A/B test isn't to always show the best performing result, it's to perform a _controlled scientific experiment_ with a control group, from which you can learn things. Also, I work in this field and I will just say that people _do_ behave differently based on traffic source: i.e. users coming from Facebook behave alike, but different than traffic from Reddit who act similarly to each other. If you wer…
> Also, I work in this field and I will just say that people _do_ behave differently based on traffic source: i.e. users coming from Facebook behave alike, but different than traffic from Reddit who act similarly to each other.
Out of curiosity, what is your hypothesis for explaining this difference in behavior? Would you say it's primarily due to differing contexts in which a link is posted, or differing populations on each platform? Or maybe these both contribute about equally?
Stated another way: would you expect the same individual to behave differently coming from Facebook versus coming from reddit, if they happen to be a user of both?
Re: The multi-armed bandit problem (2012)
#50Earlier quoted context omitted.
I disagree. I’ve spent a lot of time staring at bandit outcomes and usually they match some sort of intuition of why a variant might be exceptional.
That could be post-hoc reasoning, though. It would be interesting to pre-register your hypotheses, or see whether you could tell bandit outcomes from random ones.
Sure it’s post-hoc reasoning, but it doesn’t matter because I’m not trying to invalidate a hypothesis.
I’m looking for variants that win. When I find one that wins I look at it and try to add more of the same flavor to the product.
This process works.