The multi-armed bandit problem (2012)

Post by taeric » Tue, May 28, 2019, 1:07 AM UTC

orasis wrote: post #50

comicjk wrote: post #22

Earlier quoted context omitted.

That could be post-hoc reasoning, though. It would be interesting to pre-register your hypotheses, or see whether you could tell bandit outcomes from random ones.

Sure it’s post-hoc reasoning, but it doesn’t matter because I’m not trying to invalidate a hypothesis. I’m looking for variants that win. When I find one that wins I look at it and try to add more of the same flavor to the product. This process works.

Post by marcosdumay » Tue, May 28, 2019, 2:11 AM UTC

opportune wrote: post #39

daenz wrote: post #8

It seems close to simulated annealing in the travelling salesman problem. The basics of it is that you start with a random tour and adjust path segments incrementally. Most of the time, you choose a new path segment that decreases the global route cost, but randomly, you choose a new path segment that increases the global route cost. This random factor is decreased over time, so the route anneals and settles on a clo…

Basically you are comparing one strategy to prevent local optima in one optimization algorithm to another strategy in a different optimization algorithm. There are such strategies for basically every optimization problem. Other examples: momentum in deep learning, tabu search, random search, etc. The benefit of doing a pure binary A/B test is that the experiment is so simple that as long as you don't break the cardin…

Post by mobjack » Tue, May 28, 2019, 2:47 AM UTC

paulddraper wrote: post #61

err4nt wrote: post #9

The purpose of an A/B test isn't to always show the best performing result, it's to perform a _controlled scientific experiment_ with a control group, from which you can learn things. Also, I work in this field and I will just say that people _do_ behave differently based on traffic source: i.e. users coming from Facebook behave alike, but different than traffic from Reddit who act similarly to each other. If you wer…

> The purpose of an A/B test isn't to always show the best performing result, it's to perform a _controlled scientific experiment_ with a control group, from which you can learn things. Occasionally there is pure scientific interest. But far more frequently, the purpose of the A/B is to optimize the outcome. This is why Google Analytics has exclusively chosen multi-armed bandit for its its A/B test framework.

Post by orasis » Tue, May 28, 2019, 3:38 AM UTC

parksy wrote: post #59

orasis wrote: post #53

Earlier quoted context omitted.

Different demographics, different intent, and different mental context all play a factor.

Not a statistician by any means, but could traffic source be a factor that's evaluated alongside conversion by a bandit algorithm when calculating the chance to show a particular option? Or other factors as well (detected device capabilities, users location, etc?) These could just be weighting factors so instead of a single % chance per option, every time there's a successful interaction the victory is spread across…

Post by orasis » Tue, May 28, 2019, 3:40 AM UTC

taeric wrote: post #62

orasis wrote: post #50

Earlier quoted context omitted.

Sure it’s post-hoc reasoning, but it doesn’t matter because I’m not trying to invalidate a hypothesis. I’m looking for variants that win. When I find one that wins I look at it and try to add more of the same flavor to the product. This process works.

This is literally the logical fallacy. You could get lucky. Maybe you have obvious gains to chase. But bad logical arguments are bad because they never work forever. They are corrupted heuristics that can get you in trouble without critical thinking. Edit: added in forever. Phone dropped some wording I originally had. I think.