Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The multi-armed bandit problem (2012)

stevehanov.ca

Post reply on HN
Print view View original

71–80 of 82 posts

Re: The multi-armed bandit problem (2012)

#71

Post by orasis »

everyone wrote: post #69

"hundreds of the brightest minds of modern civilization have been hard at work not curing cancer. Instead, they have been refining techniques for getting you and me to click on banner ads." Just out of curiosity... Have you ever purposefully clicked on an ad on the internet? I honestly dont think I ever have. ps. I mean an outright overt straight up ad, not, for example, some article linked on HN that is a thinly vei…

I clicked on an ad for women’s yoga pants once. The retargeting makes the web a constant stream of delights.

Re: The multi-armed bandit problem (2012)

#72

Post by orasis »

cfarm wrote: post #41

MAB seems to find a local maxima subject to input biases whereas an AB test is aimed to figure out a scientific truth and isolates out all potential biases in the system. I'd be curious to hear where a MAB approach and an AB test did not yield the same results and why that happened.

AB testing doesn’t eliminate time based biases such as novelty of new features, special events, pay day, tax day, etc.

Re: The multi-armed bandit problem (2012)

#73

Post by taeric »

orasis wrote: post #70
taeric wrote: post #62

Earlier quoted context omitted.

This is literally the logical fallacy. You could get lucky. Maybe you have obvious gains to chase. But bad logical arguments are bad because they never work forever. They are corrupted heuristics that can get you in trouble without critical thinking. Edit: added in forever. Phone dropped some wording I originally had. I think.

Call it a genetic algorithm if you like. I’m looking for incremental wins in a world of infinite possibilities, not truth.

Incremental wins can still lead to dead ends. My phrasing was off in my post. I meant to say that the fallacies aren't that the tactics never work. Just that they can stop working without you really realizing it. A heuristics that can lead you down a dead end.

By all means, keep doing it if it is working for you. But don't confuse it as good advice. And stay vigilant.

Re: The multi-armed bandit problem (2012)

#74

Post by WrtCdEvrydy »

orasis wrote: post #71
everyone wrote: post #69

"hundreds of the brightest minds of modern civilization have been hard at work not curing cancer. Instead, they have been refining techniques for getting you and me to click on banner ads." Just out of curiosity... Have you ever purposefully clicked on an ad on the internet? I honestly dont think I ever have. ps. I mean an outright overt straight up ad, not, for example, some article linked on HN that is a thinly vei…

I clicked on an ad for women’s yoga pants once. The retargeting makes the web a constant stream of delights.

Interesting...

Re: The multi-armed bandit problem (2012)

#75

Post by barbarr »

dr_dshiv wrote: post #47

I published this work in CHI on the use of multiarmed bandits in educational games. My biggest takeaway was the importance of choosing the right metric -- because our optimization worked too well. https://www.researchgate.net/publication/301935710_Interface...

This was an interesting read. I played the game, too! Interesting to see how the optimizer converged on such a large ship size, even though you only allowed that case to verify whether the optimizer would successfully reject it.

Re: The multi-armed bandit problem (2012)

#76

Post by JohannesH »

orasis wrote: post #52
JohannesH wrote: post #31

Earlier quoted context omitted.

Very interesting. It seems to me that doing incremental work like this might end up in a local minima/maxima. Do you have any advice on how to avoid pitfalls like that? Are you testing radically different ideas along with your incremental improvements?

From a workflow perspective, MAB is a bit difficult to find radical improvements from. The radical improvements come from fundamental design changes, of which it would be very expensive to create a bunch of radically different variants. MAB is best used where you can generate a bunch of variants cheaply and hope for a 30% gain.

Cool, thanks.

Re: The multi-armed bandit problem (2012)

#77

Post by orasis »

taeric wrote: post #73
orasis wrote: post #70

Earlier quoted context omitted.

Call it a genetic algorithm if you like. I’m looking for incremental wins in a world of infinite possibilities, not truth.

Incremental wins can still lead to dead ends. My phrasing was off in my post. I meant to say that the fallacies aren't that the tactics never work. Just that they can stop working without you really realizing it. A heuristics that can lead you down a dead end. By all means, keep doing it if it is working for you. But don't confuse it as good advice. And stay vigilant.

Products exist in human reality not some science paper. There are no absolute truths, everything dead-ends eventually. It’s like trying to prove that one set of genes is better than another for future survival - an impossible task.

Re: The multi-armed bandit problem (2012)

#78

Post by cfarm »

orasis wrote: post #72
cfarm wrote: post #41

MAB seems to find a local maxima subject to input biases whereas an AB test is aimed to figure out a scientific truth and isolates out all potential biases in the system. I'd be curious to hear where a MAB approach and an AB test did not yield the same results and why that happened.

AB testing doesn’t eliminate time based biases such as novelty of new features, special events, pay day, tax day, etc.

Sure - my point is that you could design the experiment such that it could. It's much more scientific.

Re: The multi-armed bandit problem (2012)

#79

Post by paulddraper »

mobjack wrote: post #66
paulddraper wrote: post #61

Earlier quoted context omitted.

> The purpose of an A/B test isn't to always show the best performing result, it's to perform a _controlled scientific experiment_ with a control group, from which you can learn things. Occasionally there is pure scientific interest. But far more frequently, the purpose of the A/B is to optimize the outcome. This is why Google Analytics has exclusively chosen multi-armed bandit for its its A/B test framework.

It doesn't have to be a pure science interest. If you want to use the results from one test to inform what to test next, then A/B tests are better optimizing for truth. Most website changes dont make a significant difference in conversion. If you use MAB, then you dont know if the winner is really better or the result of random variation.

> If you use MAB, then you dont know if the winner is really better or the result of random variation.

No, you know the outcome of MAB is the best.

If it's clearly the best, it converges quickly. If it's less clearly the best, it converges slowly. Either way though, you haven't lost any more conversions that necessary to find that out, or needed to put in non-mathematically fail safes.

Re: The multi-armed bandit problem (2012)

#80

Post by taeric »

orasis wrote: post #77
taeric wrote: post #73

Earlier quoted context omitted.

Incremental wins can still lead to dead ends. My phrasing was off in my post. I meant to say that the fallacies aren't that the tactics never work. Just that they can stop working without you really realizing it. A heuristics that can lead you down a dead end. By all means, keep doing it if it is working for you. But don't confuse it as good advice. And stay vigilant.

Products exist in human reality not some science paper. There are no absolute truths, everything dead-ends eventually. It’s like trying to prove that one set of genes is better than another for future survival - an impossible task.

This belies a belief that science doesn't reflect the real world. It absolutely does.

Again, it may be working in your case. Argument to authority can go a long way. Even ad hom attacks often exist due to a "smell" of the person speaking. It is not, however, logically sound and can easily lead to unsupportable positions.

So, take care. And realize that a lot of the damage of poor practices may be tangential. For example, a belief that the real world can not be described by science.

Post reply on HN