The Surprising Power of Online Experiments (2017)
1–10 of 27 posts
Re: The Surprising Power of Online Experiments (2017)
#2Nice article but visual examples would really help.
Someone could change the CSS of the ads, for example to flash periodically, and that would be the reason for the increased clicks.
Re: The Surprising Power of Online Experiments (2017)
#3It feels click-baity to reference a $50M revenue improvement without specifying the revenue before the split-tests. Is $50M a 1% improvement? 100%?
Re: The Surprising Power of Online Experiments (2017)
#4[deleted]
Re: The Surprising Power of Online Experiments (2017)
#5Anybody have a recommendation for an A/B testing service?
We've talked to Optimizely but their pricing was going to come in at the same ballpark as our AWS spend (into the six-figure range), which seems absurd. They charge based on monthly users, but a lot of our traffic consists of organic search bounces.
For now we just want to run ~5 experiments per month, want to record events server-side so we can be sure not to lose any, and are wary about implementing it ourselves since there are so many ways to screw it up without realizing it.
Re: The Surprising Power of Online Experiments (2017)
#6Anybody have a recommendation for an A/B testing service? We've talked to Optimizely but their pricing was going to come in at the same ballpark as our AWS spend (into the six-figure range), which seems absurd. They charge based on monthly users, but a lot of our traffic consists of organic search bounces. For now we just want to run ~5 experiments per month, want to record events server-side so we can be sure not to…
You could try the free version of Google Optimize if you don't mind using a Google product.
Re: The Surprising Power of Online Experiments (2017)
#7Anybody have a recommendation for an A/B testing service? We've talked to Optimizely but their pricing was going to come in at the same ballpark as our AWS spend (into the six-figure range), which seems absurd. They charge based on monthly users, but a lot of our traffic consists of organic search bounces. For now we just want to run ~5 experiments per month, want to record events server-side so we can be sure not to…
You could try the free version of Google Optimize if you don't mind using a Google product.
We use Hansel
Re: The Surprising Power of Online Experiments (2017)
#8Good in-depth overview but unfortunately doesn’t get into bigger issues of potential ethical concerns of this sort of narrow maximization. At a certain point you end up becoming Facebook or YouTube and vastly amplifying toxic and dangerous content because it generates more comments or more time spent on the site. And even if you believe in pure amoral capitalism, blindly following what your algorithms tell you to do will eventually lead you into a trap and the backlash won’t necessarily be pretty.
Re: The Surprising Power of Online Experiments (2017)
#9It feels click-baity to reference a $50M revenue improvement without specifying the revenue before the split-tests. Is $50M a 1% improvement? 100%?
Clickbait scores well in A/B testing, so I'd expect an article about A/B testing to have a clickbait title.
Re: The Surprising Power of Online Experiments (2017)
#10It feels click-baity to reference a $50M revenue improvement without specifying the revenue before the split-tests. Is $50M a 1% improvement? 100%?
And volume. A/B testing a website/service with 10 visitors per day doesn't make sense at all.