Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What works in e-commerce – A meta-analysis of online experiments [pdf]

qubit.com

Post reply on HN
Print view View original

1–10 of 90 posts

Re: What works in e-commerce – A meta-analysis of online experiments [pdf]

#7

Post by sweezyjeezy »

iagovar wrote: post #4

It's good to see an analysis, albeit most of this info was common knowledge, maybe except call to action buttons causing a decrease. I thought it was the opposite. Sometimes factoring all variables when doing testing becomes impossible.

This stuff is less common knowledge than you might expect. When bringing on new clients at Qubit (particularly smaller ones), we still find so many of them obsessing over tiny UI changes expecting it to make an impact. Much of the e-commerce industry has bought into the idea that the cosmetics of a site is the most important thing to get right.

Re: What works in e-commerce – A meta-analysis of online experiments [pdf]

#8

Post by Silhouette »

Thanks for sharing some real data. It's always interesting to see.

I too am surprised to see the conclusions come out as negative as this over such a large overall data set. Just from our own experience, even making quite modest changes to small web sites, it's not that unusual to see the kinds of change that came out with a negative mean in this report actually making a very noticeable positive difference.

I wonder whether this is partly a matter of interpretation and presentation. A lot of the treatments that had a slightly negative mean also had a lot of variance, which suggests that quite often those treatments do work but it's not reliable and requires experimentation to make sure you only keep the genuinely beneficial cases. It seems plausible that there were a few of the "75% improvement in our case study!" kinds of results lost in the long tails, but that what the data is telling us is that those really are outliers and don't happen nearly as often as we might wish.

Re: What works in e-commerce – A meta-analysis of online experiments [pdf]

#9

Post by mrweasel »

sweezyjeezy wrote: post #7
iagovar wrote: post #4

It's good to see an analysis, albeit most of this info was common knowledge, maybe except call to action buttons causing a decrease. I thought it was the opposite. Sometimes factoring all variables when doing testing becomes impossible.

This stuff is less common knowledge than you might expect. When bringing on new clients at Qubit (particularly smaller ones), we still find so many of them obsessing over tiny UI changes expecting it to make an impact. Much of the e-commerce industry has bought into the idea that the cosmetics of a site is the most important thing to get right.

I believe that the main focus of any ecommerce site should be pricing, availability and having the payment and delivery options your customers expects.

E-commerce sites want to "engage" with their customers, but fail to realise that the customers don't want a relationship. They just want whatever product you're selling as quickly and cheaply as possible. Most of you customers will be coming via price comparison sites or Google, so they aren't going to you directly (unless you're Amazon or eBay). For most e-commerce sites the customers aren't going to stay long enough to notice imperfections in the UI.

What you do need is a dead simple checkout (no signup required) and an equally easy return form. Everything else will be used by only a tiny percentage of your customers, often those you don't want to deal with anyway.

For some weird reason making the checkout to simply will result in a large number of purchases cancelled right after placement. In my experience customers are more likely to cancel within 15 minutes of placing the order than at any other time.

Re: What works in e-commerce – A meta-analysis of online experiments [pdf]

#10

Post by sharemywin »

sweezyjeezy wrote: post #7
iagovar wrote: post #4

It's good to see an analysis, albeit most of this info was common knowledge, maybe except call to action buttons causing a decrease. I thought it was the opposite. Sometimes factoring all variables when doing testing becomes impossible.

This stuff is less common knowledge than you might expect. When bringing on new clients at Qubit (particularly smaller ones), we still find so many of them obsessing over tiny UI changes expecting it to make an impact. Much of the e-commerce industry has bought into the idea that the cosmetics of a site is the most important thing to get right.

I've built a couple lead generation sites and some of the things you've determined don't make a difference with e-commerce seem to have a huge impact on people filling out forms on line.
Post reply on HN