Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What works in e-commerce – A meta-analysis of online experiments [pdf]

qubit.com

Post reply on HN
Print view View original

21–30 of 90 posts

Re: What works in e-commerce – A meta-analysis of online experiments [pdf]

#21

Post by sweezyjeezy »

joe-stanton wrote: post #11

This is a really useful article. It's a shame that so much development time is wasted on large numbers of fruitless optimisations just because they are "easy" (eg. tweaking the colour of a CTA). That being said, I'm surprised many of the results are so negative. It would be great to also see the max uplift achieved for each category. A number of retailers I've worked with have been able to beat these uplifts by quite…

Two things - firstly, each of the scores you see in the key findings are just the average. We have also estimated the size of the standard deviation (see table in section 2, or appendix A). So for some treatments, large uplifts are not out of the question.

Maybe more importantly - every A/B test ever run suffers from measurement error, and usually in e-commerce this error is on the scale of the effect you are trying to measure. This means that sometimes you will 'see' massive uplifts, where in actuality most of the size of the effect was due to random noise. This is kind of the curse of e-commerce : most people have enough data to say something (we are 95% sure this test was positive), but most not with any notable precision (we are 95% sure the uplift was between +8% and +9%). Basically all the stats in this analysis is trying to remove this noise, and this is what we got.

Re: What works in e-commerce – A meta-analysis of online experiments [pdf]

#22

Post by geetfun »

castell wrote: post #5
geetfun wrote: post #2

Anyone who hangs out on Facebook group e-commerce forums basically have seen the outrageous claims the authors of this paper alludes to. Nice to see an analysis like this for a change.

can you recommend good groups?

A lot of Shopify groups.

Search up anything Shopify or ecommerce mastermind.

Be aware that a lot of them are set up as groups that simply try to sell you stuff.

The best ecommerce folks are basically the best marketers.

Re: What works in e-commerce – A meta-analysis of online experiments [pdf]

#25

Post by Iv »

tl;dr: The 3 items possibly statistically significant are:

- Saying there are just a handful items left in stock (+2.9% revenue per client) - Saying other people are watching this product (+2.3%) - Time limited offer (+1.5%)

I did not see mention of combining these factors. I doubt the gains are cumulative.

My main takeaway is that most optimizations are not worthy if you have the opportunity to spend your time/money on something else to bring value to the consumer.

Also, I think #1 and #3 are dick moves and #2 needs some good crafting to not be. I doubt the cost in reputation is worth the increase in revenue.

Re: What works in e-commerce – A meta-analysis of online experiments [pdf]

#26

Post by cylinder »

Well, running a rudimentary eBay store you realize these things help pretty quickly, but it's good to have data.

However... Starting a new e-commerce property? Good luck finding traffic in anything profitable. Amazon and other Giants dominate search rankings so I'm not sure how you will find your traffic unless you create a new niche. Maybe you're a thought leader in a hobbyist space, that can work... But you're not going to be succeeding because of these tricks

Re: What works in e-commerce – A meta-analysis of online experiments [pdf]

#27

Post by kawsper »

geetfun wrote: post #2

Anyone who hangs out on Facebook group e-commerce forums basically have seen the outrageous claims the authors of this paper alludes to. Nice to see an analysis like this for a change.

My favorite is: "Our buy now button should be orange, that converts best" without testing it, verifying the results, and just pointing at some blogposts.

Re: What works in e-commerce – A meta-analysis of online experiments [pdf]

#28

Post by watchdogtimer »

Silhouette wrote: post #18
mrweasel wrote: post #9

Earlier quoted context omitted.

I believe that the main focus of any ecommerce site should be pricing, availability and having the payment and delivery options your customers expects. E-commerce sites want to "engage" with their customers, but fail to realise that the customers don't want a relationship. They just want whatever product you're selling as quickly and cheaply as possible. Most of you customers will be coming via price comparison sites…

You seem to be taking a very narrow definition of e-commerce here. What you say seems plausible enough for sites selling low value or commodity products to casual purchasers, but I wouldn't assume the same situation for someone selling luxury products, services, digital subscriptions, etc.

+1. We run a small business that manufactures bicycle trailers which we sell direct to our customers over the internet. The only reason we've been able to succeed has been because we deal one-on-one with our customers.

Re: What works in e-commerce – A meta-analysis of online experiments [pdf]

#29

Post by baxtr »

Iv wrote: post #25

tl;dr: The 3 items possibly statistically significant are: - Saying there are just a handful items left in stock (+2.9% revenue per client) - Saying other people are watching this product (+2.3%) - Time limited offer (+1.5%) I did not see mention of combining these factors. I doubt the gains are cumulative. My main takeaway is that most optimizations are not worthy if you have the opportunity to spend your time/money…

I like your expression "dick moves", summarizes it quite well. John Gruber started calling static mobile navigation bars as "dickbars" [1]. Maybe we should build on that and call these moves accordingly ;-) e.g. "dicktimer"

[1] https://daringfireball.net/2017/06/medium_dickbars

Re: What works in e-commerce – A meta-analysis of online experiments [pdf]

#30

Post by robbiemitchell »

Iv wrote: post #25

tl;dr: The 3 items possibly statistically significant are: - Saying there are just a handful items left in stock (+2.9% revenue per client) - Saying other people are watching this product (+2.3%) - Time limited offer (+1.5%) I did not see mention of combining these factors. I doubt the gains are cumulative. My main takeaway is that most optimizations are not worthy if you have the opportunity to spend your time/money…

> I doubt the gains are cumulative

Why do you doubt that? Aren't they presented as independent results?

> bring value to the consumer

This is vague, but also a separate goal. Perhaps another way of representing customer value is churn reduction. In any case, it could be appropriate to invest in gaining 3% more revenue per additional customer, then use the gains to invest in more customer value.

> #1 and #3 are dick moves and #2 needs some good crafting

Real scarcity exists, and there's value to communicating it to the customer. For example, saying there are two copies of something available, and the rest are backordered for three weeks, can be pretty useful information.

Post reply on HN