Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vertical vs. grid product listings - a surprising AB test

westiseast.co.uk

Post reply on HN
Print view View original

31–40 of 58 posts

Re: Vertical vs. grid product listings - a surprising AB test

#31

Post by dagw »

I'd imagine this depends a lot on what you are displaying. In this case there is no significant visual difference between the products, and you have to read the text to find out what it actually is. Then I can see having the text in a list would be more natural.

In cases where there is enough visual difference between the products so that they can be distinguished between without any text, then I think a grid is probably a better layout since you can compare more products at a time. So while the results are certainly interesting, I don't think I would try to generalize them too far.

Re: Vertical vs. grid product listings - a surprising AB test

#32

Post by westiseast »

marekmroz wrote: post #10
westiseast wrote: post #3

Earlier quoted context omitted.

sorry, yes you're right. The sample size was not huge (@500 people viewed each variant) although the result was statistically significant. I just added a note to the post about this. My aim is to repeat the test at a later date to see if the results repeat themselves.

Correct me if I am wrong, but isn't statistical significance a function of sample size?

If I've understood it correctly, the smaller your sample size, the bigger the difference you need between response rates. So if the difference is 60%, then a sample of 30 people can be statistically significant. But if you're looking for a 5% improvement on something, you'll need to sample a large number of people.

I just used this:

http://www.prconline.com/education/tools/statsignificance/in...

Re: Vertical vs. grid product listings - a surprising AB test

#33

Post by guynamedloren »

scrrr wrote: post #29

Well, Amazon uses the list-view in their search results. Never noticed this but now I assume that if they do it then it's been tested and deemed better.

Not exactly. Amazon uses list view if you do not select a department for your search. As far as I can tell, they use grid view if you do select a department. There must be something to that. I'd love to learn more about this if anybody has done extensive testing here.

Edit: It looks like there may be exceptions to the grid view within departments (see "Amazon Instant Video", "Books", and "Video Games").

Re: Vertical vs. grid product listings - a surprising AB test

#34

Post by VMG »

marekmroz wrote: post #10
westiseast wrote: post #3

Earlier quoted context omitted.

sorry, yes you're right. The sample size was not huge (@500 people viewed each variant) although the result was statistically significant. I just added a note to the post about this. My aim is to repeat the test at a later date to see if the results repeat themselves.

Correct me if I am wrong, but isn't statistical significance a function of sample size?

You are right.

The author should plug the numbers in here: http://www.usereffect.com/split-test-calculator

I'd go as far as to consider the data to be worthless without the confidence value

Re: Vertical vs. grid product listings - a surprising AB test

#35

Post by peng »

If you're going to A/B test something, you need to reduce the variables down to one. For all you know, the supposedly 'pretty' framing and box-shadow you added to the grid layout may actually be what's causing customers to prefer the flat list.

Testing multiple variables in a given trial makes it impossible to discover which change (or combination thereof) actually improved the conversion rate.

Re: Vertical vs. grid product listings - a surprising AB test

#36

Post by westiseast »

peng wrote: post #35

If you're going to A/B test something, you need to reduce the variables down to one. For all you know, the supposedly 'pretty' framing and box-shadow you added to the grid layout may actually be what's causing customers to prefer the flat list. Testing multiple variables in a given trial makes it impossible to discover which change (or combination thereof) actually improved the conversion rate.

Yeah, it's a blunt tool. The next planned test is to 'prettify' the vertical listing and see what happens.

Re: Vertical vs. grid product listings - a surprising AB test

#38

Post by dave1010uk »

peng wrote: post #35

If you're going to A/B test something, you need to reduce the variables down to one. For all you know, the supposedly 'pretty' framing and box-shadow you added to the grid layout may actually be what's causing customers to prefer the flat list. Testing multiple variables in a given trial makes it impossible to discover which change (or combination thereof) actually improved the conversion rate.

The frame and box shadow may also slow the site down. Amazon found that a 100 ms increase in page load time reduces sales by 1%.

Re: Vertical vs. grid product listings - a surprising AB test

#40

Post by ot »

The ranking may have also a role in this: ranking on a grid is usually presented left-to-right top-to-bottom, but that's not the order the eye scans (while in a vertical layout the top-to-bottom order is natural).

This paper from Yahoo Research contains an interesting analysis of the problem:

Optimizing Two-Dimensional Search Results Presentation

http://www.chierichetti.name/papers/optimizing2d.pdf?attredi...

Post reply on HN