Live data from Hacker News

Vertical vs. grid product listings - a surprising AB test

westiseast.co.uk

51–58 of 58 posts

Re: Vertical vs. grid product listings - a surprising AB test

#51

The author made one glaring mistake: basing "performance" on click through rates. What really matters on an ecommerce site is revenue. So which format resulted in a higher revenue? You'd be silly to make the assumption that greater click through = greater revenue.

"You'd be silly to make the assumption that greater click through = greater revenue."

Performance is based upon how well the conversion funnel performs. Revenue of course matters, but the way you get that revenue is to get people to go through the conversion funnel. What this A/B test shows, though crudely, is one optimization of one portion of the funnel. When you get more people to click on, say, a product page, you're "widening the funnel" at that particular point.

It's entirely possible to widen the funnel higher up, and still have the same amount of revenue; however, this doesn't mean that it was a useless optimization. Whenever you widen the funnel, you get the potential for higher revenue, especially with further optimizations. That makes for an increase in performance.

Lastly, widening the funnel at any point also means that you are getting increased page views, increased product views, and increased stickiness. That is _always_ a good thing, regardless of whether you increase your revenue or not. It means that people are more likely to interact, and buy from, your site in the future-- users are naturally more likely to go to a site that they know. So, greater click throughs to a product page, though you may not immediately see an effect, will almost always lead to greater revenue.

Re: Vertical vs. grid product listings - a surprising AB test

#52

Earlier quoted context omitted.

Yep, 100% correct. Although in this case, my justification is - the 'distance' between this product listing and an actual purchase is maybe 4 steps total (ie. view product, add to basket, give details, pay), and there are decreasing response percentages at each step. If I tried to measure actual sales with my current tool the test would take much much longer to conclude. Rest assured, I've got AB tests running elsewh…

But this makes your entire article nonsense, do you not get that? People could have been clicking through more because one was better at conveying information than the other, so they didn't need to click through to make a purchasing decision. Maybe the list meant that they ignored the text? What if the end result is you still only sold 20 boxes of tea for each? Your assumption would then be wrong, the grid would be b…

You're sort of touching on two of the important metrics; product views per session, and page views per product view. You want to maximize the former, and minimize the latter. Why would you want to maximize the former? Stickiness. Your customer will be more likely to stay around and purchase, whether it's now, or later, if they see a lot of different products-- even if they aren't finding what they're looking for (which, btw, is impossible to divine from this metric).

"What if because the consumer had to click around more you actually had less sales?"

That's very difficult to measure-- you can't assume that a customer at the browse portion of the funnel is able to find what they're looking for or not, just by looking at the number. You might want to look at the number of add-to-cart actions per product view for a better answer to that question.

Re: Vertical vs. grid product listings - a surprising AB test

#53

Have you A/B tested the retention rate of those users? I'd be interested in seeing if you kept more users with the prettier layout.

No - it was a limitation of the test design, but it strictly measured just click-through performance for one step of the sales funnel. Other (equally important) considerations measuring quality of the interaction weren't measured. There's pros and cons of setting up the test this way, but I think that could be discussed for years :)

Re: Vertical vs. grid product listings - a surprising AB test

#54
post #35

If you're going to A/B test something, you need to reduce the variables down to one. For all you know, the supposedly 'pretty' framing and box-shadow you added to the grid layout may actually be what's causing customers to prefer the flat list. Testing multiple variables in a given trial makes it impossible to discover which change (or combination thereof) actually improved the conversion rate.

That was my first reaction to this test as well. It's not an A/B test and the conclusions are pretty much worthless.

From experience though, an 'uglier' site getting more clicks is actually fairly common. I've seen the results on 100,000's of visitors across a multitude of landers. The rationalization that seems most common is that people click to 'get away' or get to whatever they want.

Re: Vertical vs. grid product listings - a surprising AB test

#55
I think it really depends on how many products you need to display. If you have 200 products in a category, I think you'd want the user to be able to scan very quickly through all of them (via a grid), whereas if there are only a few, you'd want them to consider each product individually (vertical listing).

Re: Vertical vs. grid product listings - a surprising AB test

#56
post #51

The author made one glaring mistake: basing "performance" on click through rates. What really matters on an ecommerce site is revenue. So which format resulted in a higher revenue? You'd be silly to make the assumption that greater click through = greater revenue.

"You'd be silly to make the assumption that greater click through = greater revenue." Performance is based upon how well the conversion funnel performs. Revenue of course matters, but the way you get that revenue is to get people to go through the conversion funnel. What this A/B test shows, though crudely, is one optimization of one portion of the funnel. When you get more people to click on, say, a product page, yo…

A wider funnel isn't always better. The thing that should be optimised is volume multiplied by some measure of "lead quality" (and total money spent is probably a good approximation for a business to use).

For example, would you rather have 1000 people through the funnel with a 1% average likelihood of purchasing (for an expected 10 sales) or 100 people through the funnel with a 50% average likelihood of purchasing (for an expected 50 sales).

It may well be that you can get more people through the funnel but it doesn't make it a good optimisation if the people you are enticing are unlikely to buy anything from you.

Real world example: Apple products. Apple specifically target the top end of consumers (narrow funnel) who are happy to pay a little bit more, whereas other producers go for the mass market (wider funnel) sales at a lower margin.

Re: Vertical vs. grid product listings - a surprising AB test

#57

Earlier quoted context omitted.

There is also a cultural element to consider. Some regions, like Japan, prefer high-information density, cluttered UIs. A prime example is yahoo.com VS yahoo.co.jp White space is an important tool in controlling that density.

I don't think Yahoo Japan is any more dense or cluttered than regular Yahoo. Maybe it's just the unfamiliar characters giving that appearance?

Nope. They're all cluttered.

http://www.rakuten.co.jp/

http://www.bk1.jp/

http://www.goodwill.jp/

http://www.matsuzakaya.co.jp/

https://www.apita.co.jp/

http://www.minimini.co.jp/

http://www.nrt.co.jp/

Re: Vertical vs. grid product listings - a surprising AB test

#58
post #55

I think it really depends on how many products you need to display. If you have 200 products in a category, I think you'd want the user to be able to scan very quickly through all of them (via a grid), whereas if there are only a few, you'd want them to consider each product individually (vertical listing).

That's my feeling too - perhaps it's also a function of how many products you think might be suitable for a person. eg. if I'm looking at iconfinder.com I like 100 to a page, in a grid, so I can scan the icons really fast, little scrolling. I'm going to eliminate 99% of the icons very quickly.

But with teas, it's perhaps not so obvious, so the vertical listing seems to work better to allow people to choose.

Post reply on HN