Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A/B test of banner ads vs. traffic

gwern.net

Post reply on HN
Print view View original

11–20 of 149 posts

Re: A/B test of banner ads vs. traffic

#11

Post by gwern »

HugThem wrote: post #8

He seems to not show how the ads looked like on his page. He seems to only attribute it to load time. But maybe the ads were visually annoying? Why no screenshots?

I have adblock so I don't usually see them and didn't make a point of screenshotting them. When I did (usually checking in Chromium that some CSS change was working), they were fairly ordinary AdSense ads, IMO. Like on DNB FAQ, they would usually be Lumosity or college ads. They weren't flashing or visually offensive, so I never saw them as a problem.

Re: A/B test of banner ads vs. traffic

#12

Post by HugThem »

gwern wrote: post #11
HugThem wrote: post #8

He seems to not show how the ads looked like on his page. He seems to only attribute it to load time. But maybe the ads were visually annoying? Why no screenshots?

I have adblock so I don't usually see them and didn't make a point of screenshotting them. When I did (usually checking in Chromium that some CSS change was working), they were fairly ordinary AdSense ads, IMO. Like on DNB FAQ, they would usually be Lumosity or college ads. They weren't flashing or visually offensive, so I never saw them as a problem.

I looked at some of your pages on archive.org but saw no ads either. Can you give me a hint how to find the pages with the ads?

Re: A/B test of banner ads vs. traffic

#13

Post by gwern »

HugThem wrote: post #12
gwern wrote: post #11

Earlier quoted context omitted.

I have adblock so I don't usually see them and didn't make a point of screenshotting them. When I did (usually checking in Chromium that some CSS change was working), they were fairly ordinary AdSense ads, IMO. Like on DNB FAQ, they would usually be Lumosity or college ads. They weren't flashing or visually offensive, so I never saw them as a problem.

I looked at some of your pages on archive.org but saw no ads either. Can you give me a hint how to find the pages with the ads?

It was on almost all of the HTML pages (except the index & Modafinil) up to September. Perhaps the JS doesn't run when loaded from IA or something?

Re: A/B test of banner ads vs. traffic

#14

Post by hanbura »

katzgrau wrote: post #3

As someone who has founded and runs an adserver for a living (we deal only with legitimate publications, nothing sketchy), I found this thought provoking. Since I deal mostly in niche/local advertising, I might have a unique perspective. It turns out that the more relatable a banner ad is, the less offended the reader is. That may sound obvious, and it probably is. But the general effect is that users engage with and…

I think a most of the problems with ads are just caused by how Google and other big players run online advertising.

Somewhere along the way "let's add an image of some company our readers might be interested in" turned into "let's serve a huge Javascript that perceivably slows the browser in order to serve an ad from the highest bidder who might or might not run a criminal enterprise. Also make sure the ads show off the vast amounts of data we collect, we have to appear as creepy as possible".

Small ad networks or sites that organize their adverts themselves sometimes have great ads. But I am not sure if the reputation of online ads isn't already damaged beyond repair.

Add to that the small revenue from ads (the author makes $30/month from ads and $700/month from patreon)

Re: A/B test of banner ads vs. traffic

#15

Post by AJ007 »

katzgrau wrote: post #3

As someone who has founded and runs an adserver for a living (we deal only with legitimate publications, nothing sketchy), I found this thought provoking. Since I deal mostly in niche/local advertising, I might have a unique perspective. It turns out that the more relatable a banner ad is, the less offended the reader is. That may sound obvious, and it probably is. But the general effect is that users engage with and…

The other side to this is users see the ads, relate to the ads, click the ads, and thus leave your site. Your pageviews will drop.

Whether or not one agrees with the outcome, if you have a large site you definitely should be doing independent testing. Google's ad reps, much less reps from other networks, will tell you to do things that harm your site.

Re: A/B test of banner ads vs. traffic

#16

Post by soared »

I honestly couldn't follow most of the math but found this really interesting. I used to do UX analyses at a marketing agency and would constantly link clients to [1]. Page speed is huge, especially on desktop where its easier to navigate away. OP mentions this, but I imagine the effect is massive on a site where users are actively researching a topic and have tens of tabs open and so can easily switch to a similar website.

I do think this test could have been done without banner ads involved. Looking at page load speed per user and how that effects traffic would likely have the same results.

From my link below:

> A 1 second delay in page response can result in a 7% reduction in conversions.

[1] https://blog.kissmetrics.com/loading-time/

Re: A/B test of banner ads vs. traffic

#17

Post by username223 »

> A decision analysis of revenue vs readers yields an acceptable damage of ~3% total traffic loss. Power analysis of historical ... traffic data demonstrates that the high autocorrelation yields very low statistical power with standard tests & regressions but acceptable power with ARIMA models.

Was all that math worth your time, or would you have made more money selling tamales on the corner?

Re: A/B test of banner ads vs. traffic

#18

Post by soared »

I'm curious if any ad networks or SSPs are differentiating based on load times. I wouldn't think so because big sites that drive lots of impressions tend to load slower and aren't as concerned with page speed as someone like OP.

I'd also be curious to see his user to patreon donator conversion rate, and if removed ads and 'gaining' 10% of traffic would be worth it from a revenue stand point.

Re: A/B test of banner ads vs. traffic

#19

Post by HugThem »

gwern wrote: post #13
HugThem wrote: post #12

Earlier quoted context omitted.

I looked at some of your pages on archive.org but saw no ads either. Can you give me a hint how to find the pages with the ads?

It was on almost all of the HTML pages (except the index & Modafinil) up to September. Perhaps the JS doesn't run when loaded from IA or something?

Good question. Even if it does not load, it should still show the adsense code in the html. But looking at some random post, I don't see it:

http://web.archive.org/web/20150304235943/http://www.gwern.n...

Adsense is usually a script loaded from googlesyndication.com which is not in the source here.

Maybe archive.org rewrote the html so much that it is not there anymore?

Re: A/B test of banner ads vs. traffic

#20

Post by majani »

katzgrau wrote: post #3

As someone who has founded and runs an adserver for a living (we deal only with legitimate publications, nothing sketchy), I found this thought provoking. Since I deal mostly in niche/local advertising, I might have a unique perspective. It turns out that the more relatable a banner ad is, the less offended the reader is. That may sound obvious, and it probably is. But the general effect is that users engage with and…

This is the biggest problem I have with Adsense. Most of the time it seems like they couldn't find an ad that fits me but their system decided to just throw me a random ad for Forex Trading or whatever. This turns their supposedly sophisticated ad system into a random generator as far as I'm concerned.

They could fix this by:

1. Just showing blanks if they can't get something targeted to my preferences

2. Not allowing advertisers to be too general in their targeting if they are doing performance campaigns(CPC and CPA). Only allow CPM advertisers to target broad categories.

Post reply on HN