Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A/B test of banner ads vs. traffic

gwern.net

Post reply on HN
Print view View original

31–40 of 149 posts

Re: A/B test of banner ads vs. traffic

#31

Post by gwern »

imh wrote: post #23

What do you think the mechanism is on the connection between ads and pageviews? Or the mechanism for the autocorrelation of pageviews? Have you been able connect the dots on something like sharing to sites like HN?

> What do you think the mechanism is on the connection between ads and pageviews?

The mechanism that makes the most sense to me is via influence in social media resharing. This requires affecting only a few people (power users/the '1%') but can plausibly deliver the necessary effect size. The logical next step in the analysis would be to go back into Google Analytics and pull out another set of time-series and try to split between search engine, direct, and referrals; the decrease should be concentrated as search engine > Or the mechanism for the autocorrelation of pageviews?

Well, some of the mechanisms there are obvious just from the referring URLs. Many people don't read HN via the front page, they use a third-party interface or a daily newsletter; so while OP is getting substantial traffic today, it'll also get traffic tomorrow from people catching up with today's 'best of HN' etc. And from HN it gets auto-posted to Twiter, and of course many people will only see the tweets later when they log into Twitter the next morning or evening. Then you have timezones - perhaps someone is asleep now even if they do read the HN frontpage directly. It might get included in blogger link roundups or sites like The Browser weeks or months from now - more autocorrelation. What about search engines? They'll notice, and will update. No one has mentioned any comparable experiments so right now my A/B test appears to be _sui generis_, so people debating the costs and benefits of advertising might link it over the next few months. And so on. Lots of possible ways.

Re: A/B test of banner ads vs. traffic

#32

Post by kzrdude »

_8j50 wrote: post #26
katzgrau wrote: post #3

As someone who has founded and runs an adserver for a living (we deal only with legitimate publications, nothing sketchy), I found this thought provoking. Since I deal mostly in niche/local advertising, I might have a unique perspective. It turns out that the more relatable a banner ad is, the less offended the reader is. That may sound obvious, and it probably is. But the general effect is that users engage with and…

As a user,I am most offended by ads that target me. I would rather see a completely unrelatable ad. It feels like learning new information that way,as opposed to being stalked and preyed upon.

I would love to see more ads that instead target the content of the page. When I'm reading about machine learning, it should be books close to that topic, and other related things. I can't believe little that happens.

Re: A/B test of banner ads vs. traffic

#33

Post by tikhonj »

_8j50 wrote: post #26
katzgrau wrote: post #3

As someone who has founded and runs an adserver for a living (we deal only with legitimate publications, nothing sketchy), I found this thought provoking. Since I deal mostly in niche/local advertising, I might have a unique perspective. It turns out that the more relatable a banner ad is, the less offended the reader is. That may sound obvious, and it probably is. But the general effect is that users engage with and…

As a user,I am most offended by ads that target me. I would rather see a completely unrelatable ad. It feels like learning new information that way,as opposed to being stalked and preyed upon.

The other option is to target the ads to the publication. As long as the publication is on a niche topic or geographically localized, you can choose ads that will likely interest people (local restaurants) without needing to track them. Tracking is only necessary when the ad venue is incredibly broad across topics and geography.

Re: A/B test of banner ads vs. traffic

#34

Post by Ruud-v-A »

I am curious, what constitutes “traffic”, and if it is page loads, why is it the right thing to measure? Users who load the page are already visiting, regardless of whether there is an ad on the page or not. (Or does it include a delay to filter visitors who immediately close the page?) And they did not know in advance whether there was going to be an ad, so their decision to visit the site is not based on the presence of ads. (Unless perhaps if they had a particularly bad experience a previous time — bad enough to remember the domain and not follow the link.) I don’t understand how turning ads on and off every two days can be used to understand long-term effects. There appears to be the assumption that there is a relation between a user seeing an ad or not, and subsequent traffic. There might be a small correlation because of visitors browsing the site, or likeliness to share, but if outside of the spikes, the majority of visitors arrive from an old external link or search engine, and they only read a single page, then how does turning ads on and off every two days measure impact on these users?

Re: A/B test of banner ads vs. traffic

#35

Post by huac »

majani wrote: post #20
katzgrau wrote: post #3

As someone who has founded and runs an adserver for a living (we deal only with legitimate publications, nothing sketchy), I found this thought provoking. Since I deal mostly in niche/local advertising, I might have a unique perspective. It turns out that the more relatable a banner ad is, the less offended the reader is. That may sound obvious, and it probably is. But the general effect is that users engage with and…

This is the biggest problem I have with Adsense. Most of the time it seems like they couldn't find an ad that fits me but their system decided to just throw me a random ad for Forex Trading or whatever. This turns their supposedly sophisticated ad system into a random generator as far as I'm concerned. They could fix this by: 1. Just showing blanks if they can't get something targeted to my preferences 2. Not allowin…

Your concerns are typically addressed by:

1. Reserve pricing - if no ad is 'good enough', e.g. no advertiser is willing to bid above a certain threshold for you, then no ad is shown

2. ePCM pricing - Adsense predicts what the likelihood of clickthrough/conversion is for a given ad and user, and includes that in its auction algorithm. Roughly, eCPM bid = cost per click / probability of click. This addresses the typical problem of high CPC/CPA bid but low engagement probability ads crowding out otherwise good CPM-based ads.

In general responsible advertisers have aligned incentives with viewers. Intrusive or obnoxious (or malicious) advertisements hurt viewers, who are less likely to return.

Re: A/B test of banner ads vs. traffic

#36

Post by eof »

Ruud-v-A wrote: post #34

I am curious, what constitutes “traffic”, and if it is page loads, why is it the right thing to measure? Users who load the page are already visiting, regardless of whether there is an ad on the page or not. (Or does it include a delay to filter visitors who immediately close the page?) And they did not know in advance whether there was going to be an ad, so their decision to visit the site is not based on the presen…

It is probably a combination of performance impact, lower page views per visit, and lower return rate.

Re: A/B test of banner ads vs. traffic

#37

Post by majewsky »

kzrdude wrote: post #32
_8j50 wrote: post #26

Earlier quoted context omitted.

As a user,I am most offended by ads that target me. I would rather see a completely unrelatable ad. It feels like learning new information that way,as opposed to being stalked and preyed upon.

I would love to see more ads that instead target the content of the page. When I'm reading about machine learning, it should be books close to that topic, and other related things. I can't believe little that happens.

Considering the sprawl of sponsored content, maybe because it's cheaper/easier to just buy the author?

Re: A/B test of banner ads vs. traffic

#38

Post by _8j50 »

tikhonj wrote: post #33
_8j50 wrote: post #26

Earlier quoted context omitted.

As a user,I am most offended by ads that target me. I would rather see a completely unrelatable ad. It feels like learning new information that way,as opposed to being stalked and preyed upon.

The other option is to target the ads to the publication . As long as the publication is on a niche topic or geographically localized, you can choose ads that will likely interest people (local restaurants) without needing to track them. Tracking is only necessary when the ad venue is incredibly broad across topics and geography.

I have been wondering why that isn't done more often myself. I mean,it isn't like tv ads didn't work.

Re: A/B test of banner ads vs. traffic

#39

Post by figgis »

HugThem wrote: post #19
gwern wrote: post #13

Earlier quoted context omitted.

It was on almost all of the HTML pages (except the index & Modafinil) up to September. Perhaps the JS doesn't run when loaded from IA or something?

Good question. Even if it does not load, it should still show the adsense code in the html. But looking at some random post, I don't see it: http://web.archive.org/web/20150304235943/http://www.gwern.n... Adsense is usually a script loaded from googlesyndication.com which is not in the source here. Maybe archive.org rewrote the html so much that it is not there anymore?

After a few moments of searching I'm going to assume that whoever specifically scraped and uploaded the page may have removed the adsense javascript. Doesn't look to be a normal archive.org thing.

Re: A/B test of banner ads vs. traffic

#40

Post by emn13 »

huac wrote: post #35
majani wrote: post #20

Earlier quoted context omitted.

This is the biggest problem I have with Adsense. Most of the time it seems like they couldn't find an ad that fits me but their system decided to just throw me a random ad for Forex Trading or whatever. This turns their supposedly sophisticated ad system into a random generator as far as I'm concerned. They could fix this by: 1. Just showing blanks if they can't get something targeted to my preferences 2. Not allowin…

Your concerns are typically addressed by: 1. Reserve pricing - if no ad is 'good enough', e.g. no advertiser is willing to bid above a certain threshold for you, then no ad is shown 2. ePCM pricing - Adsense predicts what the likelihood of clickthrough/conversion is for a given ad and user, and includes that in its auction algorithm. Roughly, eCPM bid = cost per click / probability of click. This addresses the typica…

I think a high reserve price is actually a really neat idea - you'd think that might have the knock-on consequence of reducing the attractiveness of adblockers, and furthermore of reducing viewer fatigue (so that the ads you do show have more impact). If you're smart enough, and take into account the "damage" to your relationship with the viewer when choosing an ad (i.e. aiming primarily for the ad the viewer is most likely to appreciate, and only secondarily other effects such as how likely they are to click and how much the advertiser is willing to pay), you might even be able to make ads the viewer actually wants to see; which in the long run is probably a very good place to be.

I wonder if part of the reason that never happened is simply that ads are in a sense "reserved" layout space that need to be filled consistently and cannot be filled later on (due to layout jumpiness).

Post reply on HN